Matrix Orthogonalization Improves Memory in Recurrent Modelsayushtambde.com85 points·by at2005·قبل 13 يومًا·32 comments
Tree Search Distillation for Language Models Using PPOayushtambde.com87 points·by at2005·قبل 4 أشهر·10 comments
at2005·قبل 4 أشهر·discussI didn't compare with the harness (focused on distillation) but the original ToT paper has a section on it: https://arxiv.org/abs/2305.10601
at2005·قبل 4 أشهر·discussAh, I meant that MCTS uses more inference-time compute (over GRPO) to produce a training sample