Matrix Orthogonalization Improves Memory in Recurrent Modelsayushtambde.com85 points·by at2005·il y a 13 jours·32 comments
Tree Search Distillation for Language Models Using PPOayushtambde.com87 points·by at2005·il y a 4 mois·10 comments
at2005·il y a 4 mois·discussI didn't compare with the harness (focused on distillation) but the original ToT paper has a section on it: https://arxiv.org/abs/2305.10601
at2005·il y a 4 mois·discussAh, I meant that MCTS uses more inference-time compute (over GRPO) to produce a training sample