Matrix Orthogonalization Improves Memory in Recurrent Modelsayushtambde.com85 points·by at2005·vor 13 Tagen·32 comments
Tree Search Distillation for Language Models Using PPOayushtambde.com87 points·by at2005·vor 4 Monaten·10 comments
at2005·vor 4 Monaten·discussI didn't compare with the harness (focused on distillation) but the original ToT paper has a section on it: https://arxiv.org/abs/2305.10601
at2005·vor 4 Monaten·discussAh, I meant that MCTS uses more inference-time compute (over GRPO) to produce a training sample