I am very much skeptical on the joint optimization of the harness and the model weights, I mean we can argue that claude code itself was a joint optimization of the claude, they released the first claude model and then the claude code harness then improved claudes performance on the harness then improved the harness after the model improved, but this was a process managed by humans all the time, it would be interesting to see claude improving it's own weights or harness in the future (or it might already be happening at anthropic right now)
I just didn't find the SIA paper to be testing this out rigorously or either provide sufficient evidence that it works
Harness Engineering for Self-Improvement · HackerTrans
I just didn't find the SIA paper to be testing this out rigorously or either provide sufficient evidence that it works