Well, o3-mini-high just successfully found the root cause of a seg fault that o1 missed: mistakenly using _mm512_store_si512 for an unaligned store that should have been _mm512_storeu_si512.
I think that "Good Forth programmers arrange things so that they flow on the stack" has analogs in other languages. For example, arranging things in J so that short tacit expressions naturally provide the functions you need.
shows Sonnet 3.5 using the Google web UI in an automated fashion. Do Google's terms really permit this? Will Google permit this when it is happening at scale?
These strategies seem immediately practical. If you want to go beyond zero-shot for LLM coding, you may not need a complicated agent architecture - just start with escalation, retry, and warming.
APPS has 3 subsets by difficulty level: introductory, interview, and competition. It isn't clear which subset Claude 3 was benchmarked on. Even if it is just "introductory" it is still pretty good, but it would be good to know.