I imagine one could one-shot a basic app and then feed feature requests one by one, sounds like an obvious way to benchmark architecture/maintainability
even basic stuff like coding agents removes a major bottleneck -- when finding relevant literature and turning an idea into code takes as much time as writing it down in your notes, researchers from top tier labs can test every single hypothesis they can come up with
> hilarious
"oneshot a PoC -> oneshot a production ready game" improvement sounds like something that will eventually be done in a 1kloc diff in the RL pipeline
IMHO: "genuinely good" might never happen though because of all the non-coding stuff -- you can't really benchmaxx when the target is defined as "pleasant to humans"
if you can't tell if slop is correct, how do you know your code is correct? starting with a mental model and then writing the code yourself surely makes it feel safer, but it doesn't mean it is
besides, it doesn't even have to be about writing code; finding a bug is more time consuming than fixing it, so you could at least limit yourself to that