Why the Spec-to-Code Gap Cannot Be Closed?(causalitylimited.com)
causalitylimited.com
Why the Spec-to-Code Gap Cannot Be Closed?
https://www.causalitylimited.com/p/why-the-spec-to-code-gap-cannot-be
2 comments
Hey, have enjoyed reading the article. Not sure I agree with the conclusion though. "But the LLM has no basis for choosing between topologies. ... The developer’s job is not just translating specs into code. Any competent developer can do that. A capable LLM can do that." I believe it can. We are just talking about different level of approximation -- nothing that can't be solved with enough context. And even this “higher-level context” can be inferred from modern training approaches — not by focusing on “what” or “why,” but by inferring how components relate and interact, regardless of purpose.
Thanks for reading! The claim isn't that LLMs can't pick a topology, they obviously do, every time they generate code. The claim is that the spec doesn't determine which topology is correct, because multiple valid topologies satisfy the same behavioral spec.
"Enough context" is doing a lot of work in your argument. What is that context? At some point it includes architectural intent, performance constraints, team conventions, future extensibility assumptions ... i.e., exactly the stuff that isn't in the spec and that constitutes the engineering judgment we're discussing. If you force fold all of that into "context," then sure, but now you've just moved the goalposts from spec to "spec plus everything else the developer knows," which is then like a 1:1 map from spec to code.
On inferring how components relate from training data: that gives you statistical priors over topologies which is a good to have sure, but it is not same a justified topology selection for a specific system.
"Enough context" is doing a lot of work in your argument. What is that context? At some point it includes architectural intent, performance constraints, team conventions, future extensibility assumptions ... i.e., exactly the stuff that isn't in the spec and that constitutes the engineering judgment we're discussing. If you force fold all of that into "context," then sure, but now you've just moved the goalposts from spec to "spec plus everything else the developer knows," which is then like a 1:1 map from spec to code.
On inferring how components relate from training data: that gives you statistical priors over topologies which is a good to have sure, but it is not same a justified topology selection for a specific system.