It's a compelling parallel, but I think we need to be careful not to confuse metaphor with mechanism. Gödel's theorem shows that certain truths can't be proven within a formal system. With LLMs, the issue isn't provability. It's that there's no real model of truth in the first place, only prediction based on patterns.
The Gödel analogy is a bit much. These agents aren't formal systems; their failures come from closed design, not logical incompleteness. The real issue is self-referential loops.
What’s missing is external grounding. Even simple things like retrieving examples from real codebases, validating against actual APIs, or injecting adversarial test cases can break the illusion. Until then, most of these agents are just confidently talking to themselves.