Not exactly agents ignoring instructions, but I'm wondering if (selective) inclusion of LLM responses in a RAG stack might be suitable as a sort of long-term memory for "accepted" LLM contributions to code for example. This way, unwanted novel or alternate solutions to repeated patterns might be avoided?
Opus 4.6 usually doesn't disappoint .. No double negative auth checks or race conditions to report on, but I can say that introducing new functionality and patterns mostly requires a few cycles before the "repeatable pattern" is cleanly documented in the spec. When bugs do come up, the agent is quite good at finding the root cause and implementing a fix.
I recently shipped a "vibe-coded" project. You raise a good point: I hadn't considered the confidence gap. If it is true that LLM generated code produces more vulnerabilities in addition to there being more code overall, all while at the same time the developer feels better about their results, then that is concerning.
This is how I go about ensuring there is little to no chaos (your mileage may vary based on project size and characteristics):
- Plan your project manually, do not outsource thinking to the LLM. This includes being intentional about architecture, tech-stack, dependencies, etc..
- I have planning, orchestrating, coding, and reviewing agents. These should be self-explanatory, but there's a catch: the workflow is automated. OpenCode allows you to define "subagents" which can be called by "primary" agents. I will write a detailed Gitlab issue that my planning agent can fetch and read. It will create a detailed resolution plan that I can point the orchestration agent to. The orchestrator then delegates implementation to one or more coding agents simultaneously. Results are in turn delegated to reviewer agents. If the reviewer agents don't complain, then the results are ready for human review in an MR.
- Changes that pass all review are documented in the project spec. E.g., if new modules are added that require an auth guard pattern implementation that is already documented in the spec, they will be listed as relevant sites for that auth guard pattern, etc..
I feel like the LLM agents have been more thorough and consistent than I could have been without them. This goes for refactors too: Since the entire project is essentially mapped out in the spec.md file(s), it's hard for the agent to miss a relevant site in the code. Human review is key. Don't merge code you don't understand.
Do you have domain knowledge for each category in-house? How did you put together so many specific calculators while maintaining correctness in each calculator's respective domain?