Comparing manual vs. AI requirements gathering: 2 sentences vs. 127-point spec
2 comments
The AI approach smells of waterfall and that has been deemed a bad approach to software projects. Not only because of the time cost (which some would argue it can be reduced by agents), but because the only real product is the one in use.
Every product solve some needs, and there are risks associated to the implementations of features. Trying to map every risk is not productive as the feature is not yet proven. Experience is needed to map the path ahead, but only if one has chosen to walk it.
So writing specs is not hated. It’s just that they’re not that different than a SF novels, as their foundation are just hypotheses. If you have a mountain to climb, you do it step by step, not by tracing paths with maps drawn from reports narrating hearsay.
So the most productive is to have an overall roadmap, plan the next months a bit according to your resources, and then fully focus on the next few tasks. Do periodic reviews to assess your progress.
Every product solve some needs, and there are risks associated to the implementations of features. Trying to map every risk is not productive as the feature is not yet proven. Experience is needed to map the path ahead, but only if one has chosen to walk it.
So writing specs is not hated. It’s just that they’re not that different than a SF novels, as their foundation are just hypotheses. If you have a mountain to climb, you do it step by step, not by tracing paths with maps drawn from reports narrating hearsay.
So the most productive is to have an overall roadmap, plan the next months a bit according to your resources, and then fully focus on the next few tasks. Do periodic reviews to assess your progress.
Thanks for posting this, this is useful research.
The results were uncomfortable. The human produced a polite paragraph summarizing the "happy path." The AI produced a 127-point technical specification that highlighted every edge case, security flaw, and missing feature we usually forget until Week 8.
Here is the breakdown of the experiment and why I think "scope creep" is mostly just discovery failure.
The Problem: The "Assumption Blind Spot"
We’ve all lived through the "Week 8 Crisis." You’re 75% through a 12-week build, and suddenly the client asks, "Where is the admin panel to manage users?" The dev team assumed it was out of scope; the client assumed it was implied because "all apps have logins."
Humans have high context. When we hear "dashboard," we assume standard auth, standard errors, and standard scale. We don't write it down because it feels pedantic.
AI has zero context. It doesn't know that "auth" is implied. It doesn't know that we don't care about rate limiting for a prototype. So it asks.
The Experiment
We fed the same input to a senior human analyst and an LLM workflow acting as a technical interrogator.
Input: "We need a dashboard to track team productivity. It should pull data from Jira and GitHub and show us who is blocking who."
Path A: Human Analyst Output: ~5 bullet points. Focused on the UI and the "business value." Assumed: Standard Jira/GitHub APIs, single tenant, standard security. Result: A clean, readable, but technically hollow summary.
Path B: AI Interrogator Output: 127 distinct technical requirements. Focused on: Failure states, data governance, and edge cases. Result: A massive, boring, but exhaustive document.
The Results
The volume difference (5 vs 127) is striking, but the content difference is what matters. The AI explicitly defined requirements that the human completely "blind spotted":
- Granular RBAC: "What happens if a junior dev tries to delete a repo link?" - API Rate Limits: "How do we handle 429 errors from GitHub during a sync?" - Data Retention: "Do we store the Jira tickets indefinitely? Is there a purge policy?" - Empty States: "What does the dashboard look like for a new user with 0 tickets?"
The human spec implied these were "implementation details." The AI treated them as requirements. In my experience, treating RBAC as an implementation detail is exactly why projects go over budget.
Trade-offs and Limitations
To be fair, reading a 127-point spec is miserable. There is a serious signal-to-noise problem here.
- Bloat: The AI can be overly rigid. It suggested microservices architecture for what should be a monolith. It hallucinated complexity where none existed. - Paralysis: Handing a developer a 127-point list for a prototype is a great way to kill morale. - Filtering: You still need a human to look at the list and say, "We don't need multi-tenancy yet, delete points 45-60."
However, I'd rather delete 20 unnecessary points at the start of a project than discover 20 missing requirements two weeks before launch.
Discussion
This experiment made me realize that our hatred of writing specs—and our reliance on "implied" context—is a major source of technical debt. The AI is useful not because it's smart, but because it's pedantic enough to ask the questions we think are too obvious to ask.
I’m curious how others handle this "implied requirements" problem:
1. Do you have a checklist for things like RBAC/Auth/Rate Limits that you reuse? 2. Is a 100+ point spec actually helpful, or does it just front-load the arguments? 3. How do you filter the "AI noise" from the critical missing specs?
If anyone wants to see the specific prompts we used to trigger this "interrogator" mode, happy to share in the comments.