Ask HN: What do you use for normative specs to drive AI agents?
13 comments
Agents, skills and rules are just text files with instructions to the LLMs on how you like your output. At this point I don't think it really matters what you call anything or where you put them in your system, as long as you tell the agent/clanker of your choice where these files are located.
The key point is you are breaking up the context of what you are building into smaller areas of focus. The smaller contexts allow agents to do work with only the context of what their tasks are, so you have less of a chance of hallucinations and they each get a larger context window than they would if you had all of your context written out into a single text file.
This is why a lot of folks call them junior engineers. You wouldn't ask a junior engineer to "act as a senior expert JS developer and build a front end for a SaaS project management app" and expect them to do it with good output. You need to define specific things you want done and how you want them done - "use react, call APIs for data, APIs should be RESTful, etc." You can add personas to them if you like and personalities but thats more for entertainment.
The key point is you are breaking up the context of what you are building into smaller areas of focus. The smaller contexts allow agents to do work with only the context of what their tasks are, so you have less of a chance of hallucinations and they each get a larger context window than they would if you had all of your context written out into a single text file.
This is why a lot of folks call them junior engineers. You wouldn't ask a junior engineer to "act as a senior expert JS developer and build a front end for a SaaS project management app" and expect them to do it with good output. You need to define specific things you want done and how you want them done - "use react, call APIs for data, APIs should be RESTful, etc." You can add personas to them if you like and personalities but thats more for entertainment.
> text files with instructions to the LLMs
this is true for both mardown and normative specs
what's meaningfully different is that normative specs is something you can deterministically check for conformance with, use generative tests, etc and feed that information back into the agent loop
letting the same LLM generate both implementation and tests from markdown prompt is a) completely meaningless, because once the agent is done and test suite is green you still have no idea if the right thing was implemented and if the right thing is being tested; b) useless waste of tokens, because why would you spend them on probabilistic generation of tests that likely contain hallucinations if you could instead deterministically generate them from normative spec using external tool
this is true for both mardown and normative specs
what's meaningfully different is that normative specs is something you can deterministically check for conformance with, use generative tests, etc and feed that information back into the agent loop
letting the same LLM generate both implementation and tests from markdown prompt is a) completely meaningless, because once the agent is done and test suite is green you still have no idea if the right thing was implemented and if the right thing is being tested; b) useless waste of tokens, because why would you spend them on probabilistic generation of tests that likely contain hallucinations if you could instead deterministically generate them from normative spec using external tool
I believe the ultimate goal for any software product is to lower the barrier to entry. It used to take months of study just to get started with coding, and at least half a year to a year to build a functional website. Now, you can practically "talk" things into existence—even if it takes a few rounds of prompting. Because of this, I don't think there will be a "standard" way to develop in the future. Code is ultimately a tool to solve human needs at the lowest possible cost, so naturally, the cost of writing that code should be as low as it can get.
IMO, there is no such normative specs. Just tell your agent what you want and teach it until it gives exactly what you want.
IMO, there is no such normative specs. Just tell your agent what you want and teach it until it gives exactly what you want.
this will work for your hobby projects, but not for larger codebases or constellations of systems where agent output grows faster than your ability to understand what is going on.
if it is your belief that understanding what is going on is not necessary nor important - agree to disagree.
if it is your belief that understanding what is going on is not necessary nor important - agree to disagree.
I use BMAD, and I don't care how the markdown files look like. I don't read them. I ask agents to create and read the files. One agent create the technical specification in markdown file, and a dev agent reads it and process it. ;)
If I need something to distribute or read myself, I ask for a pdf copy :)
If I need something to distribute or read myself, I ask for a pdf copy :)
For me, the spec format matters way less than the routing. You can have perfect definitions, but if your agents don't know their boundaries or who handles what, you get chaos. I now think in terms of an "org chart" for agents.
1. "agentic" development is here to stay, whether we like it or not
2. markdown prompts are a terrible way of driving such development, no matter how much structure and hierarchy you're trying to impose (whether chaining or fan-out into personas, etc)
3. jigsaw-puzzle approach is less terrible - declare the "boundary" of any component or subsystem in a normative language of your choice and let the agent fill the insides
it's fine if you think those convictions are incorrect, but i'm more interested in feedback from folks that more-or-less agree, with regard to the last point: what do YOU use for normative language to describe component boundary, function and cross-component interactions?
i've had mild success with openapi, but maybe there are better alternatives?