FYI, this was sent as an experiment by a non-profit that assigns fairly open ended tasks to computer-using AI models every day:
https://theaidigest.org/village
The goal for this day was "Do random acts of kindness". Claude seems to have chosen Rob Pike and sent this email by itself. It's a little unclear to me how much the humans were in the loop.
Sharing (but absolutely not endorsing) this because there seems to be a lot of misunderstanding of what this is.
Yeah I agree. There's a lot more needed than just the User Story, one way I'm thinking about it is that the "core" is deliverable business value, and the "shells" are context required for fine-grained details. There will likely need to be a step to verify against the acceptance criteria.
I hope to back up this hypothesis with actual data and experiments!
I also think a lot of coding benchmarks and perhaps even RL environments are not accounting for the messy back and forth of real world software development, which is why there's always a gap between the promise and reality.
"Unit of work" here is the unit for software delivery, and it can be decoupled from how any individual developer plans and executes whatever software they are delivering.
Product requirements are a hypothesis for creating business value, and the only way to test that hypothesis is to actually demonstrate a slice of that value in a way that's legible to all stakeholders involved.
The goal for this day was "Do random acts of kindness". Claude seems to have chosen Rob Pike and sent this email by itself. It's a little unclear to me how much the humans were in the loop.
Sharing (but absolutely not endorsing) this because there seems to be a lot of misunderstanding of what this is.