Recently, my experience is that the hardest part of writing Enterprise document by AI is not how to generate a word or excel, but to generate a office document that is accountable.
First draft generation is just a small part of the whole wore,more time consuming work is validation: whether citation , number,format, or semantic assume is right.
So i think enterprise office AI suite may need 2 layers: First is document editing, and second is revision / attribution / validation, or an unaccountable document is not applicable for real enterprise usage.
I agree to split the "Brain" and "Hand" of an agent,many deficits of agent workflows is not from what LLM don't know what to do, but from LLM getting too much authority to edit the world it belongs to and break the whole process.
Beside from particular project, I would like to talk about my rule of thumb in agent usage, which is to separate “smart” from “authorized.”
That is, to build a state wrapper and separate agent from making runtime decision, a dangerous design is letting the same agent both decide and record the state of the system. If it hallucinates, you can hardly find the error.
You are right, my solution is to implement real world software engineering process, something closer to a real Software engineering team.I split multi agents into one repo and gives them different tasks"requirements check, product/spec, architecture, coding, review, tests/evals, and overall management.
Scope adjudication is extremely important in vibe coding, or agent can easily break your whole system with not applicable features.
I don't mean never check it by myself, but I want to discuss about a methodology to ensure AI generated text has the same auditability and tracebility like human wrote text.
Yes, i agree that quantitative part， like dates, numbers, amounts should be extracted and let the LLM to output original numbers and computation steps. That's not hard for a briefing harness.
However, my most confusion part is qualitative side, like market insight from a news,policy change and interpretation, and industry NEWS interpretation, they are not straight math but they need tracebility. Do you have any idea to solve those judgement claim?
Oh no I am not a attorney, but I need to analyze the market, 20-F and 10-K recurringly and make strategy suggestions, and so yes I use LLM to do business brief.
I am sorry that I did not explain my question explicitly. I write recurring weekly briefings for internal use, such as market insights and industry news.
I’m not trying to make AI-generated text “believable”. I’m asking almost the opposite question: when an AI generated text is fluent enough to hide mistakes, how do human check how to systematically check numbers, dates, cites and judgements?