1. Does the /verification/assess-risk endpoint capture what you'd actually need in a CI/CD gate? What's missing from the request/response schema?
2. The Knowledge Graph as the source of business rules, does this model make sense for your stack, or is there a simpler primitive we should define first?
3. MCP compatibility, are there gaps in how OQP maps to MCP's tool/resource model?
if that alone give you 100% coverage, 0 critical bugs leaking into production, that is sufficient for the size of app you may have, however in an enterprise setting where thousands of B2B customers with nuances in usage and customizations and 100s of permutation and combinations, that alone may not be enuf. actually with agentic code genration this problem will be exacerbated. so in my case I can tell having some linkage of production incidents and production logs back to tets plan and specs is essential to ensure zero critical bugs leak into production. these production incidents are gold mine for actual user workflows and data. A typical tester can only just anticipate how the real world user will use their app and test these scenario in a limited way, but production workflows and data if translated into test cases can enrich the existing testng suite and give you quantifiable end to end test coverage otherwise its kind of hope rather than a plan to have rock solid release.
And the other thing is tacit or tribal knowledge. Ai system is good when data is structured and available. Not so much when data is scattered and largely the connect the dots information is in dev or testers head.
My recipe is memory + context combine with seamless ui to capture dev / tester mindset will make any ai system customizable. It doesn’t have to be LLM system, it can 90% rag or some kind of graph tag and 10% LLM usage. That will create a moat easily defensible otherwise a new LLM upgrade wipe out all the moat you might have.
i agree, but i wwant to add that perhaps just specs might not give you full testing coverage, have to add other artifacts too, like prod logss and incidents and using some layer of ontology + KG to produce meaningful data connectins and understanding. vector db alone will only give semantic search and grossly incompetent to connect data artifacts. for example for vector db, word apple and company apple might both be same without outlininig the context.
I tried claude code, and it did write some good quality e2e tests but my biggest worry was the full coverage. Its really difficult to quantify e2e test coverage the way developers do unit test coverage. its really impossible. specs is just one artifact just like code is just one of many artifacts that full system wide e2e coverage needs. addng production logs + producton incidents which I tried also give me some sense of full e2e coverage. if you are using claude code for dev and testing both, its like having cake and eat it too. If claude for whatever reason misrepresent or misinterpret a requirement, that will percolate in code and testing as well. having a 3rd party testing tool is appropiate with allthe data flowing in it like specs, legacy tests, prod incidents, code and then perhaps we can expect full unbiased test coevrage. I am not talking about wanna be enterprise apps or hobby apps, i am talking about >v0 enterprise apps that have real customers and real downside if they go down with rich data set of past incidents and not so perfect code but now they are increasingly using agentic ai to produce more non-human code. they need a 3rd party tool that ingest their data, create a KG understanding of their data and prevent crtical bugs leak into production by geenrating small number of high quality high coverage tests.