Yes, I feel so. I started happening from june first weekish.
I have shifted to claude code for planning things and codex for execution (as it is faster, though dumb)
Yes a lot. So with the acceleration of AI in the software engineering. Features are being shipped faster but causes regressions. The only way to verify is either you write tests with AI and spend hours reviewing them or you do manual QA. agent-qa aims to solve the later. First your product should work for the end user, later you can write clean test etc.
Hey, I just gave it a try and ran a quick test on booking.com. It took ~3 mins for a basic test. Do you cache the test steps so that future runs are faster and they don't call LLMs for the subsequent runs?
Also your current pricing is $300 for 1K tests which means $0.3 for each test. We tried out playwright mcp and it easily consumes 1M+ tokens for a test with ~20 steps (including image input). So with this pricing are you guys default alive?
Also is there a benchmark which you ran to prove the efficacy of your testing agent? because in the current stage it is a trust me bro kinda thing.
Thanks for those kind words. The landing page's demo required lots of sculpting.
I would say that agent-qa is not only frontend focused. As you can run hooks in sandboxed env to test apis, so a better way to put it is with agent-qa you can test the product end-to-end not only UI.
But the issue with API testing / backend is that coding harnesses are really good at it. A product manager who writes user stories should be able to write tests for the product, and usually PMs don't care about the APIs.
It is being used in production at https://vostride.com/agent-qa
The issues was agent-qa have many different kinds of files tests, memory etc and there's too much FK references which LLMs need to resolve. Using id-agent worked like a charm