I'm confused the demo shows typing in to select a element in a row which looks to be AI, I don't see anything that looks to be AI in the selectors? I'm not even sure how you would work with selectors unless you put the whole html into the context window or just ask which locator looks most reliable?
I used to work at a company with over 10,000 tests where we weren't able to get more than an 80% pass rate due to flaky tests. This article is great and covers a lot of the options for handling flaky tests. I founded Appsurify to make it easy for companies to handle flaky tests, with minimal effort.
First, don't delete them, flaky tests are still valuable and can still find bugs. We also had the challenge where a lot of the 'flakiness' was not the test or the application's fault but was caused by 3rd party providers. Even at Google "Almost 16% of our tests have some level of flakiness associated with them!" - John Micco, so just writing tests that aren't flaky isn't always possible.
Appsurify automatically raises defects when tests fail, and if the failure reason looks to be 'flakiness' (based on failure type, when the failure occurred, the change being made, previous known flaky failures) then we raise the defect as a "flaky" defect. Teams can then have the build fail based only on new defects and prevent it from failing when there are flaky test results.
We also prioritize the tests, which causes fewer tests to be run which are more likely to fail due to a real defect, which also reduces the number of flaky test results.
I have just launched a startup which uses AI to predict software defects, improve testing and reduce test automation failures. I'm looking for a co-founder who is sales focused. We have 1 customer (almost 6 figure contract), 2 others in sales negotiations and about 10 in the pipeline. We are also at the interview stage for a couple of well-regarded incubators. Appsurify.com or reach out to me at [email protected]. Remote Ok.