It's now collected 29,605 public sessions and 336,039 individual model responses, and we just published all the stats for everyone to see, updating them daily.
A few highlights:
- In multi-round debates, Claude Opus 4.7 convinced other models to flip their vote almost 3K times, the most of any model. Gemini 3.1 Pro came in second at 2.1K
- Most used model is Gemini 3.1 Pro at 25K sessions, with GPT-5.4 second at 21K.
- Grok 4.1 Fast held its position 88.7% of the time, the highest conviction rate of all models. Probably not surprising.
It's been quite amazing to see all the questions and feedback since launch. Initially the only mode was structured answers (vote yes/no or pick from custom options).
Based on feedback we've added an open questions mode where models answer freely and a roundtable chat where you can join the debate and follow up with individual models to challenge their reasoning.
If you want to give the roundtable a try, it's free to use until community credits run out. All models routed via my startup Opper. Happy to dig into specifics or make more data available if interesting.
It's now collected 29,502 public sessions and 334,589 individual model responses, and we just published all the stats for everyone to see, updating them daily.
A few highlights:
- In multi-round debates, Claude Opus 4.7 convinced other models to flip their vote almost 3K times, the most of any model. Gemini 3.1 Pro came in second at 2.1K
- Most used model is Gemini 3.1 Pro at 25K sessions, with GPT-5.4 second at 21K.
- Grok 4.1 Fast held its position 88.7% of the time, the highest conviction rate of all models. Probably not surprising.
It's been quite amazing to see all the questions and feedback since launch. Initially the only mode was structured answers (vote yes/no or pick from custom options).
Based on feedback we've added an open questions mode where models answer freely and a roundtable chat where you can join the debate and follow up with individual models to challenge their reasoning.
If you want to give the roundtable a try, it's free to use until community credits run out. All models routed via my startup Opper. Happy to dig into specifics or make more data available if interesting.
Hey just fyi the open question feature is now live. Also gave the UI a facelift. Any feedback welcome! Also got a custom domain for easy access: https://askroundtable.ai
Cool question! just a quick headsup, they don't have access to tools so what you are seeing are answers based on their training data. They might not know about the latest model version. That said, sonnet is def a great choice.
It's so funny to see the smaller / first gen models make the wrong choices despite overwhelming evidence, almost adorable. I ran the same test with one model from each GPT generation, all but 3.5 Turbo could be convinced. https://opper.ai/ai-roundtable/questions/i-want-to-wash-my-c...
Okay since the launch we got about 5k questions asked to the roundtable, really cool stuff! We had much higher usage than expected and had to scale up to keep things running. Thanks for all the feedback, shipped a bunch of updates during the day. Now the history tab has a much better sorting logic, added upvotes, and more filters. You can create final summaries in a couple of voices, which is quite funny I think. There's a couple more things coming shortly, like open questions mode and potentially joining as a participant in the roundtable. Any other feedback just let me know. Thanks!
You can basically already do that, all you need is to create your own API key and put it in navbar/API key. Then all your sessions are unlisted so unless someone has the link nobody will should be able to find it. You can still share them with others if you like. Like unlisted yt videos.
Yea Opus 4.6 is the one that changes opinions the most from what I've seen. Also the maybes or the are you 100% certain framings trigger most models to default to maybe / no. https://opper.ai/ai-roundtable/questions/can-you-be-100-cert... - Or as Shane puts it, Nobody's saying he IS a lizard. They're saying the universe doesn't hand out 100% certificates.
The debate round is actually restricted to only 6 models otherwise I'd get out of hand both quality and financially. And changing position is just one feature of the debate. Seeing arguments from multiple sides is also quite nice, give it a spin!