Reddit and Instagram dont work because youre broadcasting to people who dont know they have the problem yet. When we started the services business I got the first few clients by solving their actual pain point so badly they had to keep talking to me about it - basically trapped them in conversation. What problem does your beta solve and have you actually talked to 20 people who currently have that problem, or are you just posting to strangers?
The voting thing breaks because youre treating all models equally when they shouldnt be. We ran consensus logic like this on a smaller scale and quickly realized throwing 5 mediocre models at a problem just makes them argue in circle. One good model beats three bad ones always. The synthesis round will get expensive fast too - we started with 2 models doing 3 rounds and it was already costing 40x a single pass. For brainstorm mode maybe weight models by past accuracy instead of pure voting? We do this with our team internally - the person who got it right last time gets listened to more next time, not equal voice to everyone. Could be interesting to test.
The debate round sounds good until you actually use it. I built internal tools for a 35-person team and the same thing always happens - models see each other's answers and just shuffle the phrasing around instead of actually changing their reasoning. What you're measuring is performance on persuasion, not on accuracy or clarity. The real question isnt whether Claude will convince Gemini to flip its position. Its whether having 200 models debate helps you make a better decision than asking one model well and checking its work yourself. I'd use this more as a way to find edge cases where models disagree wildly, not to find consensus.
I disagree that offloading all coding to AI makes sense. We started using Claude heavily for our services work about a year ago and what we found is you still need to understand the codebase deeply to know when the AI is hallucinating or changing wrong things. The prompt-wait-evaluate cycle you mentioned is real and kills productivity on brownfield projects. What actually worked for us was using AI for scaffolding new features or writing tests, not for debugging existing code. On debugging I still do it myself because I know our system and can fix it in 2 minutes vs 20 minutes prompting Claude. The people saying they never write code anymore probably either work on greenfield projects or they're not measuring their actual time cost accurately.