The new site uses a 3-party format where you chat with both a human and an LLM at the same time, and your task is to decide which is which. This setup is closer to Turing's original idea, and we think it will be much harder for the models to pass.
We've also added a range of models, including GPT-4o, Claude, and LLaMA, along with new prompting techniques that allow the models to interact in different ways.
The site will be live every day from 12–1 PM and 8–9 PM GMT (8 AM and 3 PM ET, 5 AM and 12 PM PT). We've done this in the hope of increasing the density of users online at the same time.
I actually agree that Turing's 1950 definition is pretty vague in places and there are a few different interpretations out there. As we discuss in the paper, it's also unclear what should constitute a pass in terms statistical analysis.
Thanks for pointing this out, we do discuss it briefly in section 4.2, but I like your analysis in the blog!
Since the models are not passing, we didn't think it was a huge issue. If models are consistently passing the 2-person version, I think the motivation for running a (more cumbersome) 3-person version would be a lot stronger.
Thanks for the feedback and that's a great insight! The models will sometimes say they don't know but are probably much more likely to engage on obscure topics than a typical human.
Really excited to share turingtest.live, a site where you can play the Turing Test with GPT-4. You get randomly matched with either a human or an AI, and you have 5 minutes to decide which it is.
The platform is part of a research project I am carrying out on LLMs ability to interact like humans. While Turing originally developed the test as a way to measure machine intelligence, we're also interested in the extent to which models can deceive humans, and effective strategies for spotting and preventing deception.
The new site uses a 3-party format where you chat with both a human and an LLM at the same time, and your task is to decide which is which. This setup is closer to Turing's original idea, and we think it will be much harder for the models to pass.
We've also added a range of models, including GPT-4o, Claude, and LLaMA, along with new prompting techniques that allow the models to interact in different ways.
The site will be live every day from 12–1 PM and 8–9 PM GMT (8 AM and 3 PM ET, 5 AM and 12 PM PT). We've done this in the hope of increasing the density of users online at the same time.