Google's new AI mode is good, actually(simonwillison.net)
simonwillison.net
Google's new AI mode is good, actually
https://simonwillison.net/2025/Sep/7/ai-mode/
71 comments
Interestingly, per the recent Google antitrust ruling documents, AI mode is extra fast because of a special FastSearch index: https://x.com/Marie_Haynes/status/1963031598829314161
Gemini in general is extremely fast, compared to ChatGPT 5 Thinking.
It also seems to excel at things ChatGPT 5 Thinking isn't good at. Simple things like "Here's a screenshot of text, please transcribe it" - ChatGPT 5 Thinking will spend 2 minutes and still get the results wrong, while Gemini Pro will spend 20-30 seconds and transcribe everything perfectly.
Obviously that's just 1 use case, but as someone who previously used ChatGPT exclusively, I'm increasingly impressed by Gemini the more I use it. Mainly due to the much faster thinking times that seem to provide equal or better results than GTP 5 Thinking.
It also seems to excel at things ChatGPT 5 Thinking isn't good at. Simple things like "Here's a screenshot of text, please transcribe it" - ChatGPT 5 Thinking will spend 2 minutes and still get the results wrong, while Gemini Pro will spend 20-30 seconds and transcribe everything perfectly.
Obviously that's just 1 use case, but as someone who previously used ChatGPT exclusively, I'm increasingly impressed by Gemini the more I use it. Mainly due to the much faster thinking times that seem to provide equal or better results than GTP 5 Thinking.
Gemini models have always been the strongest at image understanding. We're talking like 1.5 years now. It's also better at a good number of non-English/European languages, this too has been ever since 1.5 pro.
Gemini is very fast because it runs on TPUsV7 mostly
It is definitely because it's a smaller model. TPUv7 has ~10% lower flops at FP8 and 33% lower memory bandwidth than Nvidia Blackwell cards. Add CUDA to the comparison and they'll probably be even worse at real world utilization. Grok is already running on Blackwell cards and although there's little info on GPT5, I doubt they are behind.
>"Here's a screenshot of text, please transcribe it"
I took pictures of a book, asked Gemini to transcribe, then to translate them and I'm now in the process of having it reproduce the whole book in latex (lot of figures). Not sure exactly what I'm doing but but I've been wondering: Should I reproduce the publishing house logo? Or invent my own? Damn, this is fun.
I took pictures of a book, asked Gemini to transcribe, then to translate them and I'm now in the process of having it reproduce the whole book in latex (lot of figures). Not sure exactly what I'm doing but but I've been wondering: Should I reproduce the publishing house logo? Or invent my own? Damn, this is fun.
My main takeaway is that Google's AI efforts are still playing catch-up in the mindshare race even if they've long since caught up with or surpassed OpenAI technically. A huge number of people who are heavy users of ChatGPT or Claude may have never tried Gemini.
A huge number of people that use Google Docs and other products daily are trying really hard to avoid using it in spite of it being rammed down their throats to the point that leaving Google starts to look like a viable option. Oh, and of course you do pay for it even if you don't use it and never plan to use it.
Yeah, I changed my primary search engine to DuckDuckGo when Google started forcing incorrect AI results down my throat.
I chanted it when I realized that bad search results were not a lost fight against SEO, but intention. That made me try other search engines ... and both bing and duckduckgo are better.
I have Userscripts setup to hide the AI mode buttons and other AI results.
Also in Google docs, gmail and so on?
I don't use those on my computer, so I guess I haven't run into it there. My setup is mostly for Google search and YouTube on both mobile safari and macOS Safari. For YouTube, I have hidden the shorts, community posts and any videos with less than 1000 views (almost always AI generated slop).
Good stuff. If you are sharing it anywhere a pointer would be appreciated. I've already done as much as I can without spending a week hacking up scripts and I'm game for anything that will improve this further.
whimsicalism(3)
i think they are shooting themselves in the foot by automatically including results from a quite dumb model in google search
Strongly agreed. I think this is a large part of the reason behind dissonance of opinions on AI usefulness we see even here: unless you are intentional about it, your main experience with AI is that of non-SOTA or budget versions of the models.
What makes it worse is that even if you are intentional and are willing to pay, it doesn't help everywhere. Gemini as voice assistant defaults to Flash and there is no choice to change it, even though the speed of the faster model hardly matters here, while accuracy does. Same with AI suggestions everywhere, and other such "ambient" features.
The flip site of being intentional about always using SOTA models, is that you notice when you are not getting them.
What makes it worse is that even if you are intentional and are willing to pay, it doesn't help everywhere. Gemini as voice assistant defaults to Flash and there is no choice to change it, even though the speed of the faster model hardly matters here, while accuracy does. Same with AI suggestions everywhere, and other such "ambient" features.
The flip site of being intentional about always using SOTA models, is that you notice when you are not getting them.
I kind of think of them as the thing that pops up when you try googling something. A lot of people have probably come across that.
IME Gemini has a lot more non-tech mindshare than Claude. For tech it's of course the opposite.
The example itself that Simon puts up is questionable. I might be wrong about this, but I thought I read elsewhere that the “buy, scan, destroy” method was explicitly not the problem, and instead the issue was that anthropic downloaded libgen, and the settlement was for libgen.
Edit: https://news.ycombinator.com/item?id=45143392
Edit: https://news.ycombinator.com/item?id=45143392
Yeah, you're right! The answer is definitely misleading at best. It would be better if the sentence "This method was a major component of a copyright lawsuit settlement that Anthropic paid in September 2025." was removed.
I'm sure this method _did_ come under discussion in the lawsuit & settlement, but as you pointed out the settlement itself was only about pirated works.
I'm sure this method _did_ come under discussion in the lawsuit & settlement, but as you pointed out the settlement itself was only about pirated works.
Welcome to subtle misinformation as truth everywhere. "Ministry of Truth" might be a good name for this phenomenon.
Yeah, it's amazing how fast people can overlook incorrect results when the form is in the spotlight. Or the AI rhetoric is convincing or plausible.
I recall a tweet by Altman, leaking the launch of GPT-5, praising their new model's answer to a prompt about thought provoking TV shows about AI. The X thread that followed was about the form ("em-dashes are still there!") and nearly nobody cared to evaluate that neither shows recommended were about AI. They weren't, or at least, were very debatable as belonging to the genre.
I recall a tweet by Altman, leaking the launch of GPT-5, praising their new model's answer to a prompt about thought provoking TV shows about AI. The X thread that followed was about the form ("em-dashes are still there!") and nearly nobody cared to evaluate that neither shows recommended were about AI. They weren't, or at least, were very debatable as belonging to the genre.
I especially liked how the image inside of a link I shared of an example of GPT-5's (in)ability to rank states was decented sans warning...
Yeah, reading the response raised a lot of questions for me too. I find it striking that he doesn't comment on whether the presented facts were true or relevant. What does it mean that "In 2025, a search tool revealed that Meta [...]". Is it the search tool that the "AI mode" is using under the hood?
Yeah, I agree: "This method was a major component of a copyright lawsuit settlement that Anthropic paid in September 2025" is a bit misleading, I actually wrote a whole separate thing about that here: https://simonwillison.net/2025/Sep/6/anthropic-settlement/
i don’t read the AI answer as saying otherwise, but one might read that the physical purchasing was used as part of the settlement. it was used as part of the defense to mitigate though
I usually can't read blog posts, they are stale, boring and can't keep my attention span unless I know there is something interesting the paragraph, this is regardless of the interesting title. But when it comes to Simon willisons blog, I enjoy reading it, it's entertaining and interesting, it reads like a human and is enjoyable to read
Also, I can't access Google AI mode because I'm in EU but when looking at the video on YT, it looks like Perplexity, but googlified. I haven't seen any other tool that comes close to Perplexity yet, I have their app installed on all my devices and it's part of my daily life, it's so good! Especially with their pro plan (I got 12-months for free)
Also, I can't access Google AI mode because I'm in EU but when looking at the video on YT, it looks like Perplexity, but googlified. I haven't seen any other tool that comes close to Perplexity yet, I have their app installed on all my devices and it's part of my daily life, it's so good! Especially with their pro plan (I got 12-months for free)
Google is really shooting themselves in the foot with AI overviews.
It's probably the most popular AI on earth by daily queries, and likewise probably an ~8B level model, it means a whole bunch of people equate Google AI to AI overviews.
It's probably the most popular AI on earth by daily queries, and likewise probably an ~8B level model, it means a whole bunch of people equate Google AI to AI overviews.
I'm a bit biased, but I find the AI overviews to be basically great. All I want from a search engine is the correct answer, Google's knowledge graph has done that for many queries for a long time, and AI overviews seems like a good next step in that process.
I've not seen many hallucinations, fact checking is fairly straightforward with the onward links, and it's not like I can take any linked content at face value anyway, I'd still want to fact check when it makes sense even if it wasn't AI written.
I've not seen many hallucinations, fact checking is fairly straightforward with the onward links, and it's not like I can take any linked content at face value anyway, I'd still want to fact check when it makes sense even if it wasn't AI written.
You've been asking things that are verbatim in the first few links. Ask anything that isn't and it will confidently hallucinate.
I constantly get blatant hallucinations. It particularly likes to take feature requests/suggestions and tell me they're presently possible, when they're not. It's long past the point where I just ignore the AI overviews entirely.
I wonder to what extent that “generate some garbage spam along with every search” has hurt their reputation among the general public.
probably close to zero? people get the anwers for most questions.
Small-time blogs were dead before AI
Small-time blogs were dead before AI
People get an answer, and not the answer. It seems that people are fine with this. Even this article's example answer is false. The author didn't care, and considered it good.
I believe that Google has figured out (correctly, IMO) that accuracy doesn't matter to most people 99% of the time, and people will likely do a deeper search for the 1% of the time they do want it.
If people really wanted the truth and facts, we would not have misinformation spread this widely via social media and other places.
If people really wanted the truth and facts, we would not have misinformation spread this widely via social media and other places.
People prefer to be lazy and sit and do nothing, than to move around to achieve a goal.
Is that a good thing? The reality is most humans are becoming more and more intellectually lazy - as a result their cognitive function are in decline. Therefore if something looks right at face value / supports an internal bias - they take it and run with it.
Is that a good thing? The reality is most humans are becoming more and more intellectually lazy - as a result their cognitive function are in decline. Therefore if something looks right at face value / supports an internal bias - they take it and run with it.
[deleted]
They're also shooting themselves in the foot by halving the traffic to the websites that provide the training data.
yeah a lot of people i speak to i think have developed the impression that modern AI hallucinates more than it does from the dumb search model
[deleted]
I agree, AI Mode is _very good_. But there's a huge branding and discovery problem in that 100% of the people I mention this to think I'm talking about the (error riddled) AI Overviews and have never heard of AI Mode.
I do probably 40% of my searches with AI Mode now. It can't possibly be profitable (and maybe that's why it's not more discoverable), but the results are awesome.
Edit: I also tried to show my aging parents how to use it, and it was inexplicably not available on their devices. They use old (10ish year) ios devices, which is apparently incompatible even though it's a web interface.
I do probably 40% of my searches with AI Mode now. It can't possibly be profitable (and maybe that's why it's not more discoverable), but the results are awesome.
Edit: I also tried to show my aging parents how to use it, and it was inexplicably not available on their devices. They use old (10ish year) ios devices, which is apparently incompatible even though it's a web interface.
The "AI Mode" tab appears at the top of every google search for me but it's true they don't really push it.
Just tried a few side by side comparisons with perplexity.
Google answers more concisely, faster and confidently, but not convinced quality of output is better. e.g. Google pulled in info from AWS and Oracle cloud when I asked a GCP specific question. Perplexity sourced only from GCP docs
Which is an interesting outcome since I'd expect google to excel in the search aspect
Google answers more concisely, faster and confidently, but not convinced quality of output is better. e.g. Google pulled in info from AWS and Oracle cloud when I asked a GCP specific question. Perplexity sourced only from GCP docs
Which is an interesting outcome since I'd expect google to excel in the search aspect
> since I'd expect google to excel in the search aspect
You still do?
You still do?
On that note, AI mode's image search is really good
I've gotten it to identify:
- the comic from a random page of an obscure Russian comic
- obscure French comedy from a random clip
It was extra impressive because even reverse search from lens didn't immediately identity them
I've gotten it to identify:
- the comic from a random page of an obscure Russian comic
- obscure French comedy from a random clip
It was extra impressive because even reverse search from lens didn't immediately identity them
I read this as "AI Mode is God".
Strangely I was unperturbed, quietly resigning myself to our new deity.
My first read was Google AI is God mode. They would earn back some points from me if they added a Konami code like entry point to the feature.
Me too at first glance. Funny.
I too feel AI mode seems to good when it knows the topic, with two caveats:
- it works when the info is either relatively well known or quite new
- no-AI mode now becomes dumber, the old trick to "grep" the internet with +/-/"" is gone
- it works when the info is either relatively well known or quite new
- no-AI mode now becomes dumber, the old trick to "grep" the internet with +/-/"" is gone
Supposedly if you use swear words in your search string that suppresses use of AI (or at least suppresses the AI overview - maybe they don't want it to swear back at you?).
At least some of the Google search operators seem to still work, although Google themselves aren't very forthcoming about documenting these.
https://ahrefs.com/blog/google-advanced-search-operators/
At least some of the Google search operators seem to still work, although Google themselves aren't very forthcoming about documenting these.
https://ahrefs.com/blog/google-advanced-search-operators/
operators still work, but some results are gone, at least that's my experience has been.
It is pretty good, although Perplexity is better (but nominally not free, although you can get a free subscription now).
OpenAI searches are even better, but GPT5 is extremely slow with thinking. Without thinking it's roughly equivalent.
OpenAI searches are even better, but GPT5 is extremely slow with thinking. Without thinking it's roughly equivalent.
I’m not surprised by Google’s lack of search transparency. If Google told users that Google was searching an internal database linked to a particular search string there would be pitchforks and torches.
I highly doubt they are doing that. If the quality is good then presumably it's also not the direct output of an LLM with RAG tool use.
I'd guess it might be more of a structured/agentic approach - maybe having learnt how to map "search" strings to relevant data retrieval queries, then combining/summarizing the returned results.
I'd guess it might be more of a structured/agentic approach - maybe having learnt how to map "search" strings to relevant data retrieval queries, then combining/summarizing the returned results.
No one cares, they are busy watching tik tok style videos and rotting their children's brains.
I assumed all of the labs were already doing something like this to cut down on costs
Isn't that just... caching
It feels like there is a different search use case that AI accels in while it destroys another.
- When I know so little about a subject that the concepts are all vague and I'm intellectually grasping in the dark. LLMs are good at taking my imprecise wording and orienting me towards the paths/gradients others have taken.
VS
- I know this subject well enough and instead of fumbling around I want to be able to run grep over a massive amount of open text data.
While the former mode is useful, the act of training without asking has led to the repopularization of walled gardens. Early Google felt like being able to grep every library book in existence and a 3 paragraph summary in response to very poorly worded questions is a terrible trade.
I know Simon is smart and he included his search terms with their warts to be open so I don't want to elicit shame over this. But cmon, just type out bought ("Anthropic but lots of physical books..."). Complete anecdote but I have noticed my LLM reliant friends have become way worse at texting and it feels like it's worth taking 5 seconds to try to structure your thoughts, simply for the practice
- When I know so little about a subject that the concepts are all vague and I'm intellectually grasping in the dark. LLMs are good at taking my imprecise wording and orienting me towards the paths/gradients others have taken.
VS
- I know this subject well enough and instead of fumbling around I want to be able to run grep over a massive amount of open text data.
While the former mode is useful, the act of training without asking has led to the repopularization of walled gardens. Early Google felt like being able to grep every library book in existence and a 3 paragraph summary in response to very poorly worded questions is a terrible trade.
I know Simon is smart and he included his search terms with their warts to be open so I don't want to elicit shame over this. But cmon, just type out bought ("Anthropic but lots of physical books..."). Complete anecdote but I have noticed my LLM reliant friends have become way worse at texting and it feels like it's worth taking 5 seconds to try to structure your thoughts, simply for the practice
Google's AI mode recently told me the haber process was invented in 1999, and in the same paragraph told me it won the Nobel prize in 1918...
I hear you on this. My MO lately has been I never run anything through an AI unless I know what the wrong answer looks like or I know how to verify the answer. Under this MO, it's still a great tool just not as great as the investor class claims or wishes.
Was that AI mode or AI overviews?
Huh, hadn't realized that Anthropic had gone full Rainbows End. RIP Vinge :(
Current explosion in AI in general is "good". We tend to over-criticize and nit-pick it as what I believe is a natural response that comes from a "communal subconsciousness" -- even if us as individuals won't admit it, as a culture we are scared and averse to what's happening.
I think this communal subconscious response is coming from a valid place though. I will call the current explosion in AI if:
- it causes mass unemployment and social unrest
- leads to a further concentration of wealth and increase in wealth inequality
- it means I have to work more, produce more, all for the same wage or less
- it's implementation leads to large societal harms such as increased isolation/loneliness
- it ends up being overhyped causes a large economic crisis