I actually think the AI Overviews from Google have improved a lot in the last 2 years. They used to be trash. And now they are often good-enough so that I do not even switch to ChatGPT anymore.
The traditional search results suffer a lot because AI and AI content generation have enable a lot of aggressive SEO/spam plays.
Pure non-AI googling will not work since many websites now use AI to create content. And so far, no search engine has managed to reliable detect and filter that out.
I like the idea. But currently the only thing I can see is "shift from model capability discussion toward tool and workflow integration". This does not really motivate me to create an account.
In the past, you would type a question into Google. To get the full, detailed answer you had to click on the Wikipedia search result.
With ChatGPT or Google AI Mode, you get all answers directly in the chat. And you can even ask follow-up questions. There is no need to click on a link.
From the data I have seen, 40% of searches on Google used to lead to a click to another website. In ChatGPT and Google AI Mode, this number is lower than 5%. One study (with a small N) even came to the conclusion that the number is 0%.
> people having AI talk to their partner about deep relationship stuff.
I have read stories about people using AI to write their Tinder messages, eulogies, etc.
If if all this is true, LLMs would still need to create a repository of such aggregated "summarizations of secondary sources".
LLMs cannot in real time find and read thousands of secondary sources. Especially not if some of these might have already disappeared or are not digitalized.
I can see a future where LLM labs a) donate to Wikipedia and b) contribute to it with agents that suggest edits and review facts.
I like to idea of using AI to make Wikipedia better.
There are many small tasks simple AI agents could handle.
For example:
Go to every article in the English Wikipedia of a Spanish city. Check data like inhabitants, major, etc. and compare them to the Spanish Wikipedia article. (The assumption here is that the Spanish Wikipedia will be more up-to-date for Spanish cities than the English Wikipedia.) Double-check what is written in the Spanish article. Update the English article accordingly.
If such an agents is only allowed to create drafts, human editors could review them and we would get a lot of small updates in.
> putting our competitors down
I did not put anybody down. I am surprised people still start to build in this very crowded space. I was genuinely hoping for a unique twist/idea - like a focus on local, on non-western LLMs, etc.
Google is using a special version of Gemini (fast, small) and a special version of their internal ranking API (faster, fewer anti-spam/quality measures).
That makes them very fast. But that also leads to a ton of hallucinations. If you ask for non existent things (like the cats.txt protocol), AI Overviews consistently fabricate facts. Ai Overviews can pull the content of the potential source ULRs directly from Google's cache.
ChatGPT is slow because they have to make an external API call to Bing or - even worse - to a scraping provider like SerpApi/Data4SEO/Oxylabs to crawl regular Google search results. That introduced two delays. OpenAI then has to fetch some of these potential source URLs in real time. That introduces another delay. And then OpenAI also uses a better (but slower) model than Google to generate the answer.
Over time, OpenAI should be able to catch up in terms of speed with their own web/search index.
If you try more complex questions, you might find AI Overviews less to your liking.
Google gets away with this because users are used to type simple queries - often just a few keywords. Any kind of AI answer is like magic.
OpenAI cannot do the same. Their users are used to having multi-turn conversations and receiving thoughtful answers to complex questions.
Based on my experiments (across hundreds of thousands of LLM chats) llms.txt does not have any measurable impact in the vast majority of scenarios.