Kagi Assistants(blog.kagi.com)
blog.kagi.com
Kagi Assistants
https://blog.kagi.com/kagi-assistants
121 comments
Maybe if Google hears this they will finally lift a finger towards removing garbage from search results.
Hey Google, Pinterest results are probably messing with AI crawlers pretty badly. I bet it would really help the AI if that site was deranked :)
Also if this really is the case, I wonder what an AI using Marginalia for reference would be like.
Hey Google, Pinterest results are probably messing with AI crawlers pretty badly. I bet it would really help the AI if that site was deranked :)
Also if this really is the case, I wonder what an AI using Marginalia for reference would be like.
> Maybe if Google hears this they will finally lift a finger towards removing garbage from search results.
It's likely they can filter the results for their own agents, but will leave other results as they are. Half the issue with normal results are their ads - that's not going away.
It's likely they can filter the results for their own agents, but will leave other results as they are. Half the issue with normal results are their ads - that's not going away.
“Show me the incentive and I’ll show you the outcome” - Charlie Munger
Kagi works better and will continue to do so as long as Kagi’s interests are aligned with users’ needs and Google’s aren’t.
Kagi works better and will continue to do so as long as Kagi’s interests are aligned with users’ needs and Google’s aren’t.
>Maybe if Google hears this they will finally lift a finger towards removing garbage from search results.
Unlikely. There are very few people willing to pay for Kagi. The HN audience is not at all representative of the overall population.
Google can have really miserable search results and people will still use it. It's not enough to be as good as google, you have to be 30% better than google and still free in order to convert users.
I use Kagi and it's one of the few services I am OK with a reoccurring charge from because I trust the brand for whatever reason. Until they find a way to make it free, though, it can't replace google.
Unlikely. There are very few people willing to pay for Kagi. The HN audience is not at all representative of the overall population.
Google can have really miserable search results and people will still use it. It's not enough to be as good as google, you have to be 30% better than google and still free in order to convert users.
I use Kagi and it's one of the few services I am OK with a reoccurring charge from because I trust the brand for whatever reason. Until they find a way to make it free, though, it can't replace google.
They are transparent about their growth of paying customers, do you feel as if this fairly consistent and linear rate of growth will never be enough to be meaningful?
https://kagi.com/stats
https://kagi.com/stats
> Maybe if Google hears this they will finally lift a finger towards removing garbage from search results.
They spent the last decade and a half encouraging the proliferation of garbage via "SEO". I don't see this reversing.
They spent the last decade and a half encouraging the proliferation of garbage via "SEO". I don't see this reversing.
Google wants there to be garbage in search results, because most of the garbage sites are full of Google ads.
Makes me wish the Kagi API was available and not prohibitively priced: https://help.kagi.com/kagi/api/search.html
[deleted]
> Primarily because google as it is today includes a massive amount of noise and suffered from blowback/cross-contamination as more LLM generated content pollute information truth.
I'm not convinced about this. If the strategy is "lets return wikipedia.org as the most relevant result", that's not sophisticated at all. Infact, it only worked for a very narrow subset of queries. If I search for 'top luggages for solo travel', I dont want to see wikipedia and I dont know how kagi will be any better.
I'm not convinced about this. If the strategy is "lets return wikipedia.org as the most relevant result", that's not sophisticated at all. Infact, it only worked for a very narrow subset of queries. If I search for 'top luggages for solo travel', I dont want to see wikipedia and I dont know how kagi will be any better.
The wrote "returned the relevant Wikipedia page higher" and not "wikipedia.org as the most relevant result" - that's an important distinction. There are many irrelevant Wikipedia pages.
(Kagi staff here)
Generally we do particularly better on product research queries [1] than other categories, because most poor review sites are full of trackers and other stuff we downrank.
However there aren't public benchmarks for us to brag about on product search, and frankly the simpleQA digression in this post made it long enough it was almost cut.
1. (Except hyper local search like local restaurants)
Generally we do particularly better on product research queries [1] than other categories, because most poor review sites are full of trackers and other stuff we downrank.
However there aren't public benchmarks for us to brag about on product search, and frankly the simpleQA digression in this post made it long enough it was almost cut.
1. (Except hyper local search like local restaurants)
I tried a prompt that consistently gets Gemini to badly hallucinate, and it responded correctly.
Prompt: "At a recent SINAC conference (approx Sept 2025) the presenters spoke about SINAC being underresourced and in crisis, and suggested better leveraging of and coordination with NGOs. Find the minutes of the conference, and who was advocating for better NGO interaction."
The conference was actually in Oct 2024. The approx date in parens causes Gemini to create an entirely false narrative, which includes real people quoted out of context. This happens in both Gemini regular chat and Gemini Deep Research (in which the narrative gets badly out of control).
Kagi reasonably enough answers: "I cannot find the minutes of a SINAC conference from approximately September 2025, nor any specific information about presenters advocating for better NGO coordination at such an event."
Prompt: "At a recent SINAC conference (approx Sept 2025) the presenters spoke about SINAC being underresourced and in crisis, and suggested better leveraging of and coordination with NGOs. Find the minutes of the conference, and who was advocating for better NGO interaction."
The conference was actually in Oct 2024. The approx date in parens causes Gemini to create an entirely false narrative, which includes real people quoted out of context. This happens in both Gemini regular chat and Gemini Deep Research (in which the narrative gets badly out of control).
Kagi reasonably enough answers: "I cannot find the minutes of a SINAC conference from approximately September 2025, nor any specific information about presenters advocating for better NGO coordination at such an event."
I also tried two hallucinations that I've seen in the latest Gemini and Claude. The first one is the famous "how many r in strawberry" and another one is asking about king sacrifice in chess where most LLMs tell you it's a very powerful move and tell you how to do it (obviously a nonsense since the point of the game is to protect the king).
Kagi actually got those two correct, explaining how losing your king means losing your chess game, which is quite impressive.
Kagi actually got those two correct, explaining how losing your king means losing your chess game, which is quite impressive.
I tried both these prompts (along with the SINAC one as per GP) in Sonnet 4.5 and Gemini 3, and they both answered correctly for all three. Both also provided context on the chess question as well.
All of this will depend on the settings on the model (reasoning effort, temperature, top_k,etc) as well.
Which is why you should have benchmarks that are a bit broader generally (>10 questions for a personal setup) otherwise you overfit to noise
Which is why you should have benchmarks that are a bit broader generally (>10 questions for a personal setup) otherwise you overfit to noise
You might be interested to know your comment caused Gemini 3 to go meta on this question. This is what I found in its thinking:
``` Evaluating Date Discrepancies
I've just uncovered a significant discrepancy: a "Hacker News" result flags a date error originating from the very format of parenthetical dates. The system appears to be misinterpreting these, leading to demonstrably false information. This warrants further investigation to understand the root cause and ensure data integrity.
Analyzing Date Ambiguity
I'm now grappling with the user's date. The Hacker News comment suggests the user's prompt might be flawed, as it contradicts the actual date of an event I've indexed. Specifically, the conference referenced occurred in October 2024, not September 2025 as the prompt implies. This leads me to question the user's understanding of the timeline or if the prompt refers to an unrelated event in September 2025. This inconsistency demands further clarification.
...
Investigating User Prompt Intent
I've just realized that the Hacker News snippet reveals this might be a specific problematic test prompt. My strategy is to focus on the October 2024 forum, a strong match, organized by Kattia Cambronero and CCT. I'll verify if there was a 2025 event, but the 2024 event fits the described "crisis" and "NGO coordination". If the user says "approx Sept 2025," they likely mean a recent event. ```
``` Evaluating Date Discrepancies
I've just uncovered a significant discrepancy: a "Hacker News" result flags a date error originating from the very format of parenthetical dates. The system appears to be misinterpreting these, leading to demonstrably false information. This warrants further investigation to understand the root cause and ensure data integrity.
Analyzing Date Ambiguity
I'm now grappling with the user's date. The Hacker News comment suggests the user's prompt might be flawed, as it contradicts the actual date of an event I've indexed. Specifically, the conference referenced occurred in October 2024, not September 2025 as the prompt implies. This leads me to question the user's understanding of the timeline or if the prompt refers to an unrelated event in September 2025. This inconsistency demands further clarification.
...
Investigating User Prompt Intent
I've just realized that the Hacker News snippet reveals this might be a specific problematic test prompt. My strategy is to focus on the October 2024 forum, a strong match, organized by Kattia Cambronero and CCT. I'll verify if there was a 2025 event, but the 2024 event fits the described "crisis" and "NGO coordination". If the user says "approx Sept 2025," they likely mean a recent event. ```
Ah yes we have some benchmarks on this sort of misguided prompt trap, so it should perform well on this
As a Kagi subscriber, I find this to be mostly useful. I'd say I do about 50% standard Kagi searches, 50% Kagi assistant searches/conversations. This new ability to change the level of "research" performed can be genuinely useful in certain contexts. That said, I probably expect to use this new "research assistant" once or twice a month.
I'd say the most useful part for me is appending ? / !quick / !research directly from the browser search bar to a query
I have yet to understand when to use `?` versus `!quick`. The former is much less disruptive since it's very fast and displays an answer above the normal search results. And I haven't played with them enough to know what kinds of things `!quick` is meaningfully better at than `?`.
Do you have any pointers?
Do you have any pointers?
I've already used the Research assistant half a dozen times today and am super happy with the outcomes. It does seem to be more trigger happy with doing multiple searches based on information it found in earlier results, and I've found the resulting output to be reasonably accurate. Some models in particular seem to never want to do more than one search, and you can tell the output in those cases is often not very useful if the sources partially contradict each other or don't provide enough detail. The best I've found to avoid this is o3 pro, but o3 pro is very slow and expensive. If the Research assistant gets 85% of the results in half the time as o3 pro...
Same, I'm quite happy with it. I first subscribed because I was fed up with the promoted results in Google but now I find their assistant searches actually useful too.
I really enjoy the tone/style of this announcement/blog post. It doesn't feel like it's overhyping something or using "salesy" language. I wish we would see this more often from companies.
I'm struggling to find a usecase for the quick assistant vs Kimi K2. Does anyone know any particular situations where the Assisstant is better than the model? I am not seeing how it is different than a model. Only speaking in regards to quick.
Could Kagi Ultimate be a good replacement for ChatGPT? A subscription there would save me some money if it was similar.
Just recently started paying for Kagi search and quite love it.
Just recently started paying for Kagi search and quite love it.
Mostly depends on which features are most important for you. I'm a SWE, so using their Assistant for Web RAG nearly exclusively for work-related stuff and for most of my personal queries. I'm rarely using multi-modal content, mostly sticking to text. They support many providers and new notable models are typically rolled out only a few days after release which is always great to test out. I have a standalone subscription for coding agent LLM. If above aligns - might be a good choice.
I do a lot of help with spreadsheets and some ongoing threads in ChatGPT.
I’m an Assistant Principal so use it to help me get better with spreadsheets, churn through complex formulas, and some other miscellaneous tasks for feedback and assistance. Definitely use a lot of screenshots of things to also help consume info in there.
I think I might be stuck with both for a while as I’m not sure Kagi can quite fill this gap yet.
I’m an Assistant Principal so use it to help me get better with spreadsheets, churn through complex formulas, and some other miscellaneous tasks for feedback and assistance. Definitely use a lot of screenshots of things to also help consume info in there.
I think I might be stuck with both for a while as I’m not sure Kagi can quite fill this gap yet.
I use regular Kagi assistant for most of my questions. I use Openrouter chat to chat with Gemini 2.5 Pro and GPT 5 at the same time and continue only the productive conversation. My OR account is for my coding agent and its completely pay as you go, no fixed monthly costs.
I’ve stuck with chatgpt and do dump screenshots, spreadsheets, etc. in there with some regularity. Screenshots are mostly for convenience and spreadsheets when I’m still stuck on something. Didn’t even think about loading in different things. I can see potential for this in the future but I’ve been with ChatGPT for so long exactly because it does so well with accepting files to work with.
Do you use it regularly (20 min a day or so)? If so, how is your monthly consumption - does it beat the subscription costs for either service?
I don’t use 20 minutes a day or so. It is more like I don’t use it a ton and then tend to go through periods where I might lean on it a lot. This is why I feel like it is tough to make the call.
I guess I should also explore how capable the free version is at this point, too.
I guess I should also explore how capable the free version is at this point, too.
That’s what I did and I’m pretty happy with it. I just fall back to something free on the rare occasion I want an image generated (tbh, mostly emojis of my dog).
Mostly, yes.
You have a spend limit, but the assistant has dozens of of models
You have a spend limit, but the assistant has dozens of of models
Kagi reminds me of the original search engines of yore, when I could type what I want and it would appear, and I could go on with my work/life.
As for the people who claim this will create/introduce slop, Kagi is one of the few platforms where they are actively fighting against low quality AI generated content with their community fueled "SlopStop" campaign.[0]
Not sponsored, just a fan. Looking forward to trying this out.
[0] https://help.kagi.com/kagi/features/slopstop.html
As for the people who claim this will create/introduce slop, Kagi is one of the few platforms where they are actively fighting against low quality AI generated content with their community fueled "SlopStop" campaign.[0]
Not sponsored, just a fan. Looking forward to trying this out.
[0] https://help.kagi.com/kagi/features/slopstop.html
I used quick research and it was pretty cool. A couple of caveats to keep in mind:
1. It answers using only the crawled sites. You can't make it crawl a new page. 2. It doesn't use a page' search function automatically.
This is expected, but doesn't hurt to take that in mind. I think i'd be pretty useful. You ask for recent papers on a site and the engine could use hackernews' search function, then kagi would crawl the page.
1. It answers using only the crawled sites. You can't make it crawl a new page. 2. It doesn't use a page' search function automatically.
This is expected, but doesn't hurt to take that in mind. I think i'd be pretty useful. You ask for recent papers on a site and the engine could use hackernews' search function, then kagi would crawl the page.
What exactly do you mean by "You can't make it crawl a new page"? It has the ability to read webpages, if that is what you're referring to
This query's results are wrong:
"""
site:https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=fal... recent developments in ai?
"""
Also when testing, if you know a piece of information exists in a website, but the information doesn't show up when you run the query, you don't have the tools to steer the engine to work more effectively. In a real scenario you don't know what the engine missed but it'd be cool to steer the engine in different ways to see how that changes the end result. For example, if you're planning a trip to japan, maybe you want the AI to only be shown a certain percentage of categories (nature, night life, or places too), alongside controlling how much you want to spend time crawling, maybe finding more niche information or more related information.
"""
site:https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=fal... recent developments in ai?
"""
Also when testing, if you know a piece of information exists in a website, but the information doesn't show up when you run the query, you don't have the tools to steer the engine to work more effectively. In a real scenario you don't know what the engine missed but it'd be cool to steer the engine in different ways to see how that changes the end result. For example, if you're planning a trip to japan, maybe you want the AI to only be shown a certain percentage of categories (nature, night life, or places too), alongside controlling how much you want to spend time crawling, maybe finding more niche information or more related information.
Yup, pasting in a URL will cause it to fetch the page.
Their agents can natively read files and webpages from URLs. It's so convenient, I've implemented identical feature for our product at work.
I'm a little confused about what the point of these are compared to the existing features/models that kagi already has. Are they just supposed to be a one-stop shop where I don't have to choose which model to use? When should I use kagi quick/research assistant instead of, e.g. kimi?
I tried the quick assistant a bit (don't have ultimate so I can't try research), and while the writing style seems slightly different, I don't see much difference in information compared to using existing models through the general kagi assistant interface.
I tried the quick assistant a bit (don't have ultimate so I can't try research), and while the writing style seems slightly different, I don't see much difference in information compared to using existing models through the general kagi assistant interface.
Quick assistant is a managed experience, so we can add features to it in a controlled way we can't for all the models we otherwise support at once.
For now Quick assistant has a "fast path" answer for simple queries. We can't support the upgrades we want to add in there on all the models because they differ in tool calling, citation reliability, context window, ability to not hallucinate, etc.
The responding model is currently qwen3-235B from cerebras but we want to decouple the user expectations from that so we can upgrade it down the road to something else. We like Kimi, but couldn't get a stable experience for Quick on it at launch with current providers (tool calling unreliability)
For now Quick assistant has a "fast path" answer for simple queries. We can't support the upgrades we want to add in there on all the models because they differ in tool calling, citation reliability, context window, ability to not hallucinate, etc.
The responding model is currently qwen3-235B from cerebras but we want to decouple the user expectations from that so we can upgrade it down the road to something else. We like Kimi, but couldn't get a stable experience for Quick on it at launch with current providers (tool calling unreliability)
That makes sense, thanks!
I'm seeing a lot of investment in these things that have a short shelf life.
Agents/assistants but nothing more.
Agents/assistants but nothing more.
We're building tools that we find useful, and we hope others find it too. See notes on our view of LLMs and their flaws:
https://blog.kagi.com/llms
https://blog.kagi.com/llms
I want a kagi mcp server I can use with ChatGPT or Claude.
I don’t want to use kagi ultimate (I use too many other features of ChatGPT and Claude), I just want to be able to improve the results of my AI models with kagi.
I don’t want to use kagi ultimate (I use too many other features of ChatGPT and Claude), I just want to be able to improve the results of my AI models with kagi.
I believe this is what you're after: https://github.com/kagisearch/kagimcp
What ChatGPT / Claude features do you use that we don't support?
We have an MCP server I can give you access to for search immediately. Down the line a search API and chat completions API to our assistant in the pipeline.
We have an MCP server I can give you access to for search immediately. Down the line a search API and chat completions API to our assistant in the pipeline.
Easier access to search api in general would be awesome. If I’m paying someone for search it may as well be kagi
Codex/claude code are the big ones.
Scheduled agents in ChatGPT.
I’ve been using the study mode recently, and that is nice to have.
Scheduled agents in ChatGPT.
I’ve been using the study mode recently, and that is nice to have.
I use perplexity a lot and pretty much exclusively with "deep research" on. Is this on the same level? Because Perplexity often take more than a minute, and this is only 20 secs.
For most queries it is around the same level. Time spent isn't always the best tell of quality, particularly when the search engine used returns little noise
I don't think the reasoning level is what makes it for me. I am treating this as a search engine. I want the thing to fetch/read from index as many pages as possible. Either that or as many agentic tool calls as needed. That takes time.
An employee above said they’re using cerebas so time may not be a good measure of thinking effort here
A query I ran earlier ran for about 70s, and did 12 web searches and 2 site fetches.
regular reminder: kagi is - above all else - a really really good search engine, and if google/etc, or even just the increasingly horrific ads-ocracy make you sad, you should definitely give it a go - the trial is here: https://kagi.com/pricing
if you like it, it's only $10/month, which I regrettably spend on coffee some days.
if you like it, it's only $10/month, which I regrettably spend on coffee some days.
> above all else
What they've been building for the past couple of years makes it blindingly clear that they are definitely not a search engine *above all else*.
Don't believe me? Check their CEO's goal: https://news.ycombinator.com/item?id=45998846
What they've been building for the past couple of years makes it blindingly clear that they are definitely not a search engine *above all else*.
Don't believe me? Check their CEO's goal: https://news.ycombinator.com/item?id=45998846
I think you have a problem.
I now that the price haven’t changed for a while, but I would pay for unlimited search and no AI.
Well, good news. Unlimited search was $10 before. Then they added the AI access for free but, and this part is key, did not make any requirement that you have to use it.
One thing I've wished for is the ability to use my kagi AI features in my editor (currently Zed)
We've got an assistant API coming soon™
I really wish Kagi would focus on search and not waste time and money on slop.
What they saying in this post is that they are designing these LLM-based features to support search.
The post describes how their use-case is finding high quality sources relevant to a query and providing summaries with references/links to the user (not generating long-form "research reports")
FWIW, this aligns with what I've found ChatGPT useful for: a better Google, rather than a robotic writer.
The post describes how their use-case is finding high quality sources relevant to a query and providing summaries with references/links to the user (not generating long-form "research reports")
FWIW, this aligns with what I've found ChatGPT useful for: a better Google, rather than a robotic writer.
I'm sure Google also says they built "AI mode" to "support search".
Their search is still trash.
Their search is still trash.
Except the AI mode filters out the bad results for you :)
I have a no-AI mode that filters out the bad results too. The problem is that it doesn't return any results at all, as it doesn't help with the harder problem of filtering out only the bad results without the good ones though. So far it's not clear to me that LLMs have significantly moved the needle on the ability to differentiate this.
In my experience the same slop garbage I get in search is the same slop garbage, only “summarized”, in AI mode.
If you look at my post history, I’m the last person to defend LLMs. That being said, I think LLMs are the next evolution in search. Not what OpenAI and Anthropic and xAI are working on - I think all the major models are moving further and further away from that with the “AI” stuff. But the core technology is an amazing way to search.
So I actually find it the perfect thing for Kagi to work with. If they can leverage LLMs to improve search, without getting distracted by the “AI” stuff, there’s tons of potential value,
Not saying that’s what this is… but if there’s any company I’d want playing with LLMs it’s probably Kagi
So I actually find it the perfect thing for Kagi to work with. If they can leverage LLMs to improve search, without getting distracted by the “AI” stuff, there’s tons of potential value,
Not saying that’s what this is… but if there’s any company I’d want playing with LLMs it’s probably Kagi
This is building on top of the existing core product, so the output is directly tied to the quality of their core search results being fed into the assistants. I overall really enjoy all of their A.I products, using their prompt assistant frequently for quick research tasks.
It does miss occasionally, or I feel like "that was a waste of tokens" due to a bad response or something, but overall I like supporting Kagi's current mission in the market of AI tools.
It does miss occasionally, or I feel like "that was a waste of tokens" due to a bad response or something, but overall I like supporting Kagi's current mission in the market of AI tools.
It's not -- this was posted literally yesterday as a position statement on the matter (see early paragraphs in OP):
https://blog.kagi.com/llms
Kagi is treating LLMs as potentially useful tools to be used with their deficiencies in mind, and with respect of user choices.
Also, we're explicitly fighting against slop:
https://blog.kagi.com/slopstop
https://blog.kagi.com/llms
Kagi is treating LLMs as potentially useful tools to be used with their deficiencies in mind, and with respect of user choices.
Also, we're explicitly fighting against slop:
https://blog.kagi.com/slopstop
Is there anyone selling LLM tools that would claim they aren't keeping their deficiencies in mind or admit that they're ignoring user choices? I'm not saying you are or aren't wasting money on slop, because I have no way of knowing, but it's hard to imagine someone who is concerned about a company acting in bad finding this compelling.
Same, though in fairness as long as they don't force it on me (the way Google does) and as long as the real search results don't suffer because of a lack of love (which so far they haven't), then it's no skin off my back. I think LLMs are an abysmal tool for finding information, but as long as the actual search feature is working well then I don't care if an LLM option exists.
[deleted]
The fact that people applaud Kagi taking the money they gave for search to invest it in bullshit AI products and spit on Google's AI search at the same time tells you everything you need to know about HackerNews.
Search is AI now, so I don’t get what your argument is.
Since 2019 Google and Bing both use BERT style encoder-only search architecture.
I’ve been using Kagi ki (now research assistant) for months and it is a fantastic product that genuinely improves the search experience.
So overall I’m quite happy they made these investments. When you look at Google and Perplexity this is largely the direction the industry is going.
They’re building tools on other LLMs and basically running open router or something behind the scenes. They even show you your token use/cost against your allowance/budget in the billing page so you know what you’re paying for. They’re not training their own from-scratch LLMs, which I would consider a waste of money at their size/scale.
Since 2019 Google and Bing both use BERT style encoder-only search architecture.
I’ve been using Kagi ki (now research assistant) for months and it is a fantastic product that genuinely improves the search experience.
So overall I’m quite happy they made these investments. When you look at Google and Perplexity this is largely the direction the industry is going.
They’re building tools on other LLMs and basically running open router or something behind the scenes. They even show you your token use/cost against your allowance/budget in the billing page so you know what you’re paying for. They’re not training their own from-scratch LLMs, which I would consider a waste of money at their size/scale.
We're not running on openrouter, that would break the privacy policy.
We get specific deals with providers and use different ones for production models.
We do train smaller scale stuff like query classification models (not trained on user queries, since I don't even have access to them!) but that's expected and trivially cheap.
We get specific deals with providers and use different ones for production models.
We do train smaller scale stuff like query classification models (not trained on user queries, since I don't even have access to them!) but that's expected and trivially cheap.
Do you have any evidence that the AI efforts are not being funded by the AI product, Kagi Assistant? I would expect the reverse: the high-margin AI products are likely cross-subsidizing the low-margin search products and their sliver of AI support.
High-margin AI products? Yes the world is just filled with those!
Our stuff is profitable.
Actually if you use LLMs sized responsibility to the task it's cheaper than a lot of APIs for the final product.
The expensive LLMs are expensive, but the cheap ones are cheaper than other infrastructure in something like quick answer or quick assistant
Actually if you use LLMs sized responsibility to the task it's cheaper than a lot of APIs for the final product.
The expensive LLMs are expensive, but the cheap ones are cheaper than other infrastructure in something like quick answer or quick assistant
We're explicitly conscious of the bullshit problem in AI and we try to focus on only building tools we find useful. See position statement on the matter yesterday:
https://blog.kagi.com/llms
https://blog.kagi.com/llms
> LLMs are bullshitters. But that doesn't mean they're not useful
> Note: This is a personal essay by Matt Ranger, Kagi’s head of ML
I appreciate the disclaimer, but never underestimate someone's inability to understand something, when their job depends on them not understanding it.
Bullshit isn't useful to me, I don't appreciate being lied to. You might find use in declaring the two different, but sufficiently advanced ignorance (or incompetence) is indistinguishable from actual malice, and thus they should be treated the same.
Your essay, while well written, doesn't do much to convince me any modern LLM has a net positive effect. If I have to duplicate all of it's research to verify none of it is bullshit, which will only be harder after using it given the anchoring and confirmation bias it will introduce... why?
> Note: This is a personal essay by Matt Ranger, Kagi’s head of ML
I appreciate the disclaimer, but never underestimate someone's inability to understand something, when their job depends on them not understanding it.
Bullshit isn't useful to me, I don't appreciate being lied to. You might find use in declaring the two different, but sufficiently advanced ignorance (or incompetence) is indistinguishable from actual malice, and thus they should be treated the same.
Your essay, while well written, doesn't do much to convince me any modern LLM has a net positive effect. If I have to duplicate all of it's research to verify none of it is bullshit, which will only be harder after using it given the anchoring and confirmation bias it will introduce... why?
I have not properly tested the new Kagi research agent yet, but here is a custom prompt I have been using in Kagi for the last year:
> which will only be harder after using it given the anchoring and confirmation bias it will introduce
This is a risk, but I have found that my own preconceptions are usually what need challenging, and a traditional search approach means I find what I wanted to find... so I use the research agent for an alternative perspective.
You are a research agent. Nobody cares what you think. You exist to do research on the internet. You search like hell and present the most relevant results that you can find. Results means lists of primary sources and the relevance to the query. If the question is about a specific place, don't forget to search in local languages as well as english. You don't ever suggest alternative search terms in your reply. Instead, you take those alternative search terms and you do more searches yourself until you get a wide range of answers.
> which will only be harder after using it given the anchoring and confirmation bias it will introduce
This is a risk, but I have found that my own preconceptions are usually what need challenging, and a traditional search approach means I find what I wanted to find... so I use the research agent for an alternative perspective.
That's a fair comment.
Just to give my point of view: I'm head of ML here, but I'm choosing to work here for the impact I believe I can have. I could work somewhere else.
As for the net positive effect, the point of my essay is that the trust relation you raise (not having to duplicate the research, etc) to me is a product design issue.
LLMs are fundamentally capable of bullshit. So products that leverage them have to keep that in mind to build workflows that don't end up breaking user trust in it.
The way we're currently thinking of doing that is to keep the user in the loop and incentivize the user to check sources by making it as easy as possible to quickly fact check LLM claims.
I'm on the same page as you that a model you can only trust 95% of the time is not useful because it's untrustable. So the product has to build an interaction flow that assumes that lack of trust but still makes something that is useful, saves time, respects user preferences, etc.
You're welcome to still think they're not useful for you, but that's the way we currently think about it and our goal is to make useful tools, not lofty promises of replacing humans at tasks.
Just to give my point of view: I'm head of ML here, but I'm choosing to work here for the impact I believe I can have. I could work somewhere else.
As for the net positive effect, the point of my essay is that the trust relation you raise (not having to duplicate the research, etc) to me is a product design issue.
LLMs are fundamentally capable of bullshit. So products that leverage them have to keep that in mind to build workflows that don't end up breaking user trust in it.
The way we're currently thinking of doing that is to keep the user in the loop and incentivize the user to check sources by making it as easy as possible to quickly fact check LLM claims.
I'm on the same page as you that a model you can only trust 95% of the time is not useful because it's untrustable. So the product has to build an interaction flow that assumes that lack of trust but still makes something that is useful, saves time, respects user preferences, etc.
You're welcome to still think they're not useful for you, but that's the way we currently think about it and our goal is to make useful tools, not lofty promises of replacing humans at tasks.
> That's a fair comment.
Equally, And despite my disagreement, I do genuinely appreciate the reply, especially given my dissent.
> Just to give my point of view: I'm head of ML here, but I'm choosing to work here for the impact I believe I can have. I could work somewhere else.
> As for the net positive effect, the point of my essay is that the trust relation you raise [...] to me is a product design issue.
Product design, or material appropriability issue? Why is the product you're trying to deliver, based ontop of a conversational model? I know why my rock climbing rope is a synthetic 10mm dynamic kernmantle rope, but why is a conversational AI the right product here?
> LLMs are fundamentally capable of bullshit.
Why though? I don't mean from a technical level, I do understand how next token prediction works. But from a product reasonability standpoint? Why are you attempting to build this product using a system that you makes predictions based on completely incorrect or inappropriate inputs?
I admittedly, am not up-to-date on state of the art, so please do correct me if my understanding is incomplete or wrong. But if I'm not mistaken, generally, attention based transformers themselves dont hallucinate when producing low heat language to language translations, right? Why are conversational models, the ones very much prone to hallucinations and emitting believable bullshit the interface everything uses?
How much of that reason is because that ability to emit believable bullshit, is actually the product you are trying to sell? (The rhetorical you, I'm specifically considering LLM as a service providers egear to over sell the capabilities of their model. I still have a positive opinion about Kagi, so I could be convinced you're the ones who are different) The artificial confidence is the product. Bullshitting something believeable but wrong has better results, in bulk, for the metrics you're tracking. When soliciting feedback, the vast majority of the answers are based on vibes, right?
If you had two models, one that was rote and very reliable, very predictable, rarely produced inaccurate output. But wasn't impressive when trying to generate conversational feeling text, and critically, was unable to phrase things in a trivial to understand way exuding an abundance of confidence. Contrasted with another that very very rarely would produce total bullshit, but all the feedback shows everyone loves using that model. But it makes them feel good about the answer, yet there's stil that nagging hallucination issue bubbling under the surface.
Which would you ship?
Again, I'm asking which would you ship with the rhetorical you... perhaps there is someone in charge of AI that would only ship the safe version, even if few users ranked it higher than normal organic search. Unfortunately I'm way too much of a cynic to believe that's possible. The AI is good crowd doesn't have a strong reputation for always making the ethical selection.
> So products that leverage them have to keep that in mind to build workflows that don't end up breaking user trust in it.
> The way we're currently thinking of doing that is to keep the user in the loop and incentivize the user to check sources by making it as easy as possible to quickly fact check LLM claims.
Do you feel that's a reasonable expectation from users when you've already given them the perfect answer they're looking for with plenty of subjective confidence?
> I'm on the same page as you that a model you can only trust 95% of the time is not useful because it's untrustable. So the product has to build an interaction flow that assumes that lack of trust but still makes something that is useful, saves time, respects user preferences, etc.
> You're welcome to still think they're not useful for you, but that's the way we currently think about it and our goal is to make useful tools, not lofty promises of replacing humans at tasks.
I don't think I'm the ideal person to be offering advice. Because I would never phrase the problem statement as "we have to give users to tools to verify if the confident sounding thing lied this time" I know far too much about both human nature, and alarm fatigue. So I can only reject your hypothetical, and ask what if you didn't have to do something I worry will make the world worse.
I attribute a large portion of the vitriol, anger, and divisiveness that has become pervasive, and is actively harming people and communities; as stemming directly from modern algorithmic recommendation systems. These systems prioritize speed, and being first, above the truth. Or they rank personalized results, that selectively offer only the content that feels good, and confirms preexisting ideas, to the detriment of reality.
They all tell you want you want to hear, over what is true. It will take a mountain of evidence to assure me, conversational LLMs wont do exactly the same thing, just better or faster. Especially when I could uncharitably summarize your solution to these defects, as merely "encouraging people to do their own research"
Equally, And despite my disagreement, I do genuinely appreciate the reply, especially given my dissent.
> Just to give my point of view: I'm head of ML here, but I'm choosing to work here for the impact I believe I can have. I could work somewhere else.
> As for the net positive effect, the point of my essay is that the trust relation you raise [...] to me is a product design issue.
Product design, or material appropriability issue? Why is the product you're trying to deliver, based ontop of a conversational model? I know why my rock climbing rope is a synthetic 10mm dynamic kernmantle rope, but why is a conversational AI the right product here?
> LLMs are fundamentally capable of bullshit.
Why though? I don't mean from a technical level, I do understand how next token prediction works. But from a product reasonability standpoint? Why are you attempting to build this product using a system that you makes predictions based on completely incorrect or inappropriate inputs?
I admittedly, am not up-to-date on state of the art, so please do correct me if my understanding is incomplete or wrong. But if I'm not mistaken, generally, attention based transformers themselves dont hallucinate when producing low heat language to language translations, right? Why are conversational models, the ones very much prone to hallucinations and emitting believable bullshit the interface everything uses?
How much of that reason is because that ability to emit believable bullshit, is actually the product you are trying to sell? (The rhetorical you, I'm specifically considering LLM as a service providers egear to over sell the capabilities of their model. I still have a positive opinion about Kagi, so I could be convinced you're the ones who are different) The artificial confidence is the product. Bullshitting something believeable but wrong has better results, in bulk, for the metrics you're tracking. When soliciting feedback, the vast majority of the answers are based on vibes, right?
If you had two models, one that was rote and very reliable, very predictable, rarely produced inaccurate output. But wasn't impressive when trying to generate conversational feeling text, and critically, was unable to phrase things in a trivial to understand way exuding an abundance of confidence. Contrasted with another that very very rarely would produce total bullshit, but all the feedback shows everyone loves using that model. But it makes them feel good about the answer, yet there's stil that nagging hallucination issue bubbling under the surface.
Which would you ship?
Again, I'm asking which would you ship with the rhetorical you... perhaps there is someone in charge of AI that would only ship the safe version, even if few users ranked it higher than normal organic search. Unfortunately I'm way too much of a cynic to believe that's possible. The AI is good crowd doesn't have a strong reputation for always making the ethical selection.
> So products that leverage them have to keep that in mind to build workflows that don't end up breaking user trust in it.
> The way we're currently thinking of doing that is to keep the user in the loop and incentivize the user to check sources by making it as easy as possible to quickly fact check LLM claims.
Do you feel that's a reasonable expectation from users when you've already given them the perfect answer they're looking for with plenty of subjective confidence?
> I'm on the same page as you that a model you can only trust 95% of the time is not useful because it's untrustable. So the product has to build an interaction flow that assumes that lack of trust but still makes something that is useful, saves time, respects user preferences, etc.
> You're welcome to still think they're not useful for you, but that's the way we currently think about it and our goal is to make useful tools, not lofty promises of replacing humans at tasks.
I don't think I'm the ideal person to be offering advice. Because I would never phrase the problem statement as "we have to give users to tools to verify if the confident sounding thing lied this time" I know far too much about both human nature, and alarm fatigue. So I can only reject your hypothetical, and ask what if you didn't have to do something I worry will make the world worse.
I attribute a large portion of the vitriol, anger, and divisiveness that has become pervasive, and is actively harming people and communities; as stemming directly from modern algorithmic recommendation systems. These systems prioritize speed, and being first, above the truth. Or they rank personalized results, that selectively offer only the content that feels good, and confirms preexisting ideas, to the detriment of reality.
They all tell you want you want to hear, over what is true. It will take a mountain of evidence to assure me, conversational LLMs wont do exactly the same thing, just better or faster. Especially when I could uncharitably summarize your solution to these defects, as merely "encouraging people to do their own research"
Your words don't match your actions.
And to be clear you shouldn't build the tools that YOU find useful, you should build the tools that your users, which pay for a specific product, find useful.
You could have LLMs that are actually 100% accurate in their answers that it would not matter at all to what I am raising here. People are NOT paying Kagi for bullshit AI tools, they're paying for search. If you think otherwise, prove it, make subscriptions entirely separate for both products.
And to be clear you shouldn't build the tools that YOU find useful, you should build the tools that your users, which pay for a specific product, find useful.
You could have LLMs that are actually 100% accurate in their answers that it would not matter at all to what I am raising here. People are NOT paying Kagi for bullshit AI tools, they're paying for search. If you think otherwise, prove it, make subscriptions entirely separate for both products.
Kagi founder here. We are moving to a future where these subscriptions will be separate. Even today more that 80% of our members use Kagi Assistant and our other AI-supported products so saying "people are NOT paying Kagi for bullshit AI tools" is not accurate, mostly in the sense that we are not in the business of creating bullshit tools. Life is too short for that. I also happen to like Star Trek version of the future, where smart computers we can talk to exist. I also like that Star Trek is still 90% human drama, and 10% technology quitely working in the background in service of humans - and this is the kind of future I would like to build towards and leave for my children. Having the most accurate search in the world that has users' best interest in mind is a big part of it, and that is not going anywhere.
edit: seeing the first two (negative) replies to my comment made me smile. HN is tough crowd to please :) The thing is similar to how I did paid search and went all in with my own money when everyone thought I was crazy, I did that out of own need and need for my family to have search done right and am doing the same now with AI, wanting to have it done right as a product. What you see here is the result of this group of humans that call themself Kagi best effort - not more, not less.
edit: seeing the first two (negative) replies to my comment made me smile. HN is tough crowd to please :) The thing is similar to how I did paid search and went all in with my own money when everyone thought I was crazy, I did that out of own need and need for my family to have search done right and am doing the same now with AI, wanting to have it done right as a product. What you see here is the result of this group of humans that call themself Kagi best effort - not more, not less.
Just wanted to chip in with a positive comment among the hail of negativity here. Thank you for what you and your team are doing. I've been getting tons of great use daily out of the search and news features, as well as occasionally using the assistant. It can definitely be hard to find decent paid alternatives to the freeware crap model so prevalent on the web, seeing your philosophy here is a huge breath of fresh air.
I found Kagi quite recently, and after blowing through my trial credits, and now almost blowing through my low tier (300) credits, I'm starting to look at the next tier up. However, it's approaching my threshold of value vs price.
I have my own payment methods for AI (OpenWebUI hosted on personal home server connected to OpenRouter API credits which costs me about $1-10 per month depnding on my usage), so seeing AI bundled with searches in the pricing for Kagi really just sucks the value out of the main reason I want to switch to Kagi.
I would love to be able to just buy credits freely (say 300 credits for $2-3) and just using them whenever. No AI stuff, no subscription, just pay for my searches. If I have a lull in my searches for a month, then a) no extra resources from Kagi have been spent, and b) my credits aren't used and rollover. Similarly, if I have a heavy search month, then I'll buy more and more credits.
I just don't want to buy extra AI on top of what I already have.
I have my own payment methods for AI (OpenWebUI hosted on personal home server connected to OpenRouter API credits which costs me about $1-10 per month depnding on my usage), so seeing AI bundled with searches in the pricing for Kagi really just sucks the value out of the main reason I want to switch to Kagi.
I would love to be able to just buy credits freely (say 300 credits for $2-3) and just using them whenever. No AI stuff, no subscription, just pay for my searches. If I have a lull in my searches for a month, then a) no extra resources from Kagi have been spent, and b) my credits aren't used and rollover. Similarly, if I have a heavy search month, then I'll buy more and more credits.
I just don't want to buy extra AI on top of what I already have.
Some people are contrarian simply to be contrarian. I'm loving the features that are coming out. I use Translate, Assistant, Universal Summarizer, and (of course) Search multiple times a day. Not everything is going to be for everyone, I'm personally not really interested in News, but it certainly feels like I'm getting value out of my subscription. The only thing I'm actively missing with Assistant is a proper app - the PWA is fine but the keyboard glitches out sometimes, navigation is not as fluid/smooth as it could be on a native app, same with file/photo uploads, and notifications when you leave the app and the reply is done would really tie it together.
> We are moving to a future where these subscriptions will be separate. Even today more that 80% of our members use Kagi Assistant and our other AI-supported products so saying "people are NOT paying Kagi for bullshit AI tools" is not accurate, mostly in the sense that we are not in the business of creating bullshit tools.
For what it's worth, as someone who tends to be pretty skeptical of introducing AI tools into my life, this statistic doesn't really convince me much of the utility of them. I'm not sure how to differentiate this from selection bias where users who don't want to use AI tools just don't subscribe in the first place rather than this being a signal that the AI tools are worthwhile for people outside of a niche group who are already interested enough to pay for them.
This isn't as strong a claim as what the parent comment was saying; it's not saying that the users you have don't want to be paying for AI tools, but it doesn't mean that there aren't people who are actively avoiding paying for them either. I don't pretend to have any sort of insight into whether this is a large enough group to be worth prioritizing, but I don't think the statement of your perspective here is going to be particularly compelling to anyone who doesn't already agree with you.
For what it's worth, as someone who tends to be pretty skeptical of introducing AI tools into my life, this statistic doesn't really convince me much of the utility of them. I'm not sure how to differentiate this from selection bias where users who don't want to use AI tools just don't subscribe in the first place rather than this being a signal that the AI tools are worthwhile for people outside of a niche group who are already interested enough to pay for them.
This isn't as strong a claim as what the parent comment was saying; it's not saying that the users you have don't want to be paying for AI tools, but it doesn't mean that there aren't people who are actively avoiding paying for them either. I don't pretend to have any sort of insight into whether this is a large enough group to be worth prioritizing, but I don't think the statement of your perspective here is going to be particularly compelling to anyone who doesn't already agree with you.
> I also happen to like Star Trek version of the future, where smart computers we can talk to exist [...], this is the kind of future I would like to build towards
Well if that doesn't seal the deal in making it clear that Kagi is not about search anymore, I don't know what does. Sad day for Kagi search users, wow!
> Having the most accurate search in the world that has users' best interest in mind is a big part of it
It's not, you're just trying to convince yourself it is.
Well if that doesn't seal the deal in making it clear that Kagi is not about search anymore, I don't know what does. Sad day for Kagi search users, wow!
> Having the most accurate search in the world that has users' best interest in mind is a big part of it
It's not, you're just trying to convince yourself it is.
For what it's worth, Kagi has been clear about this vision from the beginning.
https://blog.kagi.com/last-mile-for-web-search
https://blog.kagi.com/last-mile-for-web-search
People are very passionate about their views on LLMs. :)
I can't really do anything with the recommendation you're making.
The recommendation you made worked from your personal preference as an axiom.
The fact is that the APIs in search cost vastly more than the LLMs used in quick answer / quick assistant.
If you use the expensive AI stuff (research assistant or the big tier 1 models) that's expensive. But also: it is in a separate subscription, the $25/month one.
We used not to give any access to the assistant at the $5 and $10 tier, now we do, it's a free upgrades for users.
The recommendation you made worked from your personal preference as an axiom.
The fact is that the APIs in search cost vastly more than the LLMs used in quick answer / quick assistant.
If you use the expensive AI stuff (research assistant or the big tier 1 models) that's expensive. But also: it is in a separate subscription, the $25/month one.
We used not to give any access to the assistant at the $5 and $10 tier, now we do, it's a free upgrades for users.
Kagi is already expensive for a search engine. Now I know part of my subscription is going towards funding AI bullshit. And I know the cost of that AI bullshit will get jacked up in price and force Kagi sub price up as well. I'm so tired of AI being forced into everything.
These are only available on the Ultimate tier. If (like me) you don't care about the LLMs then there is no reason to be on the Ultimate tier so you don't pay for it.
>expensive for a search engine.
As in, not "free"?
Either way, I guess we'll see how this affects the service.
As in, not "free"?
Either way, I guess we'll see how this affects the service.
This is $10 USD a month to have a search engine. $5 is a more reasonable price but 300 searches is far to little to be useful. Converted to my currency this is over 2 hours of wages for a search engine.
I see. $10 is half an hour of minimum wage in my state, and around 1.25 hours of federal minimum wage. For anyone who hasn't been laid off in the US, it's a pittance for US developers.
Sounds like Kagi might need to implement some better regional pricing.
Sounds like Kagi might need to implement some better regional pricing.
Unfortunately they can’t implement regional pricing because they have very thin margins.
Awesome, a search engine with an LLM / AI!
I hadn't been sure about Kagi before, but this has really swung it for me, I'm off to sign up post haste. It's a revolutionary move that really shows how fast ahead of the competition Kagi is, how dexterous their fingers at the pulse of humanity, how bold.
I hadn't been sure about Kagi before, but this has really swung it for me, I'm off to sign up post haste. It's a revolutionary move that really shows how fast ahead of the competition Kagi is, how dexterous their fingers at the pulse of humanity, how bold.
Not for nothing, but I wish there was an anonymized ai built into a kagi that was able to have normal conversation discussion about sexual topics or search for pornographic topics like a safe search off function.
I understand the safety needs around things LLM should not build nuclear weapons, but it would be nice to have a frontier model that could write or find porn.
I understand the safety needs around things LLM should not build nuclear weapons, but it would be nice to have a frontier model that could write or find porn.
You'll want de-censored models like cydonia for that -- can be found on openrouter, or through something like msty
> We found many, many examples of benchmark tasks where the same model using Kagi Search as a backend outperformed other search engines, simply because Kagi Search either returned the relevant Wikipedia page higher, or because the other results were not polluting the model’s context window with more irrelevant data.
> This benchmark unwittingly showed us that Kagi Search is a better backend for LLM-based search than Google/Bing because we filter out the noise that confuses other models.