Hallucination is a problem we'll have to live with for a long time(theregister.com)
theregister.com
Hallucination is a problem we'll have to live with for a long time
https://www.theregister.com/2025/01/07/interview_with_aws_byron_cook/
11 comments
I hate neologisms like hallucinate. Terms of art are fine for cognoscenti, but are misunderstood by naive readers as implying aspects of intelligence. It's just statistics ma'am.
I find it's a perfectly fine word to describe the result. Humans do the same as our visual system samples a low amount of data points to construct the view we see. In effect we're always hallucinating with the difference being that we maintain high context to filter it to the correct hallucinations. This shows up in dreams where we don't maintain such context, or when context is largely changed. When I got back from a vacation where there were many small and larger lizards, I swear I saw one upon returning, but it turned out to be a similarly colored leaf moving in a scurrying motion.
Edit: "mis-remembering" is another good term as it's using its vast training to output tokens and sometimes it maps them wrong.
Edit: "mis-remembering" is another good term as it's using its vast training to output tokens and sometimes it maps them wrong.
My problem isn't the word per se, but the way it gets used to suggest it is an exceptional case rather than standard.
Perhaps true. But if you are warning against the dangers of so-called "AI" - "hallucinate" is also suggestive of "dangerous", "insane", and some other strong negatives.
That's reaching for a positive in the negative to me, but undeniable. So .. I dunno you win!
I continue to respect Hinton, he uses and chooses his words carefully I think.
I continue to respect Hinton, he uses and chooses his words carefully I think.
“It’s just completely obvious that within five years deep learning is going to do better than radiologists.… It might be 10 years, but we’ve got plenty of radiologists already.” - Geoffrey Hinton 2016 [1]
[1] https://newrepublic.com/article/187203/ai-radiology-geoffrey...
I agree, lots of the body of human knowledge is wrapped up in natural reactions, moving visuals, textures, smells, and sounds.
The current batch is trained on just text afaik.
The current batch is trained on just text afaik.
People... employees... friends... lovers... "hallucinate" too.
What rational agent is infallible?
What rational agent is infallible?
LLMs don’t actually “hallucinate”. Hallucination would be the LLM starting from a different context than was actually given to it. Given the fidelity of electronic transmission these days, that’s probably never an issue.
LLMs also have no grounding in abstract concepts such as true and false. This means they output things stochastically rather than logically. Sometimes people are illogical, but people are also durable, learn over time, and learn extremely quickly. Current LLMs only learn once, so they easily get stuck in loops and pitfalls when they produce output that makes no sense to the human reader. The LLM can’t “understand” that the output makes no sense because it doesn’t “understand” anything in the sense that humans understand things.
LLMs also have no grounding in abstract concepts such as true and false. This means they output things stochastically rather than logically. Sometimes people are illogical, but people are also durable, learn over time, and learn extremely quickly. Current LLMs only learn once, so they easily get stuck in loops and pitfalls when they produce output that makes no sense to the human reader. The LLM can’t “understand” that the output makes no sense because it doesn’t “understand” anything in the sense that humans understand things.
No one is claiming that any agent is or should be considered infallible, but for every other form of technology humans create - including software - there is a minimal standard of predictability and efficiency that is considered acceptable for that software to be useful.
For some reason, LLMs are the exception. It doesn't matter how much they hallucinate, confabulate, what have you, someone will always, almost reflexively, dismiss any criticism as irrelevant because "humans do the same thing." Even though human beings who hallucinate as often as LLMs do would be committed to asylums.
In general terms, the more mission critical a technology is, the more reliable it needs to be. Given that we appear to intend to integrate LLMs into every aspect of human society as aggressively as possible, I don't believe it's unreasonable to expect it to be more reliable than a sociopathic dementia patient with Munchausen's syndrome.
But that's just me. I don't look forward to the future when my prescriptions are written by software agents that tend to make up illnesses and symptoms and filled out by software agents that cant' do basic math, and it's all considered ok because the premise that humans would always be as bad or worse and shouldn't be trusted with even basic autonomy has become so normalized we just accept the abuse of the unstable technologies rolled out to deprecate us from society as inevitable. Apparently that just makes me a Luddite. IDK.
For some reason, LLMs are the exception. It doesn't matter how much they hallucinate, confabulate, what have you, someone will always, almost reflexively, dismiss any criticism as irrelevant because "humans do the same thing." Even though human beings who hallucinate as often as LLMs do would be committed to asylums.
In general terms, the more mission critical a technology is, the more reliable it needs to be. Given that we appear to intend to integrate LLMs into every aspect of human society as aggressively as possible, I don't believe it's unreasonable to expect it to be more reliable than a sociopathic dementia patient with Munchausen's syndrome.
But that's just me. I don't look forward to the future when my prescriptions are written by software agents that tend to make up illnesses and symptoms and filled out by software agents that cant' do basic math, and it's all considered ok because the premise that humans would always be as bad or worse and shouldn't be trusted with even basic autonomy has become so normalized we just accept the abuse of the unstable technologies rolled out to deprecate us from society as inevitable. Apparently that just makes me a Luddite. IDK.
Both falsely imply that there's a solvable mechanical difference going on between results people like versus results people dislike.