Is accuracy is the next big AI challenge? Why aren't companies focusing on this?
4 comments
For the same reason they don't focus on software quality, simplicity, or reliability - those things cost money and require good ideas and critical thinking. Too hard. Much easier to sell slickly-marketed garbage.
I would love to be proven wrong, but if you ignore all the hype pieces and (from what I can tell) a TON of pseudo-scientific shit floating around, while it's possible to make these systems as accurate as possible, they would still be fundamentally unreliable since the systems aren't deterministic. Try asking the usual "is 9.11 greater than 9.9" question, on my last test with 3.7 Sonnet it first said "yes" and then eventually converged to "no". Quite hilarious at times.
Point being, LLMs don't really understand what they're saying; they're just predicting the next word. I'm not trying to downplay that, by the way - it's all very impressive. But it's not intelligent intelligent, and I think we should all be wary of comments like "oh we should just keep throwing more compute and data at it and consciousness/self awareness would eventually emerge". I think that's just fodder to raise/burn VC money (and the planet).
Point being, LLMs don't really understand what they're saying; they're just predicting the next word. I'm not trying to downplay that, by the way - it's all very impressive. But it's not intelligent intelligent, and I think we should all be wary of comments like "oh we should just keep throwing more compute and data at it and consciousness/self awareness would eventually emerge". I think that's just fodder to raise/burn VC money (and the planet).
Conciousness/self awareness can emerge if they put more effort into accuracy and reasoning, ie the human touch. I agree with you on most companies thinking that AI can arrive to its own reasoning/conciousness. The hype is getting to their brains, lol.
[deleted]