I hate to get bogged down in semantics, but with the hope that one of the stronger top-level critiques makes it into the top slot here and this conversation gets buried:
Cynicism is defined as
>An attitude of scornful or jaded negativity, especially a general distrust of the integrity or professed motives of others.
I'm not saying that cynicism is automatically wrong, just that I once could trust that, when HN is wrong, it is due to cynicism applied in excess.
Science means the pursuit of knowledge. It doesn't mean "only believing proven things". If we're going to be rude, lets at least take the time to be right.
I think you may be overindexing on the criticisms here. OpenAI has absolutely done impressive work in math already, and the criticisms are almost always based on the article that they initially published, usually available here in the HN comments within a few hours at most. Headlines will be headlines and hype guys will be hype guys, but OpenAI and Anthropic aren't lying and their bots are doing impressive work.
This one is a well-known problem with a brief, approachable proof, and they published the prompt.
I think, and I may be totally off base, that the labs are specifically avoiding art and (non-technical) writing as an endpoint. It's bad PR for them- it calls attention to the copyright question and threatens the 'human flourishing' kind of jobs- and there's no money in it because people prefer art to be human made and there's hardly any money in that anyway.
Have you heard of ACLU v. Clapper, HLP v. DoJ, Doe v. Ashcroft, or US v. Moalin? Parent acknowledges decades of degradation and points to thorns like these.
The difference is whether you say "thank you" to the government that does (a) or "screw you" to the government that fails to do (b). It's the difference between me not killing you and me granting you the right to be alive- if I am doing the latter, you are my subject rather than my equal. It implies that I have the right to take it away. Welfare, the FDA, and protectionism are examples of (a) while your basic human rights fall under (b).
Oh, believe me, I've been on the losing side of that issue with you for ages. I absolutely agree it is unethical for anybody to violate anybody's free speech, etc, and that that multiplies with corporate power.
But that means I'm familiar with the counterpoints. A government has the power to use violence: to operate a military that can kill non-citizens and a police force that can put citizens in prison. It's a lot more important to put a check on this than on a corp, even though quantity has a quality of its own. One thing to suppress dissidents by saying they can't use your website; another to put them in the gulags.
And then the other big issue is that corporations are just a bunch of people shaking hands. You have the right to free press, so you can write a newspaper that says what you like. You can sell that paper and you can publish people's op-eds if they give you the copyright permissions. You can refuse to publish the ones you disagree with. You can make agreements with the printing company to scale up and with other authors to contribute as you become popular, and, while this shifts the practical considerations, no amount of these agreements changes your principle right to free press.
For these reasons, it's best not to include this in your constitution. I have no idea what to do instead apart from shaming and boycotting unethical companies, which of course doesn't work when most people don't care an iota about the principle of free speech. Look at Athenian democracy and sigh?
An AI chatbot diagnosed my rolled shoulders. I had assumed I had bad workout form. I would google "discomfort in upper back" or whatever and it would say "back pain after exercise means take a rest" and I would rest and it wouldn't get better until I lost interest in working out. It never occurred to me that this could be a medical problem, so I never asked a doctor (not that I had one to ask, as I was working retail at the time). Last time I started working out, I went to ChatGPT instead of google and it said "here's your problem, here's how to fix it" and it was right and I continue to work out and my quality of life has risen tremendously. YMMV
That page specifically says the constitution grants rights to the government and reserves the rights of the people. There is a lot in there about the case against the bill of rights, a big part of which is that itemizing the rights implies that they are limited to the list.
Editing to clarify that this isn't just semantics: under the 'grant rights to the people' model, a government that grants one set of rights is just as legitimate as one that grants another. It was the position of the founders that governments which deny certain rights are infringing on the pre-existing rights of the people. This is the basis for their position on revolution.
They phrased it poorly, but from context it seems clear they intend the LLM as a less biased third-party measure of the tone which agrees with their own assessment.
I'm quite sure that LLMs are not one-to-one with our brains. It would at least be surprising if instrumental convergence was that powerful, and, even in that case, they're still not getting hungry or sleepy or having dreams, etc. At best they'd match up with a smaller region of our brains, and I'd take 9:1 odds against a close match.
But the paper argues a much stronger point than "LLMs are not similar to human brains": it claims LLMs cannot engage with meaning because they model human language instead of sensory input or human thought.
I also don't think human brains are particularly deterministic.
>It wouldn't be a bird anymore, but a dysfunctional cyborg with false perceptions.
The criticism in the paper is of the architecture of LLMs, isn't it? The paper contends
"""Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind.
[...] an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot"""
They're saying that the model cannot learn anything about reality irrespective of training data. Your point is an interesting one, but I think it's distinct. To your point though, this is unfalsifiable from the perspective of the "bird". I can't prove that I'm not a dysfunctional cyborg with false perceptions, which makes me wonder if that's a meaningful distinction.
>Animal sensory organs do not produce false information,
I happen to be in possession of some of these and I think this needs a "usually, under ordinary conditions". (Nitpicky and not critical to my point, but I liked the beginning of this sentence too much to edit it out)
>nor do they provide the brain an interpretation of what they perceive
Whereas this I'd argue is not true at all. My cones interact with wave-particle photons at particular wavelengths. I can't even conceptualize wave-particle duality (though some humans can), but "red" and "blue" are the bread and butter of my visual consciousness. These correspond to firings of my sensory neurons much more than they correspond to anything in reality. If that's not interpretation, what is it/what is interpretation?
>The word "real" is itself meaningless to it; they're both real in that they appear in its training corpus.
I expect we agree that I can show a multimodal model a real and a CGI picture and it can tell me which is which. I can take a CGI dragon to GPT 5 and say "look what I found in my backyard" and it will say "Yeah right". Are you saying this is something only possible thanks to RLHF or other modern techniques? That may be the case, unsure how to test that without access to pretraining-only models. Or would you say my experiment is faulty here and doesn't get to your underlying claim?
On the flipside, I could show some meh drawings of fairies to Arthur Conan Doyle, and he'd say "Whoa, this changes everything". I consider him to be one of the great rational minds of history, but he was unable to pass your test here. (In fairness, he was in his 60s and his senses may have dulled, though his belief in spiritualism at large dates to his prime).
A bird doesn't learn gravity or aerodynamics, it has no 'sense of physics'. It has sensory neural activity that a scientist can show is tied to these things, but you could, at least in theory, falsify the entire experience of the bird. There is nothing in a bird's brain that directly percieves reality. Colors and sounds and textures and so on are all false primitives that don't exist in nature without us, if that's clarifying. And because evolution acts through organisms, its only access to base reality is through their senses. After thousands of years of effort, we have some pretty good models of what reality actually is, but they're still not 'grounded' in the sense you describe.
If someone substituted all of your sensory inputs for something else for your entire life, how would you notice? If you wore contacts from birth that made the sky red and earbuds that censored when people said it was blue, on what basis would you realize that was wrong? I don't see what this says about the architecture of your brain, and I don't think it's the point being made in the paper. That the training data must statistically connect to reality in order for the model to model reality doesn't seem that important.
Fair point, and on its own it would be surprising to learn what "five" means from that sentence. But you can extrapolate- across a billion sentences, there will be "the next sentence has five words"s and "this sentence are grammared wrong" and so on. It would not be at all impossible to ground a world model on pure text for that reason. And 'not impossible' is sufficient to invalidate the paper's argument.
I don't think this tone is at all justified. If you think otherwise, I do ask that you point out where I went too far in a comment that I feared was overburdened by caveats and admissions of my own human flaws.
"This sentence has five words" is going to appear far more often than "This sentence has four words". This is the entire premise of LLMs working at all, stochastic parrots or otherwise.
The contention that there is no grounding because the training data is linguistic and thus can only reference a world model is disproven in "This sentence has five words"- there's real, grounded information about what "five" means within that sentence. While that's a trivial counterexample, I don't know that it's an obvious one (I didn't come up with it myself).
It's not a criticism of the paper itself, but multimodal models came shortly after and provide grounding that is more of the sort the paper is getting at, and it didn't seem like anybody updated on that at all. If multimodal models were still stochastic parrots by the original argument, humans would have to be as well; we don't have any way to ground anything beneath sense data and evolution can't have programmed some innate grounding into us because it didn't either. But (and maybe this is my own misperception) nobody threw in the towel at that point.
I confess I never read the original paper until now, opting to absorb by osmosis instead, and I was quite surprised that they don't really make a deeper case than that. After just a few paragraphs about how they can't be grounded because humans don't express their thoughts directly, it lurches into a page about how they can be biased by training. And they certainly can be, but that has little to say about their stochastic nature- humans are biased as a rule with no exception. (For the record, I only read the Stochastic Parrots section before this reply.)
It's not really a bad paper, but I don't see why it ever carried the esteem it did. Hating on it is like hating on Taylor Swift- she's fine, yes, but for her level of success, one is inclined to question every dumb lyric where others get a pass. (Apologies to Swift fans, substitute a successful artist you don't care for here.)
This is not why she was fired and wouldn't have been a plausible reason in 2020 when it happened.
>Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.
This is Google's side of it; I think the following is a fair piece of primary-source journalism if you want to go deeper:
Cynicism is defined as
>An attitude of scornful or jaded negativity, especially a general distrust of the integrity or professed motives of others.
I'm not saying that cynicism is automatically wrong, just that I once could trust that, when HN is wrong, it is due to cynicism applied in excess.