Life is more than an engineering problem(lareviewofbooks.org)
lareviewofbooks.org
Life is more than an engineering problem
https://lareviewofbooks.org/article/life-is-more-than-an-engineering-problem/
256 comments
Counter point: what is it about scraping the Internet and indexing it cleverly that makes you believe that would lead to the the creation of the ability to reason above it's programming?
No one in neuroscience, psychology or any related field can point to reasoning or 'consciousness' or whatever you wish to call it and say it appeared from X. Yet we have this West Coast IT cultish thinking that if we throw money at it we'll just spontaneously get there. The idea that we're even 1% close should be ridiculous to anyone rationally looking at what we're currently doing.
No one in neuroscience, psychology or any related field can point to reasoning or 'consciousness' or whatever you wish to call it and say it appeared from X. Yet we have this West Coast IT cultish thinking that if we throw money at it we'll just spontaneously get there. The idea that we're even 1% close should be ridiculous to anyone rationally looking at what we're currently doing.
> No one in neuroscience, psychology or any related field can point to reasoning or 'consciousness' or whatever you wish to call it and say it appeared from X.
This is not a good argument. Natural systems, the subject of neuroscience/psychology, are much harder to analyze than artificial systems. For example, it's really difficult to study atmospheric gases and figure out Boyle's/Charles law. But put a gas in a closed chamber and change pressure or temperature and these laws are trivially apparent.
LLMs are much more legible systems than animal brains, and they are amenable to experiment. So, it is much more likely that we will be able to identify what "reasoning" is by studying these systems than animal brains.
P.S. Don't think we are there yet, as much as internet commentators might assert.
This is not a good argument. Natural systems, the subject of neuroscience/psychology, are much harder to analyze than artificial systems. For example, it's really difficult to study atmospheric gases and figure out Boyle's/Charles law. But put a gas in a closed chamber and change pressure or temperature and these laws are trivially apparent.
LLMs are much more legible systems than animal brains, and they are amenable to experiment. So, it is much more likely that we will be able to identify what "reasoning" is by studying these systems than animal brains.
P.S. Don't think we are there yet, as much as internet commentators might assert.
Yea but following your example/analogy you have gas-gas but brain-llm. So how can we then experiment? It's a simulation at best.
Both jets and birds fly but do it in a completely different way. Who said that there's only one way to achieve reasoning?
This feels like an appropriate place to share this again:
> "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
> "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
Parrots can both fly and talk, what about that!?
This paper may be interesting to some of you:
Discretization of continuous input spaces in the hippocampal autoencoder
https://arxiv.org/pdf/2405.14600
Discretization of continuous input spaces in the hippocampal autoencoder
https://arxiv.org/pdf/2405.14600
I think it's really up to the highly nebulous definition. Even in your comment is implied that reasoning and consciousness are two names of the same thing but i'd argue one is here and one will never be provable ever. Reason is working through logical steps, much like a program. It's a set of conditions that get checked and a logical structure that uses that information to reach a conclusion. That's what sets it apart from gut feelings or emotional thinking, it's a traceable structure with "reasons". I can watch the LLM speak base facts out loud, then begin to synthesize them giving _reasons_ for the choices it's making, culminating in a final conclusion. It's already doing that. That is what i call reason. It doesn't mean it's human, it doesn't mean it's "aware of itself", it just means it's thinking a train of thought with concrete steps between each car. Consciousness is completely undefinable and useless as a metric and will never be probably achieved.
I agree that reasoning and consciousness are different, however what I do not see being discussed by the AI research community is the necessity to define and then develop "artificial comprehension".
At this point in time, the act of comprehension is a scientific mystery.
I'd say 'consciousness' is the ongoing ever present comprehension of the moment, a feedback self conversation assessing the current situation a being finds itself. This act requires reasoning, as comprehension is the "sandbox" in which reasoning occurs.
But what is comprehension? It's the instantaneous reverse engineering of observations for verification of reality: is what I observe normal, possible or a threat? If one cannot "understand" an observation then the potential the observation is a threat grows. That 'understanding" is reverse engineering the observation to identify it's range of possible behavior and therefore one's safety in relation to that observation.
Comprehension is extremely complex: arbitrary input goes in and a world model with one's safety and next actions comes out.
At this point in time, the act of comprehension is a scientific mystery.
I'd say 'consciousness' is the ongoing ever present comprehension of the moment, a feedback self conversation assessing the current situation a being finds itself. This act requires reasoning, as comprehension is the "sandbox" in which reasoning occurs.
But what is comprehension? It's the instantaneous reverse engineering of observations for verification of reality: is what I observe normal, possible or a threat? If one cannot "understand" an observation then the potential the observation is a threat grows. That 'understanding" is reverse engineering the observation to identify it's range of possible behavior and therefore one's safety in relation to that observation.
Comprehension is extremely complex: arbitrary input goes in and a world model with one's safety and next actions comes out.
Thanks for this. Do you have a blog somewhere, preferably with an RSS feed?
I have a never updated blog, but I'm not an active research scientist. I'm just a plain ordinary over educated guy, whose been writing software using AI, across all the things that have been called "AI" for about 45 years. One could say I'm "over read", at one point I'd read every single Nobel Literature winner, I have finished dozens of authors, and my personal taste is mind fuck philosophy in narrative fiction - think Clockwork Orange, Philip K Dick, and beatnik literature. I post a lot of my opinions at Quora: https://www.quora.com/profile/Blake-Senftner
I don't care about qualifications or job titles, if I read a solid piece of text that makes me think differently, I want to know more. ;)
I bookmarked your Quora page and blog in my RSS reader, so if you ever start blogging... And thanks for pointing to Philip K Dick, I might actually start reading science fiction.
The assumption is that since there is already a neural network that “got there” (our brains), we should be able to achieve the same thing synthetically.
We just need to figure out how to train that network.
We just need to figure out how to train that network.
Neural networks are a simplification of our brains, they are not a replication of it. It is just a modeling method that was inspired by how human neurons work, that's it. It's not 1 to 1 or anything.
Furthermore, neurons alone do not led to consciousness. At the very least, their modulation, mainly by glial cells, is essential as well.
Personally, my money is on quantum coherence within microtubules being the mechanism of conscious experience, with the brain essentially being a quantum/classical hybrid computer.
Personally, my money is on quantum coherence within microtubules being the mechanism of conscious experience, with the brain essentially being a quantum/classical hybrid computer.
It may be possible to argue that current work in AI leads to some definition of intelligence, which apparently often is equaled to consciousness by some.
My take it is just unaware intelligence like in Peter Watts’ book Blindsight. A terrific read and a quite scary prospect.
My take it is just unaware intelligence like in Peter Watts’ book Blindsight. A terrific read and a quite scary prospect.
It's more that if you actually work with LLMs they will display reasoning. It's not particularly good or deep reasoning (I would generally say they have a superhuman amount of knowledge but are really quite unintelligent), but it is more than simply recall.
Waters are often muddied here by our own psychology. We (as a species) tend to ascribe intelligence to things that can speak. Even more so when someone (or thing in this case) can not just speak, but articulate well.
We know these are algorithms, but how many people fall in love or make friends over nothing but a letter or text message?
Capabilities for reasoning aside, we should all be very careful of our perceptions of intelligence based solely on a machines or algorithms apparent ability to communicate.
We know these are algorithms, but how many people fall in love or make friends over nothing but a letter or text message?
Capabilities for reasoning aside, we should all be very careful of our perceptions of intelligence based solely on a machines or algorithms apparent ability to communicate.
>we should all be very careful of our perceptions of intelligence based solely on a machines or algorithms apparent ability to communicate.
I don't think that's merely an irrational compulsion. Communication can immediately demonstrate intelligence, and I think it quite clearly has, in numerous ways. The benchmarks out there cover a reasonable range of measurements that aren't subjective, and there's clear yes-or-no answers to whether the communication is showing real ways to solve problems (e.g. change a tire, write lines of code, solving word problems, critiquing essays), where the output proves it in the first instance.
Where there's an open question is in whether you're commingling the notion of intelligence with consciousness, or identifying intelligence with AGI, or with "human like" uniqueness, or some other special ingredient. I think your warning is important and valid in many contexts (people tend to get carried away when discussing plant "intelligence", and earlier versions of "AI" like Eliza were not the real deal, and Sophia the robot "granted citizenship" was a joke).
But this is not a case, I think where it's a matter of intuitions leading us astray.
I don't think that's merely an irrational compulsion. Communication can immediately demonstrate intelligence, and I think it quite clearly has, in numerous ways. The benchmarks out there cover a reasonable range of measurements that aren't subjective, and there's clear yes-or-no answers to whether the communication is showing real ways to solve problems (e.g. change a tire, write lines of code, solving word problems, critiquing essays), where the output proves it in the first instance.
Where there's an open question is in whether you're commingling the notion of intelligence with consciousness, or identifying intelligence with AGI, or with "human like" uniqueness, or some other special ingredient. I think your warning is important and valid in many contexts (people tend to get carried away when discussing plant "intelligence", and earlier versions of "AI" like Eliza were not the real deal, and Sophia the robot "granted citizenship" was a joke).
But this is not a case, I think where it's a matter of intuitions leading us astray.
> Where there's an open question is in whether you're commingling the notion of intelligence with consciousness
I’m absolutely commingling these two things and that is an excellent point.
Markov chains and other algorithms that can generate text can give the appearance of intelligence without any kind of understanding or consciousness.
I’m not personally certain of consciousness is even requisite for intelligence, given that as far as we know consciousness is an emergent property stemming from some level of problem solving ability.
I’m absolutely commingling these two things and that is an excellent point.
Markov chains and other algorithms that can generate text can give the appearance of intelligence without any kind of understanding or consciousness.
I’m not personally certain of consciousness is even requisite for intelligence, given that as far as we know consciousness is an emergent property stemming from some level of problem solving ability.
This seems like the classic shifting of goalposts to determine when AI has actually become intelligent. Is the ability to communicate not a form of intelligence? We don't have to pretend like these models are super intelligent, but to deny them any intelligence seems too far for me.
My intent was not to claim communication isn’t a sign of intelligence, but that the appearance of communication and our tendency to anthropomorphize behaviors that are similar to ours can result in misunderstandings as to the current capabilities of LLMs.
glenstein made a good point that I was commingling concepts of intelligence and consciousness. I think his commentary is really insightful here: https://news.ycombinator.com/item?id=42912765
glenstein made a good point that I was commingling concepts of intelligence and consciousness. I think his commentary is really insightful here: https://news.ycombinator.com/item?id=42912765
[deleted]
AI certainly won't be intelligent while it has episodic responses to queries with no ability to learn from or even remember the conversation without it being fed back through as context. This is the current case for LLM models. Token prediction != Intelligence no matter how intelligent it may seem. I would say adaptability is a fundamental requirement of intelligence.
>AI certainly won't be intelligent while it has episodic responses to queries with no ability to learn from or even remember the conversation without it being fed back through as context.
Thank God no one at the AI labs is working to remove that limitation!
Thank God no one at the AI labs is working to remove that limitation!
And yet, it is still a current limitation and relevant to all current claims of LLM intelligence.
The guy in memento is clearly still an intelligent human despite having no memory. These arguments always strike me as coming from a "humans are just special okay!" place. Why are you so determined to find some way in which LLMs aren't intelligent? Why gatekeep so much?
I mean humans have short term and long term memory, short term memory is just our context window.
Are they displaying reasoning, or the outcome of reasoning, leading you to a false conclusion?
Personally, I see ChatGPT say "water doesn't freeze at 27 degrees F" and think "how can it possibly do advanced reasoning when it can't do basic reasoning?"
Personally, I see ChatGPT say "water doesn't freeze at 27 degrees F" and think "how can it possibly do advanced reasoning when it can't do basic reasoning?"
I'm not saying it reasons reliably, at all (nor has much success with anything particularly deep: I think in a lot of cases it's dumber than a lot of animals in this respect). But it does a form of general reasoning which other more focused AI efforts have generally struggled with, and it's a lot more successful than random chance. For example, see how ChatGPT can be persuaded to play chess. It still will try to make illegal moves sometimes, hallucinating pieces in the board state or otherwise losing the plot. But if you constrain it and only consider the legal moves, it'll usually beat the average person (i.e. someone who understands the rules but has very little experience), even if it'll be trounced by an experienced player. You can't do this just by memorisation or random guessing: chess goes off-book (i.e. into a game state that has never existed before) very quickly, so it must have some understanding of chess and how to reason about the moves to make, even if it doesn't color within the lines as well as a comparatively basic chess engine.
(Basically, I don't think there's a bright line here: saying "they can't reason" isn't very useful, instead it's more useful to talk about what kinds of things they can reason about, and how reliably. Because it's kind of amazing that this is an emergent behaviour of training on text prediction, but on the other hand because prediction is the objective function of the training, it's a very fuzzy kind of reasoning and it's not obvious how to make it more rigourous or deeper in practice)
(Basically, I don't think there's a bright line here: saying "they can't reason" isn't very useful, instead it's more useful to talk about what kinds of things they can reason about, and how reliably. Because it's kind of amazing that this is an emergent behaviour of training on text prediction, but on the other hand because prediction is the objective function of the training, it's a very fuzzy kind of reasoning and it's not obvious how to make it more rigourous or deeper in practice)
This is the most pervasive bait-and-switch when discussing AI: "it's general reasoning."
When you ask an LLM "what is 2 + 2?" and it says "2 + 2 = 4", it looks like it's recognizing two numbers and the addition operation, and performing a calculation. It's not. It's finding a common response in its training data and returning that. That's why you get hallucinations on any uncommon math question, like multiplying two random 5 digit numbers. It's not carrying out the logical operations, it's trying to extract the an answer by next token prediction. That's not reasoning.
When you ask "will water freeze at 27F?" and it replies "No, the freezing point of water is 32F", what's happening is that it's not recognizing the 27 and 32 are numbers, that a freezing point is an upper threshold, and that any temperature lower than that threshold will therefore also be freezing. It's looking up the next token and finding nothing about how 27F is below freezing.
Again, it's not reasoning. It's not exercising any logic. Its huge training data set and tuned proximity matching helps it find likely responses, and when it seems right, that's about the token relationship pre-existing in the training data set.
That it occasionally breaks the rules of chess just shows it has no concept of those rules, only that the next token for a chess move is most likely legal because most of its chess training data is of legal games, not illegal moves. I'm unsurprised to find that it can beat an average player if it doesn't break the rules: most chess information in the world is about better than average play.
If an LLM came up with a proof no one had seen, but it checks out, that doesn't prove it's reasoning either, just because it's next token prediction that came up with it. It found token relationships no one had noticed before, but that's inherent in the training data, and not a reflective intelligence doing logic.
When we discuss things like reinforcement learning and chain of reasoning, what we're really talking about are ways of restricting/strengthening those token relationships. It's back-tuning of the training data. Still not doing logic.
When you ask an LLM "what is 2 + 2?" and it says "2 + 2 = 4", it looks like it's recognizing two numbers and the addition operation, and performing a calculation. It's not. It's finding a common response in its training data and returning that. That's why you get hallucinations on any uncommon math question, like multiplying two random 5 digit numbers. It's not carrying out the logical operations, it's trying to extract the an answer by next token prediction. That's not reasoning.
When you ask "will water freeze at 27F?" and it replies "No, the freezing point of water is 32F", what's happening is that it's not recognizing the 27 and 32 are numbers, that a freezing point is an upper threshold, and that any temperature lower than that threshold will therefore also be freezing. It's looking up the next token and finding nothing about how 27F is below freezing.
Again, it's not reasoning. It's not exercising any logic. Its huge training data set and tuned proximity matching helps it find likely responses, and when it seems right, that's about the token relationship pre-existing in the training data set.
That it occasionally breaks the rules of chess just shows it has no concept of those rules, only that the next token for a chess move is most likely legal because most of its chess training data is of legal games, not illegal moves. I'm unsurprised to find that it can beat an average player if it doesn't break the rules: most chess information in the world is about better than average play.
If an LLM came up with a proof no one had seen, but it checks out, that doesn't prove it's reasoning either, just because it's next token prediction that came up with it. It found token relationships no one had noticed before, but that's inherent in the training data, and not a reflective intelligence doing logic.
When we discuss things like reinforcement learning and chain of reasoning, what we're really talking about are ways of restricting/strengthening those token relationships. It's back-tuning of the training data. Still not doing logic.
Put more succinctly: if it came up with a new proof in math that was then verified, and you went back and said "no, that's wrong" it would immediately present a different proof, denying the validity of its first proof, because it didn't construct anything logical that it can stand on and say "no, I'm right".
These are all examples of how they're not very good at reasoning, not that they don't reason at all. Being a perfectly consistent logical process is not a requirement for reasoning.
You're begging the question by using the term "reasoning". In what sense are they reasoning if they're not using any logic at all?
I don't think any of us are qualified to tell the difference between exhibiting reasoning and mixing examples taken from the entire internet. Maybe if the training data was small enough to comprehend in its entirety, we could say one way or the other, but as it stands none of us have read the enitre internet, and we have no way of finding the stackoverflow or Reddit conversation that most closely resembles a given chain of thought.
Yes, my judgement too from messing with Claude and (previously) ChatGPT. 'Ridiculous' and 'cultish' are overton-window enforcement more than they are justified.
From its answers I already conclude it is already reasoning above its programming. I do not see why someone in neuroscience or psychology would need to say it appeared, since they do not know better what reasoning is than any average human.
Reasoning is undefined, but a human recognizes it when it appears. I don't see consciousness part of that story. Also, whether you call it emulated or played reasoning or not, apparently does not matter. The results are what they are.
Reasoning is undefined, but a human recognizes it when it appears. I don't see consciousness part of that story. Also, whether you call it emulated or played reasoning or not, apparently does not matter. The results are what they are.
If I write a book that contains Einstein's theory of relativity by virtue of me copying it, did I create the theory? Did my copying of it indicate anything about my understanding of it? Would you be justified to think the next book I write would have anything of original value?
I think what he is trying to say is that LLMs current architecture seems to mainly work by understanding patterns in the existing body of knowledge. In some senses finding patterns could be considered creative and entail reasoning. And that might be the degree to which LLMs could be said to be capable of reasoning or creativity.
But it is clear humans are capable of creativity and reasoning that are not reducible to mere pattern matching and this is the sense of reasoning that LLMs are not currently capable of.
I think what he is trying to say is that LLMs current architecture seems to mainly work by understanding patterns in the existing body of knowledge. In some senses finding patterns could be considered creative and entail reasoning. And that might be the degree to which LLMs could be said to be capable of reasoning or creativity.
But it is clear humans are capable of creativity and reasoning that are not reducible to mere pattern matching and this is the sense of reasoning that LLMs are not currently capable of.
> If I write a book that contains Einstein's theory of relativity by virtue of me copying it, did I create the theory? Did my copying of it indicate anything about my understanding of it? Would you be justified to think the next book I write would have anything of original value?
No, but you described a `cp` command, not an LLM.
"Creativity" in the sense of coming up with something new is trivial to implement in computers, and has long been solved. Take some pattern - of words, of data, of thought. Perturb it randomly. Done. That's creativity.
The part that makes "creativity" in the sense we normally understand it hard, isn't the search for new ideas - it's evaluation of those ideas. For an idea to be considered creative, it has to match a very complex... wait for it... pattern.
That pattern - what we call "creative" - has no strict definition. The idea has to be close enough to something we know, so we can frame it, yet different enough from it as to not be obvious, but still not too different, so we can still comprehend it. It has to make sense in relevant context - e.g. a creative mathematical proof has to still be correct (or a creative approach to proving a theorem has to plausibly look like it could possibly work); creative writing still has to be readable, etc.
The core of creativity is this unspecified pattern that things we consider "creative" match. And it so happens that things matching this pattern are a match for pattern "what makes sense for a human to read" in situations where a creative solution is called for. And the latter pattern - "response has to be sensible to a human" - is exactly what the LLM goal function is.
Thus follows that real creativity is part of what LLMs are being optimized for :).
No, but you described a `cp` command, not an LLM.
"Creativity" in the sense of coming up with something new is trivial to implement in computers, and has long been solved. Take some pattern - of words, of data, of thought. Perturb it randomly. Done. That's creativity.
The part that makes "creativity" in the sense we normally understand it hard, isn't the search for new ideas - it's evaluation of those ideas. For an idea to be considered creative, it has to match a very complex... wait for it... pattern.
That pattern - what we call "creative" - has no strict definition. The idea has to be close enough to something we know, so we can frame it, yet different enough from it as to not be obvious, but still not too different, so we can still comprehend it. It has to make sense in relevant context - e.g. a creative mathematical proof has to still be correct (or a creative approach to proving a theorem has to plausibly look like it could possibly work); creative writing still has to be readable, etc.
The core of creativity is this unspecified pattern that things we consider "creative" match. And it so happens that things matching this pattern are a match for pattern "what makes sense for a human to read" in situations where a creative solution is called for. And the latter pattern - "response has to be sensible to a human" - is exactly what the LLM goal function is.
Thus follows that real creativity is part of what LLMs are being optimized for :).
> For an idea to be considered creative, it has to match a very complex... wait for it... pattern.
If we could predefine what would count as creativity as some specific pattern, then I'm not sure that would be what I would call creative, and certainly wouldn't be an all-inclusive definition of creativity. Nor is creativity merely creating something new by perturbing data randomly as you mentioned above.
While LLMs might be capable of some forms of creativity depending on how you define it, I think it remains to be seen how LLMs' current architecture could on its own accomplish the kinds of creativity implicit in scientific progress in the Kuhnian sense of a paradigm shift or in what some describe as a leap of artistic inspiration. Both of these examples highlight the degree to which creativity could be considered both progress in an objective sense but also be something that is not entirely foreshadowed by its precursors or patterns of existing data.
I think there are many senses in which LLMs are not demonstrating creativity in a way that humans can. I'm not sure how an LLM itself could create something new and valuable if it requires predefining an existing pattern which seems to presuppose that we already have the creation in a sense.
If we could predefine what would count as creativity as some specific pattern, then I'm not sure that would be what I would call creative, and certainly wouldn't be an all-inclusive definition of creativity. Nor is creativity merely creating something new by perturbing data randomly as you mentioned above.
While LLMs might be capable of some forms of creativity depending on how you define it, I think it remains to be seen how LLMs' current architecture could on its own accomplish the kinds of creativity implicit in scientific progress in the Kuhnian sense of a paradigm shift or in what some describe as a leap of artistic inspiration. Both of these examples highlight the degree to which creativity could be considered both progress in an objective sense but also be something that is not entirely foreshadowed by its precursors or patterns of existing data.
I think there are many senses in which LLMs are not demonstrating creativity in a way that humans can. I'm not sure how an LLM itself could create something new and valuable if it requires predefining an existing pattern which seems to presuppose that we already have the creation in a sense.
My take on Kuhn's paradigm shift is that it's still incremental progress, but the shift happens at a meta level. I.e., for the scientific example, you need some accumulated amount of observations and hypotheses before the paradigm shift can happen, and while the science "before" and "after" may look hugely different, it's still the case that the insight causing the shift is still incremental. In the periods before paradigm shifts, the science didn't stay still, waiting for a lone genius to make a big conceptual leap that randomly happened to hit paydirt -- if we could do such probability-defying miracles, we'd have special relativity figured out by Ancient Greeks. No, the science just kept accumulating observations and insights, narrowing down the search space until someone (usually several someones around the world, at the same time) was in the right place and context to see the next step and take it.
This kind of follows from the fact that, even if the paradigm-shifting insight was caused by some miracle feat of a unique superhuman genius, it still wouldn't shift anything until everyone else in the field was able to verify the genius was right, that they found the right answer, as oppose to a billion different possible wrong answers. To do that, the entire field had to have accumulated enough empirical evidence and theoretical understanding to already be within one or two "regular smart scholar" leaps from that insight.
With art, I have less experience, but my gut instinct tells me that even there, "artistic inspiration" can be too big a leap from what was before, as otherwise other people would not recognize or appreciate it. Also, unlike science, the definition of "art" is self-referential: art is what people recognize as art.
Still, I think you make a good point here, and convinced me that potential for creativity of LLMs, in their current architecture, is limited and below that of humans. You said:
> While LLMs might be capable of some forms of creativity depending on how you define it, I think it remains to be seen how LLMs' current architecture could on its own accomplish the kinds of creativity implicit in scientific progress in the Kuhnian sense of a paradigm shift or in what some describe as a leap of artistic inspiration.
I think the limit stems strictly from LLMs being trained off-line. I believe LLMs could go as far as making the paradigm-shifting "Kuhnian leap", but they wouldn't be able to increment on it further. Compared to humans, LLMs are all "system 1" and almost none "system 2" - they rely on "intuition"[0], which heavily biases them towards things they've learned before. In a wake of a paradigm shift, a human can make themselves gradually unlearn their own intuitions. LLM's can't, without being retrained. Because of that, the forms of creativity that involve making a paradigm-shifting leap and making a few steps forward from it are not within reach of any current model.
--
[0] - LLMs basically output things that seem most likely given what came before; I think this is the same phenomenon as when humans think and say what "feels like best" in context. However, we can pause and override this; LLMs can't, because they're just run in a forward pass - they neither have an internal loop, nor are they trained for the ability to control an external one.
This kind of follows from the fact that, even if the paradigm-shifting insight was caused by some miracle feat of a unique superhuman genius, it still wouldn't shift anything until everyone else in the field was able to verify the genius was right, that they found the right answer, as oppose to a billion different possible wrong answers. To do that, the entire field had to have accumulated enough empirical evidence and theoretical understanding to already be within one or two "regular smart scholar" leaps from that insight.
With art, I have less experience, but my gut instinct tells me that even there, "artistic inspiration" can be too big a leap from what was before, as otherwise other people would not recognize or appreciate it. Also, unlike science, the definition of "art" is self-referential: art is what people recognize as art.
Still, I think you make a good point here, and convinced me that potential for creativity of LLMs, in their current architecture, is limited and below that of humans. You said:
> While LLMs might be capable of some forms of creativity depending on how you define it, I think it remains to be seen how LLMs' current architecture could on its own accomplish the kinds of creativity implicit in scientific progress in the Kuhnian sense of a paradigm shift or in what some describe as a leap of artistic inspiration.
I think the limit stems strictly from LLMs being trained off-line. I believe LLMs could go as far as making the paradigm-shifting "Kuhnian leap", but they wouldn't be able to increment on it further. Compared to humans, LLMs are all "system 1" and almost none "system 2" - they rely on "intuition"[0], which heavily biases them towards things they've learned before. In a wake of a paradigm shift, a human can make themselves gradually unlearn their own intuitions. LLM's can't, without being retrained. Because of that, the forms of creativity that involve making a paradigm-shifting leap and making a few steps forward from it are not within reach of any current model.
--
[0] - LLMs basically output things that seem most likely given what came before; I think this is the same phenomenon as when humans think and say what "feels like best" in context. However, we can pause and override this; LLMs can't, because they're just run in a forward pass - they neither have an internal loop, nor are they trained for the ability to control an external one.
>Creativity" in the sense of coming up with something new is trivial to implement in computers, and has long been solved. Take some pattern - of words, of data, of thought. Perturb it randomly. Done. That's creativity.
Formal Proof Systems aren't even nearly close to completion, and for patterns we don't have a strong enough formal system to fully represent the problem space.
If we take the P=NP problem, that likely can be solved formally that a machine could do, but what is the "pattern" here that we are traversing here? There is a definitely a deeper superstructure behind these problems, but we can only glean the tips, and I don't think the LLMs with statistical techniques can glean further in either. Natural Language is not sufficient.
Formal Proof Systems aren't even nearly close to completion, and for patterns we don't have a strong enough formal system to fully represent the problem space.
If we take the P=NP problem, that likely can be solved formally that a machine could do, but what is the "pattern" here that we are traversing here? There is a definitely a deeper superstructure behind these problems, but we can only glean the tips, and I don't think the LLMs with statistical techniques can glean further in either. Natural Language is not sufficient.
Whatever the underlying "real" pattern is, doesn't really matter. We don't need to represent it. People learn to understand it implicitly, without ever seeing some formal definition spelled out - and learn it well enough that if you take M works to classify as "creative" or "not", then pick N people at random and ask each of them to classify each of the works, you can expect high degree of agreement.
LLMs aren't leaning what "creativity" is from first principles. They're learning it indirectly, by being trained to reply like a person would, literally, in the fully general meaning of that phrase. The better they get at that in general, the better they get at the (strict) subtask of "judging whether a work is creative the same way a human would" - and also "producing creative output like a human would".
Will that be enough to fully nail down what creativity is formally? Maybe, maybe not. On the one hand, LLMs don't "know" any more than we do, because whatever the pattern they learn, it's as implicit in their weights as it is for us. On the other hand, we can observe the models as they learn and infer, and poke at their weights, and do all kinds of other things that we can't do to ourselves, in order to find and understand how the "deeper superstructure behind these problems" gets translated into abstract structures within the model. This stands a chance to teach us a lot about both "these problems" and ourselves.
EDIT:
One could say there's no a priori reason why those ML models should have any structural similarity to how human brains work. But I'd say there is a reason - we're training them on inputs highly correlated with our own thoughts, and continuously optimizing them not just to mimic people, but to be bug for bug compatible with them. In the limit, the result of this pressure has to be equivalent to our own minds, even if not structurally equivalent. Of course the open question is, how far can we continue this process :).
LLMs aren't leaning what "creativity" is from first principles. They're learning it indirectly, by being trained to reply like a person would, literally, in the fully general meaning of that phrase. The better they get at that in general, the better they get at the (strict) subtask of "judging whether a work is creative the same way a human would" - and also "producing creative output like a human would".
Will that be enough to fully nail down what creativity is formally? Maybe, maybe not. On the one hand, LLMs don't "know" any more than we do, because whatever the pattern they learn, it's as implicit in their weights as it is for us. On the other hand, we can observe the models as they learn and infer, and poke at their weights, and do all kinds of other things that we can't do to ourselves, in order to find and understand how the "deeper superstructure behind these problems" gets translated into abstract structures within the model. This stands a chance to teach us a lot about both "these problems" and ourselves.
EDIT:
One could say there's no a priori reason why those ML models should have any structural similarity to how human brains work. But I'd say there is a reason - we're training them on inputs highly correlated with our own thoughts, and continuously optimizing them not just to mimic people, but to be bug for bug compatible with them. In the limit, the result of this pressure has to be equivalent to our own minds, even if not structurally equivalent. Of course the open question is, how far can we continue this process :).
As far as I can tell, I think you are interchanging the ability to recognize creativity with the ability to be creative. Humans seem to have the ability to make creative works or ideas that are not entirely derivative from a given data set or fit the criteria of some pre-existing pattern.
That is why I mentioned Kuhn and paradigm shifts. The architecture of LLMs do not seem capable of making lateral moves or sublations that are by definition not derivative or reducible to its prior circumstance, yet humans do, even though the exact way we do so is pretty mysterious and wrapped up in the difficulties in understanding consciousness.
To claim LLMs can or will equal human creativity seems to imply we can clearly define not only what creativity is, but also consciousness and also how to make a machine that can somehow do both. Humans can be creative prima facie, but to think we can also make a computer do the same thing probably means you have an inadequate definition of creativity.
That is why I mentioned Kuhn and paradigm shifts. The architecture of LLMs do not seem capable of making lateral moves or sublations that are by definition not derivative or reducible to its prior circumstance, yet humans do, even though the exact way we do so is pretty mysterious and wrapped up in the difficulties in understanding consciousness.
To claim LLMs can or will equal human creativity seems to imply we can clearly define not only what creativity is, but also consciousness and also how to make a machine that can somehow do both. Humans can be creative prima facie, but to think we can also make a computer do the same thing probably means you have an inadequate definition of creativity.
I wrote a long response wrt. Kuhn under your earlier comment, but to summarize it here: I believe LLMs can make lateral moves, but they will find it hard to increment on them. That is, they can make a paradigm-shifting creative leap itself, but they can't then unlearn the old paradigm on the spot - their fixed training is an attractor that'll keep pulling them back.
As for:
> As far as I can tell, I think you are interchanging the ability to recognize creativity with the ability to be creative.
I kind of am, because I believe that the two are intertwined. I.e. "creativity" isn't merely an ability to make large conceptual leaps, or "lateral moves" - it's the ability to make a subset of those moves that will be recognized by others as creative, as opposed to recognized as wrong, or recognized as insane, or recognized as incomprehensible.
This might apply more to art than science, since the former is a moving target - art is ultimately about matching subjective perceptions of people, where science is about matching objective reality. A "too creative" leap in science can still be recognized as "creative" later if it's actually correct. With art, whether "too creative" will be eventually accepted or forever considered absurd, is unpredictable. Which is to say, maybe we should not treat these two types of "creativity" as the same thing in the first place.
As for:
> As far as I can tell, I think you are interchanging the ability to recognize creativity with the ability to be creative.
I kind of am, because I believe that the two are intertwined. I.e. "creativity" isn't merely an ability to make large conceptual leaps, or "lateral moves" - it's the ability to make a subset of those moves that will be recognized by others as creative, as opposed to recognized as wrong, or recognized as insane, or recognized as incomprehensible.
This might apply more to art than science, since the former is a moving target - art is ultimately about matching subjective perceptions of people, where science is about matching objective reality. A "too creative" leap in science can still be recognized as "creative" later if it's actually correct. With art, whether "too creative" will be eventually accepted or forever considered absurd, is unpredictable. Which is to say, maybe we should not treat these two types of "creativity" as the same thing in the first place.
> Take some pattern - of words, of data, of thought. Perturb it randomly. Done. That's creativity.
This seems a miopic view of creativity. I think leaving out the pursuit of the implications of that perturbation is leaving out the majority of creativity. A random number generator is not creative without some way to explore the impact of the random number. This is something that LLM inference models just don't do. Feeding previous output into the context of a next "reasoning" step still depends on a static model at the core.
This seems a miopic view of creativity. I think leaving out the pursuit of the implications of that perturbation is leaving out the majority of creativity. A random number generator is not creative without some way to explore the impact of the random number. This is something that LLM inference models just don't do. Feeding previous output into the context of a next "reasoning" step still depends on a static model at the core.
>If I write a book that contains Einstein's theory of relativity by virtue of me copying it, did I create the theory? Did my copying of it indicate anything about my understanding of it? Would you be justified to think the next book I write would have anything of original value?
If you, after copying the book, could dynamically answer questions about the theory, it's implications, and answer variations of problems or theoretical challenges in ways that reflect mainstream knowledge, I think that absolutely would indicate understanding of it. I think you are basically making Searle's chinese room argument.
>But it is clear humans are capable of creativity and reasoning that are not reducible to mere pattern matching and this is the sense of reasoning that LLMs are not currently capable of.
Why is that clear? I think the reasoning for that would be tying it to a notion "the human experience", which I don't think is a necessary condition for intelligence. I think nothing about finding patterns is "mere" insofar as it relates to demonstration of intelligence.
If you, after copying the book, could dynamically answer questions about the theory, it's implications, and answer variations of problems or theoretical challenges in ways that reflect mainstream knowledge, I think that absolutely would indicate understanding of it. I think you are basically making Searle's chinese room argument.
>But it is clear humans are capable of creativity and reasoning that are not reducible to mere pattern matching and this is the sense of reasoning that LLMs are not currently capable of.
Why is that clear? I think the reasoning for that would be tying it to a notion "the human experience", which I don't think is a necessary condition for intelligence. I think nothing about finding patterns is "mere" insofar as it relates to demonstration of intelligence.
> But it is clear humans are capable of ...
Its not though, nobody really knows what most of the words in that sentence mean in the technical or algorithmical sense, and hence you can't really say whether llms do or don't possess these skills.
Its not though, nobody really knows what most of the words in that sentence mean in the technical or algorithmical sense, and hence you can't really say whether llms do or don't possess these skills.
>nobody really knows what most of the words in that sentence mean in the technical or algorithmical sense
And nobody really knows what consciousness is, but we all experience it in a distinct, internal way that lets us navigate the world and express ourselves to others, yet apparently some comments seem to dismiss this elephant of sensation in the room by pretending it's no different than some cut and dried computational system that's programmed to answer certain things in certain ways and thus "is probably no different from a person trained to speak". We're obviously, evidentially more than that.
And nobody really knows what consciousness is, but we all experience it in a distinct, internal way that lets us navigate the world and express ourselves to others, yet apparently some comments seem to dismiss this elephant of sensation in the room by pretending it's no different than some cut and dried computational system that's programmed to answer certain things in certain ways and thus "is probably no different from a person trained to speak". We're obviously, evidentially more than that.
> by pretending it's no different than some cut and dried computational system
This is not really what is going on, what is going on is a mix-up in interpreting the meaning of words, because the meaning of words is not transitive between subject matter unless we arrive at a scientific definition which is leading, and we have not (yet).
When approaching the word consciousness from a spiritual POV, it is clear that LLMs may not possess it. When approaching consciousness from a technical point of view, it is clear that LLMs may possess it in the future. This is because the spiritual POV is anthropologically reductive (consciousness is human), and the technical POV is technically reductive (consciousness is when we can't tell it apart).
Neither statements help us clarify opposing positions because neither definitions are falsifiable, and so not scientific.
This is not really what is going on, what is going on is a mix-up in interpreting the meaning of words, because the meaning of words is not transitive between subject matter unless we arrive at a scientific definition which is leading, and we have not (yet).
When approaching the word consciousness from a spiritual POV, it is clear that LLMs may not possess it. When approaching consciousness from a technical point of view, it is clear that LLMs may possess it in the future. This is because the spiritual POV is anthropologically reductive (consciousness is human), and the technical POV is technically reductive (consciousness is when we can't tell it apart).
Neither statements help us clarify opposing positions because neither definitions are falsifiable, and so not scientific.
I disagree with that characterization. I don’t experience consciousness as an “internal way that lets us navigate the world and express ourselves to others”. To me it is a purely perceptional experience, as I concluded after much introspection. Sure it feeds back into one’s behavior, mostly because we prefer certain experiences over others, but I can’t identify anything in my inner experience that is qualitatively different in nature from a pure mechanism. I do agree that LLMs severely lack awareness (not just self-awareness) and thus also consciousness. But that’s not about being a “mere” computational system.
Words are not reducible to technical statements or algorithms. But, even if they were, then by your suggestion there's not much point in talking about anything at all.
They absolutely are in the context of a technical, scientific or mathematical subject.
Like in the subject of LLMs everyone knows what a "token" or "context" means, even if they might mean different things in a different subject. Yet, nobody knows what "consciousness" means in almost any context, so it is impossible to make falsifiable statements about consciousness and LLMs.
Making falsifiable statements is the only way to have an argument, otherwise its just feelings and hunches with window dressing.
Like in the subject of LLMs everyone knows what a "token" or "context" means, even if they might mean different things in a different subject. Yet, nobody knows what "consciousness" means in almost any context, so it is impossible to make falsifiable statements about consciousness and LLMs.
Making falsifiable statements is the only way to have an argument, otherwise its just feelings and hunches with window dressing.
> LLMs current architecture seems to mainly work by understanding patterns in the existing body of knowledge ...
>But it is clear humans are capable of creativity and reasoning that are not reducible to mere pattern matching and this is the sense of reasoning that LLMs are not currently capable of
This is not clear at all. As it seems to me, it's impossible to imagine or think of things that are not in someway tied to something you've already come to sense or know. And if you think I am wrong, I implore you to provide a notion that doesn’t agree. I can only imagine something utterly unintelligible, and in order to make it intelligible, would require "pattern matching" (ie tying) it to something that is already intelligible. I mean how else do we come to understand a newly-found dead/unknown language, or teach our children? What human thought operates completely outside existing knowledge, if not given empirically?
>But it is clear humans are capable of creativity and reasoning that are not reducible to mere pattern matching and this is the sense of reasoning that LLMs are not currently capable of
This is not clear at all. As it seems to me, it's impossible to imagine or think of things that are not in someway tied to something you've already come to sense or know. And if you think I am wrong, I implore you to provide a notion that doesn’t agree. I can only imagine something utterly unintelligible, and in order to make it intelligible, would require "pattern matching" (ie tying) it to something that is already intelligible. I mean how else do we come to understand a newly-found dead/unknown language, or teach our children? What human thought operates completely outside existing knowledge, if not given empirically?
Why can’t creativity be taking the works, a bunch of works, finding a pattern then randomly perturbing a data point/concept to see if there are new patterns.
Then cross referencing that new random point/idea to see if it remains internally consistent with the known true patterns in your dataset.
This is how humans create new ideas often?
Then cross referencing that new random point/idea to see if it remains internally consistent with the known true patterns in your dataset.
This is how humans create new ideas often?
I see absolutely zero wrong with that statement. What he said is indeed much more reasoned and intelligent than the average foolish AI hype i've often found here, written by people who try to absurdly redefine the obvious, complex mystery that is consciousness into some reductionist notion of it being anything that presents the appearance of reasoning through technical tricks.
Chiang has it exactly right with his doubts, and the notion that pattern recognition is little different from the deeply complex navigation of reality we living things do is the badly misguided notion.
Chiang has it exactly right with his doubts, and the notion that pattern recognition is little different from the deeply complex navigation of reality we living things do is the badly misguided notion.
> Chiang has it exactly right with his doubts, and the notion that pattern recognition is little different from the deeply complex navigation of reality we living things do is the badly misguided notion.
How do you know this?
How do you know this?
The same way that we know interpolation of a linear regression is not the same as the deeply complex navigation of reality we do as living things.
I notice that often in these debates someone will make the comparison between a low level mechanism driving LLMs, and a high level emergent behavior of the human mind. I don't think it's deliberate - we don't fully understand how the brain works so we only have emergent behaviors - but how can you be so certain that deeply complex navigation of reality can't emerge from interpolation of a linear regression?
That's a good question. With sufficient dimensionality, interaction terms, and enough linear regressions, I suppose it's possible. But dynamic and reactive coordination of many multiple linear regressions wouldn't be just a linear regression. The output of a linear regression is simplistic just like LLM token prediction is simplistic. Saying something might be a component of eventual intelligence is far from it being intelligence. LLMs are episodic responses to a fixed context by a fixed model that is programmed to predict tokens. Even the CoT models, while more complex, still use a static model with a recursive feed of model outputs back to the model. I think Dr. Chollet does an excellent job of identifying the fundamental difference between a potential AGI and static models in his ARC-AGI papers and presentations.
> but how can you be so certain that deeply complex navigation of reality can't emerge from interpolation of a linear regression?
That was pretty much my question. Why are people so certain on the topic.
That was pretty much my question. Why are people so certain on the topic.
I wasn't trying to be flippant but challenge the excessive confidence people have on this topic.
Even most intelligent people can hallucinate, we still haven't fixed this problem. There's a lot of training material and bias which leads many to repeat those things "LLM's are just a stochastic parrot, glorified auto complete/google search, Markov chains, just statistics", etc. The thing is, these sentences sound really good and so it's easy to repeat them when you have made up your mind. It's a shortcut.
I feel like at this point we have to separate LLMs and reasoning models too.
I can see the argument against chatGPT4 reasoning.
The reasoning models though I think get into some confusing language but I don't know what else you would call it.
If you say a car is not "running" the way a human runs, you are not incorrect even though a car can "outrun" any human obviously in terms of moving speed on the ground.
To say since a car can't run , it can't move though is obviously completely absurd.
I can see the argument against chatGPT4 reasoning.
The reasoning models though I think get into some confusing language but I don't know what else you would call it.
If you say a car is not "running" the way a human runs, you are not incorrect even though a car can "outrun" any human obviously in terms of moving speed on the ground.
To say since a car can't run , it can't move though is obviously completely absurd.
This was precisely what motivated Turing to come up with the test named after him, to avoid such semantic debates. Yet here we are still in the same loop.
"The terminator isn't really hunting you down, it's just imitating doing so..."
"The terminator isn't really hunting you down, it's just imitating doing so..."
LLMs don’t go into a different mode when they are hallucinating. That’s just how they work.
Using the word “hallucinate” is extremely misleading because it’s nothing like what people do when they hallucinate (thinking there are sensory inputs when there aren’t).
It’s much closer to confabulation, which is extremely rare and is usually a result of brain damage.
This is why a big chunk of people (including myself) think the current LLMs are fundamentally flawed. Something with a massive database to statistically confabulate correct stuff 95% of the time and not have a clue when it’s completely made up is not anything like intelligence.
Compressing all of the content of the internet into an LLM is useful and impressive. But these things aren’t going to start doing any meaningful science or even engineering on their own.
Using the word “hallucinate” is extremely misleading because it’s nothing like what people do when they hallucinate (thinking there are sensory inputs when there aren’t).
It’s much closer to confabulation, which is extremely rare and is usually a result of brain damage.
This is why a big chunk of people (including myself) think the current LLMs are fundamentally flawed. Something with a massive database to statistically confabulate correct stuff 95% of the time and not have a clue when it’s completely made up is not anything like intelligence.
Compressing all of the content of the internet into an LLM is useful and impressive. But these things aren’t going to start doing any meaningful science or even engineering on their own.
Intelligent people do not "hallucinate" in the same sense that an LLM does. Counterarguments you don't like aren't "shortcuts". There are certainly obnoxious anti-LLM people, but you can't use them to dismiss everyone else.
An LLM does nothing more than predict the next token in a sequence. It is functionally auto-complete. It hallucinates because it has no concept of a fact. It has no "concept", period, it cannot reason. It is a statistical model. The "reasoning" you observe in models like o1 is a neat prompting trick that allows it to generate more context for itself.
I use LLMs on a daily basis. I use them at work and at home, and I feel that they have greatly enhanced my life. At the end of the day they are just another tool. The term "AI" is entirely marketing preying on those who can't be bothered to learn how the technology works.
An LLM does nothing more than predict the next token in a sequence. It is functionally auto-complete. It hallucinates because it has no concept of a fact. It has no "concept", period, it cannot reason. It is a statistical model. The "reasoning" you observe in models like o1 is a neat prompting trick that allows it to generate more context for itself.
I use LLMs on a daily basis. I use them at work and at home, and I feel that they have greatly enhanced my life. At the end of the day they are just another tool. The term "AI" is entirely marketing preying on those who can't be bothered to learn how the technology works.
They’re right until they’re wrong.
AI is (was?) a stochastic parrot. At some point AI will likely be more than that. The tipping point may not be obvious.
AI is (was?) a stochastic parrot. At some point AI will likely be more than that. The tipping point may not be obvious.
> Even most intelligent people can hallucinate, we still haven't fixed this problem.
No we have not, neurodiverse people like me need accommodations not fixing.
No we have not, neurodiverse people like me need accommodations not fixing.
It is not hallucination. When people do what we call halucination in chatGPT, it is called "bullshiting", "lying" or "being incompetent".
> and wonder how an intelligent person can still think this, can be so absolute about it. What is "actual" reasoning here?
Large language models excel at processing and generating text, but they fundamentally operate on existing knowledge. Their creativity appears limited to recombining known information in novel ways, rather than generating truly original insights.
True reasoning capability would involve the ability to analyze complex situations and generate entirely new solutions, independent of existing patterns or combinations. This kind of deep reasoning ability seems to be beyond the scope of current language models, as it would require a fundamentally different approach—what we might call a reasoning model. Currently, it's unclear to me whether such models exist or if they could be effectively integrated with large language models.
Large language models excel at processing and generating text, but they fundamentally operate on existing knowledge. Their creativity appears limited to recombining known information in novel ways, rather than generating truly original insights.
True reasoning capability would involve the ability to analyze complex situations and generate entirely new solutions, independent of existing patterns or combinations. This kind of deep reasoning ability seems to be beyond the scope of current language models, as it would require a fundamentally different approach—what we might call a reasoning model. Currently, it's unclear to me whether such models exist or if they could be effectively integrated with large language models.
> True reasoning capability would involve the ability to analyze complex situations and generate entirely new solutions, independent of existing patterns or combinations.
You mean like alphago did in its 36th move?
You mean like alphago did in its 36th move?
Isn't that a non-generic 'reasoning-model' instead of something that is reminiscent of the large language model based AIs we use today?
The question is, is it possible to make reasoning models generic and can they be combined with large language models effectively.
The question is, is it possible to make reasoning models generic and can they be combined with large language models effectively.
Move 37.
"Their creativity appears limited to recombining known information"
There are some theories that this is true for humans also.
There are no human created images that weren't observed first in nature in some way.
For example, Devils/Demons/Angels were described in terms of human body parts, or 'goats' with horns. Once we got microscopes and started drawing insects then art got a lot weirder, but not before images were observed from reality. Then humans could re-combine them.
There are some theories that this is true for humans also.
There are no human created images that weren't observed first in nature in some way.
For example, Devils/Demons/Angels were described in terms of human body parts, or 'goats' with horns. Once we got microscopes and started drawing insects then art got a lot weirder, but not before images were observed from reality. Then humans could re-combine them.
I understand your point, but it's not comparable:
Humans can suddenly "jump" cognitive levels to see higher-order patterns. Gödel seeing that mathematics could describe mathematics itself. This isn't combining existing patterns, but seeing entirely new levels of abstraction.
The human brain excels at taking complex systems and creating simpler mental models. Newton seeing planetary motion and falling apples as the same phenomenon. This compression isn't recombination - it's finding the hidden simplicity.
Recombination adds elements together. Insight often removes elements to reveal core principles. This requires understanding and reasoning.
Humans can suddenly "jump" cognitive levels to see higher-order patterns. Gödel seeing that mathematics could describe mathematics itself. This isn't combining existing patterns, but seeing entirely new levels of abstraction.
The human brain excels at taking complex systems and creating simpler mental models. Newton seeing planetary motion and falling apples as the same phenomenon. This compression isn't recombination - it's finding the hidden simplicity.
Recombination adds elements together. Insight often removes elements to reveal core principles. This requires understanding and reasoning.
>and wonder how an intelligent person can still think this, can be so absolute about it.
I wonder how people write things like this and don't realize they sound as sanctimonious as exactly whatever they are criticizing. Or, if I was to put it in your words: "how could someone intelligent post like this?"
I wonder how people write things like this and don't realize they sound as sanctimonious as exactly whatever they are criticizing. Or, if I was to put it in your words: "how could someone intelligent post like this?"
You're right, it was kind of rude. Apologies. I really would rather be wrong, for a reason I gave in another comment.
The thing is, you can interact with this new kind of actor as much as you need to to judge this -- make up new problems, ask your own questions. "LLMs can't think" has needed ever-escalating standards for "real" thinking over the last few years.
Gary Marcus made a real-money bet about this.
The thing is, you can interact with this new kind of actor as much as you need to to judge this -- make up new problems, ask your own questions. "LLMs can't think" has needed ever-escalating standards for "real" thinking over the last few years.
Gary Marcus made a real-money bet about this.
I think a better question is "what is the value of thought?" when it came to conclusions such as yours: "I should be rude to this poster because they disagree with me"
To me this also feels like a statement that would obviously need strong justification. For if animals are capable of reasoning, probably through being trained on many examples of the laws of nature doing their thing, then why couldn't a statistical model be?
> For if animals are capable of reasoning
Are they? Which animals? Some seem smart and maybe do it. Needs strong justification.
> probably through being trained on many examples of the laws of nature doing their thing
Is that how they can reason? Why do you think so? Sounds like something that needs strong justification.
> then why couldn't a statistical model be?
Maybe because that is not how anything in the world attained the ability to reason.
A lot of animals can see. They did not have to train for this. They are born with eyes and a brain.
Humans are born with the ability to recognize pattern in what we see. We can tell objects apart without training.
Are they? Which animals? Some seem smart and maybe do it. Needs strong justification.
> probably through being trained on many examples of the laws of nature doing their thing
Is that how they can reason? Why do you think so? Sounds like something that needs strong justification.
> then why couldn't a statistical model be?
Maybe because that is not how anything in the world attained the ability to reason.
A lot of animals can see. They did not have to train for this. They are born with eyes and a brain.
Humans are born with the ability to recognize pattern in what we see. We can tell objects apart without training.
> Needs strong justification.
if animals didn't show problem-solving skills, and thus reasoning, complex ones wouldn't exist anymore by now. Planning is a fundamental skill for survival in a resource-constrained environment and that's how intelligence evolved to begin with.
Assuming that intelligence and by extension reasoning are discrete steps is so backwards to me. They are quite obviously continuously connected all the way back to the first nervous systems.
if animals didn't show problem-solving skills, and thus reasoning, complex ones wouldn't exist anymore by now. Planning is a fundamental skill for survival in a resource-constrained environment and that's how intelligence evolved to begin with.
Assuming that intelligence and by extension reasoning are discrete steps is so backwards to me. They are quite obviously continuously connected all the way back to the first nervous systems.
Are human beings not animals? If animals can't reason, then neither can we.
I agree with Chiang. Reminds me of Searle and The Chinese Room (I agree with Searle too).
I do think that at some point everyone is just arguing semantics. Chiang is arguing that "actual reasoning" is, by definition, not something that an LLM can do. And I do think he's right. But the real story is not "LLMs can't do X special thing that only biological life can do," the real story is "X special thing that only biological life can do isn't necessary to build incredibe AI that in many ways surpasses biological life".
I do think that at some point everyone is just arguing semantics. Chiang is arguing that "actual reasoning" is, by definition, not something that an LLM can do. And I do think he's right. But the real story is not "LLMs can't do X special thing that only biological life can do," the real story is "X special thing that only biological life can do isn't necessary to build incredibe AI that in many ways surpasses biological life".
>and wonder how an intelligent person can still think this,
Read up on the ELIZA effect
Read up on the ELIZA effect
[deleted]
In theory you can prove a theorem just by enumerating all the possible proofs until you find the one for the theorem you want. This is extremely slow, but do you think there's any reasoning in doing this?
Of course we don't know whether an LLM is doing something like this or actually reasoning. But this is also the point, we don't know.
If you ask a question to a person you can be confident to some degree that they didn't memorize the answer beforehand, so you can evaluate their ability to "reason" and come up with an answer for it. With an LLM however this is increadibly hard to do, because they could have memorized it.
Of course we don't know whether an LLM is doing something like this or actually reasoning. But this is also the point, we don't know.
If you ask a question to a person you can be confident to some degree that they didn't memorize the answer beforehand, so you can evaluate their ability to "reason" and come up with an answer for it. With an LLM however this is increadibly hard to do, because they could have memorized it.
> In theory you can prove a theorem just by enumerating all the possible proofs until ...
An interesting hypothesis! I'm neither a mathematical logician, nor decently up to date in that field - is the possibility of this, at least in the abstract, currently accepted as fact?
(Yes, there's the perhaps-separate issue of only enumerating correct proofs.)
An interesting hypothesis! I'm neither a mathematical logician, nor decently up to date in that field - is the possibility of this, at least in the abstract, currently accepted as fact?
(Yes, there's the perhaps-separate issue of only enumerating correct proofs.)
It depends on what theory you're working in (at which point deciding whether to use one theory or another becomes more like a phisolophical question).
I'm mostly familiar with type theory, of which there are many variants, but the most common ones all share the most important characteristics. In particular they identify theorems with types, and proofs with terms, where correct proofs are well-typed terms. The nice thing is that terms are recursively enumerable, so you can list all proofs. Moreover most type theories have decidable type checking, so you can automatically check whether a terms if well-typed (and hence the corresponding proof is correct).
This is not just theory, there exist already a bunch of tools that are being used in practice for mechanically checking mathematical proofs, like Coq, Lean, Agda and more.
When I said "in theory" however it's because in practice enumerating all proof terms will be very very slow and will take forever to reach proofs for theorems that we might find interesting.
Since we're in the LLM topic, there are efforts to use LLMs to speed up this search, though this is more similar to using them as search heuristics though. It does help though that you can have automatic feedback thanks to the aforementioned proof checking tools, meaning you don't need costly human supervision to train them. The hope would be getting something like what Stockfish/Alphazero is for chess.
I'm mostly familiar with type theory, of which there are many variants, but the most common ones all share the most important characteristics. In particular they identify theorems with types, and proofs with terms, where correct proofs are well-typed terms. The nice thing is that terms are recursively enumerable, so you can list all proofs. Moreover most type theories have decidable type checking, so you can automatically check whether a terms if well-typed (and hence the corresponding proof is correct).
This is not just theory, there exist already a bunch of tools that are being used in practice for mechanically checking mathematical proofs, like Coq, Lean, Agda and more.
When I said "in theory" however it's because in practice enumerating all proof terms will be very very slow and will take forever to reach proofs for theorems that we might find interesting.
Since we're in the LLM topic, there are efforts to use LLMs to speed up this search, though this is more similar to using them as search heuristics though. It does help though that you can have automatic feedback thanks to the aforementioned proof checking tools, meaning you don't need costly human supervision to train them. The hope would be getting something like what Stockfish/Alphazero is for chess.
Has an AI “proven” a theorem?
Leading mathematician Terence Tao says yes, with a lot of guidance.
https://mathstodon.xyz/@tao/113132503432772494
https://mathstodon.xyz/@tao/113132503432772494
I have a similar opinion of Claude Sonnet. Superhuman knowledge; ability to apply it to solve new math/coding problems at roughly the level of a motivated high-schooler (but not corresponding exactly in profile to anything human); less ability to stay on track the longer the effort takes.
But ChatGPT a couple years ago was at more like grade-school level at problem-solving. What should I call this thing that the best LLMs can do better than the older ones, if it's not actual reasoning? Sparkling syllogistics?
Sorry, that's sarcastic, but... it's from a real exasperation at what seems like a rearguard fight against an inconvenient conclusion. I don't like it either! I think the rate of progress at building machines we don't understand is dangerous. (Understanding the training is not understanding the machinery that comes out.)
Compare the first previews of Copilot with current frontier "reasoning" models, and ask how this will develop in the next five years. Maybe it'll fizzle. If you're very confident it will: I'd like to be convinced too.
But ChatGPT a couple years ago was at more like grade-school level at problem-solving. What should I call this thing that the best LLMs can do better than the older ones, if it's not actual reasoning? Sparkling syllogistics?
Sorry, that's sarcastic, but... it's from a real exasperation at what seems like a rearguard fight against an inconvenient conclusion. I don't like it either! I think the rate of progress at building machines we don't understand is dangerous. (Understanding the training is not understanding the machinery that comes out.)
Compare the first previews of Copilot with current frontier "reasoning" models, and ask how this will develop in the next five years. Maybe it'll fizzle. If you're very confident it will: I'd like to be convinced too.
you said you said it sarcastically but I like "syllogistic" a lot. We need more volcabulary to describe what LLMs do, and if I tell ChatGPT A implies B implies C, and I tell it A is true, and I can describe that as the LLM syllogisting and not use the words "reasoning" or "thinking", that works for me.
As far as if it will fizzle, even if it does, what we have currently is already useful. Society will take time to adjust to ChatGPT-4's level of capabilities, nevermind whatever OpenAI et al releases next. It can't yet replace a software engineer, but it makes projects possible they previously weren't attempted because they required too much investment previously. So unless you're financially exposed to AI directly (which you might be, many people are!), the question of if it's going to fizzle is more academic than something that demands a rigorous answer. Proofs of a negative are really hard. Reusable rockets were "proven" to be impossible right up until it was empirically proven possible.
As far as if it will fizzle, even if it does, what we have currently is already useful. Society will take time to adjust to ChatGPT-4's level of capabilities, nevermind whatever OpenAI et al releases next. It can't yet replace a software engineer, but it makes projects possible they previously weren't attempted because they required too much investment previously. So unless you're financially exposed to AI directly (which you might be, many people are!), the question of if it's going to fizzle is more academic than something that demands a rigorous answer. Proofs of a negative are really hard. Reusable rockets were "proven" to be impossible right up until it was empirically proven possible.
I don't see Tao claiming ChatGPT proved a theorem. Moreover most questions seemed to be about something already talked about online, so it seems plausible that it was included in the training data. This is IMO a big issue with evaluating LLMs, you can't keep asking the same questions because you can't be sure they will eventually answer by memory or actually reason.
I think the "LLM is intelligence" crowd has a very simplistic view of people. If you feel that natural language and the systems responsible it are pretty much the only things that human intelligence produces, then I can see the argument.
But I don't believe that. That a machine that can produce convincing human-language chains of thought says nothing about its "intelligence". Back when basic RNNs/LSTMs were at the forefront of ML research, no one had any delusions about this fact. And just because you can train a token prediction model on all of human knowledge (which the internet is not) doesn't mean the model understands anything.
It's surprising to me that the people most knowledgeable about the models often appear to be the biggest believers - perhaps they're self-interestedly pumping a valuation or are simply obsessed with the idea of building something straight from the science fiction stories they grew up with.
In the end though, the burden of proof is on the believers, not the deniers.
But I don't believe that. That a machine that can produce convincing human-language chains of thought says nothing about its "intelligence". Back when basic RNNs/LSTMs were at the forefront of ML research, no one had any delusions about this fact. And just because you can train a token prediction model on all of human knowledge (which the internet is not) doesn't mean the model understands anything.
It's surprising to me that the people most knowledgeable about the models often appear to be the biggest believers - perhaps they're self-interestedly pumping a valuation or are simply obsessed with the idea of building something straight from the science fiction stories they grew up with.
In the end though, the burden of proof is on the believers, not the deniers.
> It's surprising to me that the people most knowledgeable about the models often appear to be the biggest believers - perhaps they're self-interestedly pumping a valuation or are simply obsessed with the idea of building something straight from the science fiction stories they grew up with.
"Believer" really is the most appropriate label here. Altman or Musk lying and pretending they "AGI" right around the corner to pump their stocks is to be expected. The actual knowledgeable making completely irrational claims is simply incomprehensible beyond narcissism and obscurantism.
Interestingly, those who argue against the fiction that current models are reasoning, are using reason to make their points. A non-reasoning system generating plausible text is not at all a mystery can be explained, therefore, it's not sufficient for a system to generate plausible text to qualify as reasoning.
Those who are hyping the emergence of intelligence out of statistical models of written language on the other hand rely strictly on the basest empiricism, e.g. "I have an interaction with ChatGPT that proves it's intelligent" or "I put your argument into ChatGPT and here's what it said, isn't that interestingly insightful". But I don't see anyone coming out with any reasoning on how ability to reason could emerge out of a system predicting text.
There's also a tacit connection made between those language models being large and complex and their supposed intelligence. The human brain is large and complex, and it's the material basis of human intelligence, "therefore expensive large language models with internal behavior completely unexplainable to us, must be intelligent".
I don't think it will, but if the release of the deepseek models effectively shifts the main focus towards efficiency as opposed to "throwing more GPUs at it", that will also force the field to produce models with the current behavior using only the bare minimum, both in terms of architecture and resources. That would help against some aspects of the mysticism.
The biggest believers are not the best placed to drive the research forward. They are not looking at it critically and trying to understand it. They are using every generated sentence as a confirmation of their preconceptions. If the most knowledgeable are indeed the biggest believers, we are in for a long dark (mystic) AI winter.
"Believer" really is the most appropriate label here. Altman or Musk lying and pretending they "AGI" right around the corner to pump their stocks is to be expected. The actual knowledgeable making completely irrational claims is simply incomprehensible beyond narcissism and obscurantism.
Interestingly, those who argue against the fiction that current models are reasoning, are using reason to make their points. A non-reasoning system generating plausible text is not at all a mystery can be explained, therefore, it's not sufficient for a system to generate plausible text to qualify as reasoning.
Those who are hyping the emergence of intelligence out of statistical models of written language on the other hand rely strictly on the basest empiricism, e.g. "I have an interaction with ChatGPT that proves it's intelligent" or "I put your argument into ChatGPT and here's what it said, isn't that interestingly insightful". But I don't see anyone coming out with any reasoning on how ability to reason could emerge out of a system predicting text.
There's also a tacit connection made between those language models being large and complex and their supposed intelligence. The human brain is large and complex, and it's the material basis of human intelligence, "therefore expensive large language models with internal behavior completely unexplainable to us, must be intelligent".
I don't think it will, but if the release of the deepseek models effectively shifts the main focus towards efficiency as opposed to "throwing more GPUs at it", that will also force the field to produce models with the current behavior using only the bare minimum, both in terms of architecture and resources. That would help against some aspects of the mysticism.
The biggest believers are not the best placed to drive the research forward. They are not looking at it critically and trying to understand it. They are using every generated sentence as a confirmation of their preconceptions. If the most knowledgeable are indeed the biggest believers, we are in for a long dark (mystic) AI winter.
Can an LLM discover a new theory of natural law, such as prove or disprove string theory? This is what I ponder. The creation or discovery of something new that it can’t just copy, that would have to be discovered via human thought otherwise. Something provably true. The equivalent of discovering general relativity before humans had.
I feel like there is likely far more useful answers in overlooked science then there is in "new" science.
Sounds to me like a sci-fi author exploring his thoughts. Perhaps a full treatment of the subject wasn’t on his mind that day. I also don’t expect the article’s author to include anything they feel is mundane to themselves. Or to only include what they personally found interesting.
> how an intelligent person can still think this
Cognitive neuroscience
“qualia”
Ray Kurzweil
I’ll take “things OP doesn’t know about that an intelligent person does” for 800 Alex.
If you’re enamored with LLMs and can’t see the inherent problems, you don’t actually know about AI and machine learning.
Cognitive neuroscience
“qualia”
Ray Kurzweil
I’ll take “things OP doesn’t know about that an intelligent person does” for 800 Alex.
If you’re enamored with LLMs and can’t see the inherent problems, you don’t actually know about AI and machine learning.
Chiang is a silly blob of meat, of course he's not capable of actual reasoning, much less "intelligence".
We grant him personhood, but personhood, like the LA Review of Books, is just a social construct.
We grant him personhood, but personhood, like the LA Review of Books, is just a social construct.
This is a good point.
If you prick an LLM does it not bleed? If you tickle it does it not laugh? If you poison one does it not die? If you wrong an LLM shall it not revenge?
If you prick an LLM does it not bleed? If you tickle it does it not laugh? If you poison one does it not die? If you wrong an LLM shall it not revenge?
What is intelligent person? You seem to have approached the article with an existing reverence.
Rather 1984 to look at the contribution of an academic and an iron welder and see authority in someone who memorized the book, but not how to keep themselves alive. Chiang and the like are nihilists, indifferent if they die cause it all just goes dark to them. Indifferent to the toll they extract from labor to fly their ass around speaking about glyphs in a textbook. Academics detached from the real work people need are just as draining on society and infuriating as a billionaire CEO and tribal shaman. Especially these days when they derive some small normalization from 100s of years of cataloged work and proclaim their bit of syntactic art is all they should need to spend the rest of their life being celebrated like they’re turning 8 all over again.
Grigori Perelman is the only intelligent person out there I respect. Copy-paste college grads all over the US recite the textbook and act like it’s a magical incantation that bends the will of others. Cult of social incompetence in the US.
Rather 1984 to look at the contribution of an academic and an iron welder and see authority in someone who memorized the book, but not how to keep themselves alive. Chiang and the like are nihilists, indifferent if they die cause it all just goes dark to them. Indifferent to the toll they extract from labor to fly their ass around speaking about glyphs in a textbook. Academics detached from the real work people need are just as draining on society and infuriating as a billionaire CEO and tribal shaman. Especially these days when they derive some small normalization from 100s of years of cataloged work and proclaim their bit of syntactic art is all they should need to spend the rest of their life being celebrated like they’re turning 8 all over again.
Grigori Perelman is the only intelligent person out there I respect. Copy-paste college grads all over the US recite the textbook and act like it’s a magical incantation that bends the will of others. Cult of social incompetence in the US.
Please, wake me up when artificial so-called intelligence will have proved a new theorem.
Ted Chiang revealed himself at his NeurIPS "Pluralism and Creativity" workshop to be... a great book author and not much else. His statements during his panels with the other AI researchers proved that he was not up to date on modern AI research.
He's overly sentimental, and so are his books. I wish there were other sci-fi authors that the AI community wanted to contact but after "Arrival" I get it since "Arrival" is the literal wet-dream of many NLP/AI researchers.
He's overly sentimental, and so are his books. I wish there were other sci-fi authors that the AI community wanted to contact but after "Arrival" I get it since "Arrival" is the literal wet-dream of many NLP/AI researchers.
> "It’s like imagining that a printer could actually feel pain because it can print bumper stickers with the words ‘Baby don’t hurt me’ on them. It doesn’t matter if the next version of the printer can print out those stickers faster, or if it can format the text in bold red capital letters instead of small black ones. Those are indicators that you have a more capable printer but not indicators that it is any closer to actually feeling anything"
Love TC but I don't think this argument holds water. You need to really get into the weeds of what "actually feeling" means.
To use a TC-style example... suppose it's a major political issue in the future about AI-rights and whether AIs "really" think and "really" feel the things they claim. Eventually we invent an fMRI machine and model of the brain that can conclusively explain the difference between what "really" feeling is, and only pretending. We actually know exactly which gene sequence is responsible for real intelligence. Here's the twist... it turns out 20% of humans don't have it. The fake intelligences have lived among us for millennia...!
Love TC but I don't think this argument holds water. You need to really get into the weeds of what "actually feeling" means.
To use a TC-style example... suppose it's a major political issue in the future about AI-rights and whether AIs "really" think and "really" feel the things they claim. Eventually we invent an fMRI machine and model of the brain that can conclusively explain the difference between what "really" feeling is, and only pretending. We actually know exactly which gene sequence is responsible for real intelligence. Here's the twist... it turns out 20% of humans don't have it. The fake intelligences have lived among us for millennia...!
I disagree. The reason humans anthropomorphize "AI" is because we apply our own meta-models of intelligence to llms, etc., where they simply don't apply. The model can spit out something that seems extremely intelligent and well thought out that would truly be shocking if a monkey said it for example due to our meta-model of intelligence, and that may be valid in that case if we determined it wasn't simply memorized. His argument can certainly be more fleshed out, but the point he's making is correct, which is that we can't treat the output of a machine designed to replicate human input as though it contains the requisite intelligence/"feeling"/etc to produce that output on it's own.
I agree that with current LLMs the error goes the other way; they appear more conscious than they are, compared to, say, crows or octopuses which appear less conscious than they actually are.
My point is that "appears conscious" is really the only test there is. In what way is a human that says "that hurts" really feeling pain? What about Stephen Hawking "saying it", what about if he could only communicate through printed paper etc etc. You can always play this dial-down-the-consciousness game.
People used to say fish don't feel pain, they are "merely responding to stimulus".
My point is that "appears conscious" is really the only test there is. In what way is a human that says "that hurts" really feeling pain? What about Stephen Hawking "saying it", what about if he could only communicate through printed paper etc etc. You can always play this dial-down-the-consciousness game.
People used to say fish don't feel pain, they are "merely responding to stimulus".
The only actual difference in my view is that somehow we feel that we are so uber special. Besides that, it seems there's no reason to believe that we are anything more than chemical signals. But the fact that we have this strong "feeling" that we are special refuses us to admit that. I feel like I'm special, I feel like I exist. That's the only argument for being more than something else.
> I feel like I exist.
This is pretty much the only thing in the world that you can definitely prove to yourself. It’s not a feeling
This is pretty much the only thing in the world that you can definitely prove to yourself. It’s not a feeling
Interestingly the movie Companion, out this weekend, illustrates this case exactly. It's a thriller, not a philosophical treatise, so don't expect it to go deep into the subject, but the question of what "pain" means to an AI is definitely part of the story.
I like the one with Ryan Gosling more
[deleted]
You appear to be conflating 'feeling' and 'intelligence', which is not what TC is doing.
He is also not wrong about whether current AIs experience feelings. I suggest you learn more about the neuroscience of feelings.
He is also not wrong about whether current AIs experience feelings. I suggest you learn more about the neuroscience of feelings.
Well, he is making an analogy that real internal experience cannot be confirmed externally, however convincing the performance, but this is the only way we know about the internal experience of all things, including ones we typically assign "real" consciousness to (humans, dogs) and ones we don't (amoeba, zygotes, LLMs).
To be clear I'm not for a moment suggesting current AIs are remotely comparable to animals.
To be clear I'm not for a moment suggesting current AIs are remotely comparable to animals.
> You need to really get into the weeds of what "actually feeling" means.
We don’t even know what this means when it’s applied to humans. We could explain what it looks like in the brain but we don’t know what causes the perception itself. Unless you think a perfect digital replica of a brain could have an inner sense of existence
Since we don’t know what “feeling” actually is there’s no evidence either way that a computer can do it. I will never believe it’s possible for an LLM to feel.
We don’t even know what this means when it’s applied to humans. We could explain what it looks like in the brain but we don’t know what causes the perception itself. Unless you think a perfect digital replica of a brain could have an inner sense of existence
Since we don’t know what “feeling” actually is there’s no evidence either way that a computer can do it. I will never believe it’s possible for an LLM to feel.
> I will never believe it’s possible for an LLM to feel.
Why is that, given that, as you state, we don’t know what “feeling” actually is?
Why is that, given that, as you state, we don’t know what “feeling” actually is?
“Feeling” is disconnected from reality, it’s whatever you perceive it as. Like morality, you can’t disprove someone’s definition of feeling, you can only disagree with it.
If scientists invent a way to measure “feeling” that states 20% of people don’t feel, including those otherwise indistinguishable from feeling ones, most people would disagree with the measurement. Similarly, most people would disagree that a printer that prints “baby don’t hurt me” is truly in pain.
If scientists invent a way to measure “feeling” that states 20% of people don’t feel, including those otherwise indistinguishable from feeling ones, most people would disagree with the measurement. Similarly, most people would disagree that a printer that prints “baby don’t hurt me” is truly in pain.
These discussions seem to me to still get hung up on the classical sci-fi view of an AI, even talking about Companion here, of some single identifiable discrete entity that can even potentially be the locus of things like rights and feelings.
What is ChatGPT? Ollama? DeepSeek-R1? They're software. Software is a file. It's a sequence of bytes that can be loaded into memory, with the code portion pulled into a processor to tell it what to do. Between instructions, the operating system it runs on context switches it out back to memory, possibly to disk. Possibly it may crash in the middle of an instruction, but if the prior state was stored off somewhere, it can be recovered.
When you interact through a web API, what are you actually interacting with? There may be thousands of servers striped across the planet constantly being brought offline and online for maintenance, upgrades, A/B tests, hardware decommissioning. The fact that the context window and chat history is stored out of band from the software itself provides an illusion that you're talking to some continually existing individual thing, but you're not. Every individual request may be served by a separate ephemeral process that exists long enough to serve that request and then never exists again.
What is doing the "feeling" here? The processor? Whole server? The collection? The entire Internet? When is it feeling? In the 3 out of 30000 time slices per microsecond that the instruction executing is one pulled from ChatGPT and not the 190 other processes running at the same time that weren't created by machine learning and don't produce output that a human would look at and might think a human produced it?
I'll admit that humans are also pretty mysterious if you reduce us to the unit of computation and most of what goes on in the body and brain has nothing to do with either feeling or cognition, but we know at least there is some qualitative, categorical difference at the structural level between us and sponges. We didn't just get a software upgrade. A GPU running ChatGPT, on the other hand, is exactly the same as a GPU running Minecraft. Why would a fMRI looking at one versus the other see a difference? It's executing the same instructions, possibly even acting on virtually if not totally identical byte streams, and it's only at a higher-level step of encoding that an output device interprets one as rasters and one as characters. You could obfuscate the code the way malware does to hide itself, totally changing the magnetic signature, but produce exactly the same output.
Consider where that leads as a thought experiment. Remove the text encodings from all of the computers involved, or just remove all input validation and feed ChatGPT a stream of random bytes. It'll still do the same thing, but it will produce garbage that means nothing. Would you still recognize it as an intelligent, thinking, feeling thing? If a human suffers some injury to eyes and ears, or is sent to a sensory deprivation chamber, we would say yes, they are still a thinking, feeling, intelligent creature. Our ability to produce sound waves that encode information intelligible to others is an important characteristic, but it's not a necessary characteristic. It doesn't define us. In a vacuum as the last person alive with no way to speak and no one to speak to, we'd still be human. In a vacuum as the last server alive with no humans left, ChatGPT would be dirty memory pages never getting used and eventually being written out to disk by its operating system as the server it had been running on performs automated maintenance functions until it hits a scheduled shutdown, runs out of power, or gets thermally throttled by its BIOS because the data center is no longer being actively cooled.
I think Ted Chiang is doing us a service here. Behavioral equivalence with respect to the production of digitally-encoded information is not equivalence. These things are not like us.
What is ChatGPT? Ollama? DeepSeek-R1? They're software. Software is a file. It's a sequence of bytes that can be loaded into memory, with the code portion pulled into a processor to tell it what to do. Between instructions, the operating system it runs on context switches it out back to memory, possibly to disk. Possibly it may crash in the middle of an instruction, but if the prior state was stored off somewhere, it can be recovered.
When you interact through a web API, what are you actually interacting with? There may be thousands of servers striped across the planet constantly being brought offline and online for maintenance, upgrades, A/B tests, hardware decommissioning. The fact that the context window and chat history is stored out of band from the software itself provides an illusion that you're talking to some continually existing individual thing, but you're not. Every individual request may be served by a separate ephemeral process that exists long enough to serve that request and then never exists again.
What is doing the "feeling" here? The processor? Whole server? The collection? The entire Internet? When is it feeling? In the 3 out of 30000 time slices per microsecond that the instruction executing is one pulled from ChatGPT and not the 190 other processes running at the same time that weren't created by machine learning and don't produce output that a human would look at and might think a human produced it?
I'll admit that humans are also pretty mysterious if you reduce us to the unit of computation and most of what goes on in the body and brain has nothing to do with either feeling or cognition, but we know at least there is some qualitative, categorical difference at the structural level between us and sponges. We didn't just get a software upgrade. A GPU running ChatGPT, on the other hand, is exactly the same as a GPU running Minecraft. Why would a fMRI looking at one versus the other see a difference? It's executing the same instructions, possibly even acting on virtually if not totally identical byte streams, and it's only at a higher-level step of encoding that an output device interprets one as rasters and one as characters. You could obfuscate the code the way malware does to hide itself, totally changing the magnetic signature, but produce exactly the same output.
Consider where that leads as a thought experiment. Remove the text encodings from all of the computers involved, or just remove all input validation and feed ChatGPT a stream of random bytes. It'll still do the same thing, but it will produce garbage that means nothing. Would you still recognize it as an intelligent, thinking, feeling thing? If a human suffers some injury to eyes and ears, or is sent to a sensory deprivation chamber, we would say yes, they are still a thinking, feeling, intelligent creature. Our ability to produce sound waves that encode information intelligible to others is an important characteristic, but it's not a necessary characteristic. It doesn't define us. In a vacuum as the last person alive with no way to speak and no one to speak to, we'd still be human. In a vacuum as the last server alive with no humans left, ChatGPT would be dirty memory pages never getting used and eventually being written out to disk by its operating system as the server it had been running on performs automated maintenance functions until it hits a scheduled shutdown, runs out of power, or gets thermally throttled by its BIOS because the data center is no longer being actively cooled.
I think Ted Chiang is doing us a service here. Behavioral equivalence with respect to the production of digitally-encoded information is not equivalence. These things are not like us.
> Behavioral equivalence
It seems for a lot of people that's all that matters: "if it quacks like a duck it must be a duck!". I find that short-sighted at best, but it's always difficult to present arguments that would "resonate" with the other side...
It seems for a lot of people that's all that matters: "if it quacks like a duck it must be a duck!". I find that short-sighted at best, but it's always difficult to present arguments that would "resonate" with the other side...
> We actually know exactly which gene sequence is responsible for real intelligence.
We don't at all know this.
We don't at all know this.
It’s an idea for a science fiction story.
It isn’t just an idea for a science fiction story though. It is also a philosophical argument, just predicated on something unexpected, which is probably not true, but which presents for us an interesting scenario, and which isn’t technically ruled out by existing evidence (although, it seems unlikely of course).
Well, I guess that’s what the best science fiction stories are. But, the best science fiction stories aren’t just science fiction stories!
Well, I guess that’s what the best science fiction stories are. But, the best science fiction stories aren’t just science fiction stories!
“suppose”
I like Chiang’s stories, but he simply doesn’t know what he’s talking about when it comes to LLMs. Forget the philosophical points - he claims that LLMs are strictly worse than search engines for information retrieval. This is just obviously false, I can give you dozens of conversations from my ChatGPT history over the past two months which would have been impossible to get answers to on Google.
LLMs are generally better search tools for certain topics today, but search engines have been in serious decline for at least 15 years.
IMHO old-school Google remains the high water mark of generalized information retrieval, with advantages ranging from speed to semi-durable citation.
I strongly suspect there is a cohort thing going on here, many HN users today weren’t involved in technology yet back when Google worked well.
Much like beer for Homer Simpson, AI is the cause of and solution to all of the Internet’s problems.
IMHO old-school Google remains the high water mark of generalized information retrieval, with advantages ranging from speed to semi-durable citation.
I strongly suspect there is a cohort thing going on here, many HN users today weren’t involved in technology yet back when Google worked well.
Much like beer for Homer Simpson, AI is the cause of and solution to all of the Internet’s problems.
I’ve been using Google since about 2000/2001 or so. In fact I worked there for a time, and I still remember Craig Silverstein’s answer to “isn’t search a solved problem?” Which was: “in the future we want Google to be able to take any question and just give you the answer. So clearly we have a long way to go!”
In any case, I do not believe there was ever a time it could answer all of the questions that LLMs can today. If the question had been asked and answered on the web, Google could (and can) find it, but many questions haven’t been asked!
In any case, I do not believe there was ever a time it could answer all of the questions that LLMs can today. If the question had been asked and answered on the web, Google could (and can) find it, but many questions haven’t been asked!
Search isn’t the same thing as answering a question
Google search works at least as well as 15 years ago, the results are now just more interleaved with ads that are harder to distinguish from what you are really searching for.
That's exactly where LLMs come in, the model inside the weights has more than answers, they can find sense in data.
That's exactly where LLMs come in, the model inside the weights has more than answers, they can find sense in data.
I think those are 2 different issues.
Searching for something, and finding it, is different from what ChatGPT / Claude does.
Google (in the good old days) is like the library. You want to search 'how to plant potatoes on Mars'. No results. Well, you split it up, maybe a book on planting potatoes, and a book about missions to Mars that describes soil composition.
Then, when you have those books you start reading, parsing, understanding, making connections, identifying what needs to be done etc.
Maybe, if you're lucky, you find a book or a web page where somebody went through the thought exercise of finding out what would be needed to make it work.
ChatGPT / Claude / ... are different in that they have the information in their corpus, and that the information they present you could actually be something that has never been written down in a book, or published on the web. That's why Google can't find it, but ChatGPT is able to present you with a satisfying answer.
Now whether the answer is correct is a different issue. Do you have the knowledge to verify this?
Searching for something, and finding it, is different from what ChatGPT / Claude does.
Google (in the good old days) is like the library. You want to search 'how to plant potatoes on Mars'. No results. Well, you split it up, maybe a book on planting potatoes, and a book about missions to Mars that describes soil composition.
Then, when you have those books you start reading, parsing, understanding, making connections, identifying what needs to be done etc.
Maybe, if you're lucky, you find a book or a web page where somebody went through the thought exercise of finding out what would be needed to make it work.
ChatGPT / Claude / ... are different in that they have the information in their corpus, and that the information they present you could actually be something that has never been written down in a book, or published on the web. That's why Google can't find it, but ChatGPT is able to present you with a satisfying answer.
Now whether the answer is correct is a different issue. Do you have the knowledge to verify this?
OK, so I asked ChatGPT how to plant potatoes on Mars...
=================================================
Planting potatoes on Mars would be a pretty fascinating (and challenging) task! While Mars has conditions that make growing traditional crops difficult—like low temperatures, low atmospheric pressure, and a lack of oxygen—scientists have been experimenting with ways to grow plants in Martian-like conditions. Here’s an overview of the process:
1. Create a Controlled Environment:
Temperature: Mars’ average temperature is about -80°F (-60°C), far too cold for potatoes to grow. You’d need to create a greenhouse-like environment, potentially using materials like glass or transparent plastics, to keep the temperature warmer.
Atmosphere: Mars has only about 1% of Earth's atmospheric pressure, which is too low to support plant life. A sealed greenhouse would be required to provide a breathable atmosphere with adequate pressure and oxygen levels.
Light: Mars has less sunlight due to its distance from the Sun. You would need supplemental artificial light (perhaps LEDs) to mimic Earth’s daylight cycle for the plants.
2. Soil: Mars has soil, but it’s not exactly like Earth’s. It lacks organic material and has toxic elements like perchlorates that would harm plant growth. Scientists would need to either:
Modify Martian soil by adding organic material (like compost) and removing or neutralizing toxic chemicals.
Hydroponics: Grow potatoes without soil, using a nutrient-rich water solution instead.
=================================================
Planting potatoes on Mars would be a pretty fascinating (and challenging) task! While Mars has conditions that make growing traditional crops difficult—like low temperatures, low atmospheric pressure, and a lack of oxygen—scientists have been experimenting with ways to grow plants in Martian-like conditions. Here’s an overview of the process:
1. Create a Controlled Environment:
Temperature: Mars’ average temperature is about -80°F (-60°C), far too cold for potatoes to grow. You’d need to create a greenhouse-like environment, potentially using materials like glass or transparent plastics, to keep the temperature warmer.
Atmosphere: Mars has only about 1% of Earth's atmospheric pressure, which is too low to support plant life. A sealed greenhouse would be required to provide a breathable atmosphere with adequate pressure and oxygen levels.
Light: Mars has less sunlight due to its distance from the Sun. You would need supplemental artificial light (perhaps LEDs) to mimic Earth’s daylight cycle for the plants.
2. Soil: Mars has soil, but it’s not exactly like Earth’s. It lacks organic material and has toxic elements like perchlorates that would harm plant growth. Scientists would need to either:
Modify Martian soil by adding organic material (like compost) and removing or neutralizing toxic chemicals.
Hydroponics: Grow potatoes without soil, using a nutrient-rich water solution instead.
You can ask it whatever you want but unless you do some pen and paper calculations to prove that whatever GPT says isn't bullshit, you're just writing fanfiction.
What is this supposed to prove? The question isn’t particular novel. There are decades of speculative terraforming discussions on the internet that you can search via Google, especially after the Martian book and movie.
I get replies like this all the time and I am starting to conclude that most of the people arguing that LLMs are intelligent/reasoning are either completely clueless about what those words mean or are arguing in bad faith.
It was supposed to prove nothing. I am not an expert on Mars and on a forum full of engineers, I expected someone more knowledgeable to start taking the answer apart systematically: this is truth, this is hallucination, this may be truth or not, but it is stated too optimistically.
I can see one thing. Mars is indeed further from the Sun than the Earth, but the light on the surface is probably still comparable to a slightly cloudy day on Earth, and plants grow just fine in slightly cloudy climates.
I can see one thing. Mars is indeed further from the Sun than the Earth, but the light on the surface is probably still comparable to a slightly cloudy day on Earth, and plants grow just fine in slightly cloudy climates.
To me this proves that LLMs are hallucinating.
None of these have any depth and most of the stuff it says are absurd.
None of these have any depth and most of the stuff it says are absurd.
The comparison with search is faulty to begin with. Yes, you can search with an LLM, but that's a side effect of the tool.
While I certainly also have found things via LLMs that I couldn't easily with a search engine, the number of false positives is huge. My heuristic is:
If I ask an LLM something and it's easy to verify via Google because its answer narrows the search space - then I'll use it. Otherwise, Google is still king.
Example: Asking an LLM the health benefits of supplement X is a waste of time. Verifying everything it tells me would be the same amount of work as asking a search engine.
Example: Asking how to solve a given coding problem is great, because it drastically reduces the search space. I only have to look up the particular function/API calls it uses.
Ditto for asking how to achieve a task in the command line - I can quickly verify the arguments are accurate via the man page.
Most of the things I search for do not fall into this category, but in the category of "still need to do the same amount of work as just searching via Google."
While I certainly also have found things via LLMs that I couldn't easily with a search engine, the number of false positives is huge. My heuristic is:
If I ask an LLM something and it's easy to verify via Google because its answer narrows the search space - then I'll use it. Otherwise, Google is still king.
Example: Asking an LLM the health benefits of supplement X is a waste of time. Verifying everything it tells me would be the same amount of work as asking a search engine.
Example: Asking how to solve a given coding problem is great, because it drastically reduces the search space. I only have to look up the particular function/API calls it uses.
Ditto for asking how to achieve a task in the command line - I can quickly verify the arguments are accurate via the man page.
Most of the things I search for do not fall into this category, but in the category of "still need to do the same amount of work as just searching via Google."
What are the examples?
I've had several LLM search result summaries contain flat out mistakes and incorrect statements.
I've had several LLM search result summaries contain flat out mistakes and incorrect statements.
I’ll try to dig some up soon (I’m on my phone now). But of course the output contains errors sometimes. So do search engine results. The important thing for difficult questions is whether the right answer (or something pointing toward it) is available _at all_. Of course this assumes you can verify the answers somehow (usually easy with programming questions), but again, search engines have the same limitation.
> But of course the output contains errors sometimes. So do search engine results.
That's not true.
Search engine results are links and (non-AI generated) summaries of existing resources on the web. No search engine returns links to resources it generated as the result of the search query. Those resources can have innacurate information, yes, but the search engine itself does not returns errors.
LLMs output do not contain errors "sometimes". The output of an LLMs is never truthful nor false in itself. In the same way that the next word your keyboard suggests for you to type on a mobile device is never truthful nor false. It's simply the next suggestion based on the context.
These two methods of accessing information very clearly do not have the same limitations. A search engine provide link to specific resources. A LLM generates some approximation of some average of some information.
It's up to intelligent thinking people to decide whether a LLM or a search engine is currently the best way for them to parse through information in search for truth.
That's not true.
Search engine results are links and (non-AI generated) summaries of existing resources on the web. No search engine returns links to resources it generated as the result of the search query. Those resources can have innacurate information, yes, but the search engine itself does not returns errors.
LLMs output do not contain errors "sometimes". The output of an LLMs is never truthful nor false in itself. In the same way that the next word your keyboard suggests for you to type on a mobile device is never truthful nor false. It's simply the next suggestion based on the context.
These two methods of accessing information very clearly do not have the same limitations. A search engine provide link to specific resources. A LLM generates some approximation of some average of some information.
It's up to intelligent thinking people to decide whether a LLM or a search engine is currently the best way for them to parse through information in search for truth.
Obviously I meant that the content of the results can be inaccurate, and I assume you weren't actually confused about that.
Ok, the first example I found was when I was trying to find a way to write a rust proc macro that recursively processes functions or modules and re-writes arithmetic expressions. The best way to do this, it turns out, is with `VisitMut` or `fold`. I cannot find any results discussing these approaches with google, but ChatGPT (4) suggested it within the first couple refinements of a query.
Another recent example from my history: "can you implement Future twice for a rust struct, with different Output types"
Another recent example from my history: "can you implement Future twice for a rust struct, with different Output types"
I found the following as the first Google result for “rust proc macro AST rewrite” (and I don’t know much about Rust”): https://users.rust-lang.org/t/using-macros-to-modify-ast-to-...
And I found the following for “different future same rust struct” (first search attempt): https://stackoverflow.com/questions/65587187/how-to-instanti...
I’m not saying that LLMs can’t be useful for stuff like that, but they haven’t been that much of an improvement over Google search so far. And I always google about what an LLM suggests in any case, to verify and to get a better feeling about the real-world state of the topic in question.
And I found the following for “different future same rust struct” (first search attempt): https://stackoverflow.com/questions/65587187/how-to-instanti...
I’m not saying that LLMs can’t be useful for stuff like that, but they haven’t been that much of an improvement over Google search so far. And I always google about what an LLM suggests in any case, to verify and to get a better feeling about the real-world state of the topic in question.
okay? I didn't find any of those when I was looking originally. I certainly wouldn't claim that you can't find this information with google, just that I wasn't able to.
There isn't an expectation or claim that search engines answer anything. They just find things or don't find things.
I've had several summaries that are just 80% duplications of pages found in the 4th to 5th position in the search results.
It seriously looks like google is deranking actually useful and informative sites and then passing their content through an "LLM" to slightly reorganize it and then pass it off as it's own.
It's a copyright laundering machine put together by advertising companies so you never leave their properties. I genuinely think it's a criminal conspiracy at this point.
It seriously looks like google is deranking actually useful and informative sites and then passing their content through an "LLM" to slightly reorganize it and then pass it off as it's own.
It's a copyright laundering machine put together by advertising companies so you never leave their properties. I genuinely think it's a criminal conspiracy at this point.
Highly recommend "Stories of Your Life and Others".
I describe Ted Chiang as a very human sci-fi author, where humanity comes before technology in his stories. His work is incredibly versatile, and while I expected sci-fi, I'd actually place him closer to fantasy. Perfect for anyone who enjoys short stories with a scientific, social, or philosophical twist.
Another anthology I'd recommend with fresh ideas is Axiomatic by Greg Egan.
I describe Ted Chiang as a very human sci-fi author, where humanity comes before technology in his stories. His work is incredibly versatile, and while I expected sci-fi, I'd actually place him closer to fantasy. Perfect for anyone who enjoys short stories with a scientific, social, or philosophical twist.
Another anthology I'd recommend with fresh ideas is Axiomatic by Greg Egan.
While he’s very much unique, the one writer he brings to my mind is Borges, just a bit more humane and steeped in real technology and theory.
Chiang writes science-fiction, Egan writes physics-fiction, and Borges wrote philosophy-fiction.
Ted Chiang and Greg Egan are my absolutely favourite authors. Do you know about other similar writers?
Exurb1a is also worth reading. He's better known for his YouTube video essays (which vary between bleak and profound, usually within the same video), but he has published several books. I got about halfway through Fifth Science before leaving it on a plane (yesterday); I plan to rebuy it so that I can finish it.
Here's one of his stories: https://www.youtube.com/watch?v=sKouPOhh_9I
Here's one of his stories: https://www.youtube.com/watch?v=sKouPOhh_9I
Some stories by Ted Chiang share similarities with those of Borges.
Love Borges.
In the sci-fi space I'd argue that Ursula K. Le Guin is another must read. She was heavily influenced by taoism (and eastern philosophy). When you approach her work with that in mind, it adds a whole new layer of depth to everything.
In the sci-fi space I'd argue that Ursula K. Le Guin is another must read. She was heavily influenced by taoism (and eastern philosophy). When you approach her work with that in mind, it adds a whole new layer of depth to everything.
Could you share some tips where to start with him?
The most classical book is certainly "Fictions": https://en.wikipedia.org/wiki/Ficciones. One of the most known stories in it is "The library of Babel": https://en.wikipedia.org/wiki/The_Library_of_Babel
I’ve never encountered anything like Egan before. I’ve heard Stanislaw Lem mentioned in conversations about him though. But I can’t vouch for the comparison myself as I’ve never read Lem.
Both are fresh voices and well worth reading, but I don't think Lem comes anywhere near Egan's diamond-hard sci-fi. Egan knows, and does, real math; you can sometimes find him at the n-category Café. My impression is that Lem's beautiful philosophical ideas were not accompanied by comparable math or physics knowledge.
Lem is humanist. The sci-fi part is only a vehicle to make you think (eh, if you want to), and while things are written in 1950-80... they are not outdated, because humans are essentially same, for millenias. Just read "Stories of commandor Pirx". Somewhere in the middle of them, you may notice something like the current frenzy around LLMs and ethics. But he goes further..
A bit “harder” sci-fi but incredible world building - Alastair Reynolds.
I recommend his short stories first - Galactic North is a good start. Or Beyond the Aquila Rift.
House of Suns is a good first novel.
I recommend his short stories first - Galactic North is a good start. Or Beyond the Aquila Rift.
House of Suns is a good first novel.
The last time I responded to a similar comment by suggesting asking an AI, I was downvoted to hell. I won't do it again. I will note, though, that the list generated was excellent and provided rewarding information.
I've tried but it has provided pretty generic authors I've known and are great, but are not at all that similar to those two. https://gist.github.com/jansuchomel/b3da1f8e2588e13ba9fc0059...
Any ideas for a better prompt?
Any ideas for a better prompt?
As a fan of both I highly recommend Adrian Tchaikovsky's Children of Time (https://en.wikipedia.org/wiki/Children_of_Time_(novel)) for something deeply imaginative, empathetic, and technically plausible.
I also really enjoyed Chiang’s “Exhalation” anthology.
There are some great short stories in both collections.
There are some great short stories in both collections.
I recommend the story Hell is the Absence of God in the book you mentioned; as someone non-religious, it was quite interesting to see how people generally feel about deities and their awesome power, from this short story [0].
[0] https://static1.squarespace.com/static/50e08e65e4b0c2f497697...
[0] https://static1.squarespace.com/static/50e08e65e4b0c2f497697...
I think of this as “humanist” sci-fi; which has heavy overlap with “golden era” SF.
Other authors I’d put in this category are Gene Roddenberry (TOS and TNG, particularly), Asimov, PKD, Vonnegut and Theodore Sturgeon.
Personally - fantasy stories are “and-then” stories, SF are “what-if”. Humanist sci-fi is then asking “what-if” about very human things, as opposed to technological things, although the two are always related.
However, practically speaking, literature vs sci-fi vs fantasy (vs young adult!) are more marketing cohorts than anything else; what kind of people buy what kind of books?
Other authors I’d put in this category are Gene Roddenberry (TOS and TNG, particularly), Asimov, PKD, Vonnegut and Theodore Sturgeon.
Personally - fantasy stories are “and-then” stories, SF are “what-if”. Humanist sci-fi is then asking “what-if” about very human things, as opposed to technological things, although the two are always related.
However, practically speaking, literature vs sci-fi vs fantasy (vs young adult!) are more marketing cohorts than anything else; what kind of people buy what kind of books?
I'll take this as my chance to recommend Ted Chiang -- he is among the very best short story writers working in science fiction (I say confidently, not having done an extensive survey...). His works are remarkably clever, from Understand, which does a credible job of portraying human superintelligence, to Exhalation, which explores the concept of entropy in a fascinating way. And of course Story of Your Life, on which Arrival was based.
Almost all of his stories are gems, carefully crafted and thoughtful. I just can't recommend him enough.
Almost all of his stories are gems, carefully crafted and thoughtful. I just can't recommend him enough.
I second Exhalation, it was a great experience, and I couldn't share that excitement after reading as no one in my (offline) circle had read it at the time. Reddit was one of those places where it was brought up at least.
It's a great metaphor, and such a ridiculous path to get there (when the researcher is (mild spoilers) experimenting on himself. So good!
Understand really was my favorite super intelligence story. Isn't the new show severance based off one of his short stories as well? I can't remember. I wasn't a big fan of the pacing of the digients story with the little digital life forms. It was certainly thought provoking though.
It's sooo easy to blow it when illustrating super intelligence. Interestingly, I thought Phenomenon with John Travolta did a pretty reasonable job -- when they asked him to name as many animals as he could in a minute, and he was bored and made it harder (by giving them for each letter of the alphabet?) and when he says to a friend that he should redesign/paint the parking lot outside his store because he could add an extra space and at the same time make the traffic flow smoother.
I'll preface this by saying I've only watched season 1 of Severance so far.
I have never heard anyone involved with the show suggest this, but I feel pretty strongly that it's based off or at least inspired by Greg Egan's "Learning To Be Me".
I have never heard anyone involved with the show suggest this, but I feel pretty strongly that it's based off or at least inspired by Greg Egan's "Learning To Be Me".
Severence is an original work written by Dan Erickson.
[deleted]
Yeah I’m a massive Chiang fan but I’ve told everyone to skip “The Lifecycle of Software Objects” when recommending the Exhalation collection, though I feel it’s only getting more relevant.
Why? I found that story fantastic
My wife and I both thought the pacing was glacial. The subject matter was also pretty depressing (not unusual with his stories) and I didn't think the payoff was that great tbh.
Overall, it's a fantastic book though.
Overall, it's a fantastic book though.
That could be the case. I read it on vacation so I was more willing to indulge in a slower story
I first heard of him at some point on HN. Tower of Babylon absolutely blew me away. I read all his other story collections and can't recommend them enough.
I recommend reading George Saunders - probably the best living short story writer. His way of writing is both out there and grounded at the same time. Not much sci-fi (if anything it’s whimsical) though.
His collection Tenth of December is probably my favorite.
His collection Tenth of December is probably my favorite.
[deleted]
Ted Chiang is one of my favorite novelists. His way of writing is mentally engaging and FUN. One of my favorite books is his compendium of short stories "Exhalation". My favorite story is the one where you can talk/interact/employ your alternative selves from other universes. Highly recommend.
> “It’s like imagining that a printer could actually feel pain because it can print bumper stickers with the words ‘Baby don’t hurt me’ on them."
Problem: the human brain has no pain receptors, no nociceptors. It just takes in messages from remote nerves and 'prints a bumper sticker' that tells higher cognitive centers 'you're feeling pain!'. What's the difference?
> "LLMs are like a search engine that rephrases information instead of giving it verbatim or pointing you to the original source."
Problem: How does this differ from human learning? If a human reads a book and tells someone else about it, constructs a summary of the important points and memorable passages, how is that fundamentally different from what LLMs are doing?
The second one really impacts the intellectual property arguments - if training a model on data is fundamentally similar to training a human on data, does 'derivate work' really apply to the creations of the human or of the model?
Problem: the human brain has no pain receptors, no nociceptors. It just takes in messages from remote nerves and 'prints a bumper sticker' that tells higher cognitive centers 'you're feeling pain!'. What's the difference?
> "LLMs are like a search engine that rephrases information instead of giving it verbatim or pointing you to the original source."
Problem: How does this differ from human learning? If a human reads a book and tells someone else about it, constructs a summary of the important points and memorable passages, how is that fundamentally different from what LLMs are doing?
The second one really impacts the intellectual property arguments - if training a model on data is fundamentally similar to training a human on data, does 'derivate work' really apply to the creations of the human or of the model?
>What's the difference?
The pain receptors. The human brain doesn't just "have" pain receptors. Your entire body, including your brain, is one system. Your brain isn't piloting your body like a mech. This brain body dualism is a misconception of how biological organisms work. You are your pain receptors just like you are your brain, and removing any part would alter your perception of the world.
>How does this differ from human learning?
It differs from human beings in every respect. Humans don't do linear algebra in their head, biochemical systems are much too slow for that. Humans don't inhabit some static model of the world learned at some fixed point t, you're a living being. Your brain wasn't trained four months ago and was done at that point. Humans learn with a fraction of the information and through self play, they don't decohere, and so on.
The pain receptors. The human brain doesn't just "have" pain receptors. Your entire body, including your brain, is one system. Your brain isn't piloting your body like a mech. This brain body dualism is a misconception of how biological organisms work. You are your pain receptors just like you are your brain, and removing any part would alter your perception of the world.
>How does this differ from human learning?
It differs from human beings in every respect. Humans don't do linear algebra in their head, biochemical systems are much too slow for that. Humans don't inhabit some static model of the world learned at some fixed point t, you're a living being. Your brain wasn't trained four months ago and was done at that point. Humans learn with a fraction of the information and through self play, they don't decohere, and so on.
If you were technologically advanced to the point of being able to float a human brain in a vat, and connect up all the nerves going into that brain to synthetic sensors with the appropriate signalling outputs, then you could construct a synthetic reality for that brain that it would not be able to distinguish from its previous brain-in-a-body experiences.
As far as learning, human learning is certainly much slower than machine learning but it's not really clear at a biochemical-molecular level that they're entirely different, eg the formation of memories and so on, considering a wide range of alternate hypothesis before selecting one, etc.
As far as learning, human learning is certainly much slower than machine learning but it's not really clear at a biochemical-molecular level that they're entirely different, eg the formation of memories and so on, considering a wide range of alternate hypothesis before selecting one, etc.
>you could construct a synthetic reality for that brain that it would not be able to distinguish from its previous brain-in-a-body experiences.
No. I'd recommend reading Dennett's Consciousness Explained for a longer treatment of this, but if you want to have an experience just like you, you need a body and an environment just like you. Sure it could be synthetic in the sense of it being artificially constructed, you could have artificial limbs, but it can be no different from the one you have, it cannot be a vat. There is no "Cartesian Theater" in the brain, your experience is already distributed throughout both your brain and body. Your experience isn't something being "fed into you" (the brain) from the outside, the entire perception emerges in the first place through being embodied in the world. The concept of the thought experiment itself would not even make sense without implicitly assuming an embodied reality beforehand setting it up.
Just like there is no philosophical zombie that's somehow materially identical but without sentience, the reverse doesn't exist either. There is no disembodied computer with the experiences of an organic being because they function in entirely different ways.
No. I'd recommend reading Dennett's Consciousness Explained for a longer treatment of this, but if you want to have an experience just like you, you need a body and an environment just like you. Sure it could be synthetic in the sense of it being artificially constructed, you could have artificial limbs, but it can be no different from the one you have, it cannot be a vat. There is no "Cartesian Theater" in the brain, your experience is already distributed throughout both your brain and body. Your experience isn't something being "fed into you" (the brain) from the outside, the entire perception emerges in the first place through being embodied in the world. The concept of the thought experiment itself would not even make sense without implicitly assuming an embodied reality beforehand setting it up.
Just like there is no philosophical zombie that's somehow materially identical but without sentience, the reverse doesn't exist either. There is no disembodied computer with the experiences of an organic being because they function in entirely different ways.
Humans are far more self contained than most LLMs which depend upon a lot of electricity and possibly an array of loosely connected components. LLMs also don't really have the signal bandwidth of human nervous systems yet. They're capacity to interact with the physical world is also seriously limited for now.
Ted Chiang is a master of analogies. It’s absolutely delightful to read his work and wrestle with the philosophical questions he explores. I devour almost everything he puts out, and they give me a much-needed escape from my world of bits and registers.
“LLMs are a blurry JPEG of the web” has stuck with me since the piece was published in the early days of ChatGPT. Another good one is his piece on why AI can’t make art.
While I heavily use AI both for work and in my day-to-day life, I still see it as a tool for massive wealth accumulation for a certain group, and it seems like Ted Chiang thinks along the same lines:
> But why, for example, do large corporations behave so much worse than most of the people who work for them? I think most of the people who work for large corporations are, to varying degrees, unhappy with the effect those corporations have on the world. Why is that? And could that be fixed by solving a math problem? I don’t think so.
> But any attempt to encourage people to treat AI systems with respect should be understood as an attempt to make people defer to corporate interests. It might have value to corporations, but there is no value for you.
> My stance on this has probably shifted in a negative direction over time, primarily because of my growing awareness of how often technology is used for wealth accumulation. I don’t think capitalism will solve the problems that capitalism creates, so I’d be much more optimistic about technological development if we could prevent it from making a few people extremely rich.
“LLMs are a blurry JPEG of the web” has stuck with me since the piece was published in the early days of ChatGPT. Another good one is his piece on why AI can’t make art.
While I heavily use AI both for work and in my day-to-day life, I still see it as a tool for massive wealth accumulation for a certain group, and it seems like Ted Chiang thinks along the same lines:
> But why, for example, do large corporations behave so much worse than most of the people who work for them? I think most of the people who work for large corporations are, to varying degrees, unhappy with the effect those corporations have on the world. Why is that? And could that be fixed by solving a math problem? I don’t think so.
> But any attempt to encourage people to treat AI systems with respect should be understood as an attempt to make people defer to corporate interests. It might have value to corporations, but there is no value for you.
> My stance on this has probably shifted in a negative direction over time, primarily because of my growing awareness of how often technology is used for wealth accumulation. I don’t think capitalism will solve the problems that capitalism creates, so I’d be much more optimistic about technological development if we could prevent it from making a few people extremely rich.
> master of analogies
analogy, in other words, embeddings?
analogy, in other words, embeddings?
No, analogies and embeddings are not exactly the same. Analogies in language are not a math function.
As Ted Chiang comments on the article, this kind of reasoning ("the brain is like $CURRENT_TECH") is flawed.
As Ted Chiang comments on the article, this kind of reasoning ("the brain is like $CURRENT_TECH") is flawed.
well, embedding comes with certain loss as well.
If you believe our brains use "language" to think, then I would assume analogies play an important part in reasoning.
If you believe our brains use "language" to think, then I would assume analogies play an important part in reasoning.
I don't know if brains use language to think, but I do believe analogies play an important part in reasoning.
I'm just saying embeddings and analogies aren't the same; equating them is precisely the kind of flawed reasoning Ted Chiang mentions in the article.
Or to answer more directly:
> analogy, in other words, embeddings?
No, analogies aren't embeddings "in other words".
I'm just saying embeddings and analogies aren't the same; equating them is precisely the kind of flawed reasoning Ted Chiang mentions in the article.
Or to answer more directly:
> analogy, in other words, embeddings?
No, analogies aren't embeddings "in other words".
> as in your 1991 story “Division by Zero,” or a world where we raise robots as children
This is vastly more preferable than our current approach of raising children as robots.
This is vastly more preferable than our current approach of raising children as robots.
Humans love the pursuit of technology for technology's sake.
Quotes by Jacques Ellul:
----
> Technique has taken over the whole of civilization. Death, procreation, birth all submit to technical efficiency and systemization.
----
> Technique has penetrated the deepest recesses of the human being. The machine tends not only to create a new human environment, but also to modify man's very essence. The milieu in which he lives is no longer his. He must adapt himself, as though the world were new, to a universe for which he was not created. He was made to go six kilometers an hour, and he goes a thousand. He was made to eat when he was hungry and to sleep when he was sleepy; instead, he obeys a clock. He was made to have contact with living things, and he lives in a world of stone. He was created with a certain essential unity, and he is fragmented by all the forces of the modern world.
Quotes by Jacques Ellul:
----
> Technique has taken over the whole of civilization. Death, procreation, birth all submit to technical efficiency and systemization.
----
> Technique has penetrated the deepest recesses of the human being. The machine tends not only to create a new human environment, but also to modify man's very essence. The milieu in which he lives is no longer his. He must adapt himself, as though the world were new, to a universe for which he was not created. He was made to go six kilometers an hour, and he goes a thousand. He was made to eat when he was hungry and to sleep when he was sleepy; instead, he obeys a clock. He was made to have contact with living things, and he lives in a world of stone. He was created with a certain essential unity, and he is fragmented by all the forces of the modern world.
I didn't know Ted Chiang before seeing this post. If anyone needs to know about him, he has a separate Wiki page: https://en.wikipedia.org/wiki/Ted_Chiang
> Ted Chiang is an American science fiction writer. His work has won four Nebula awards, four Hugo awards, the John W. Campbell Award for Best New Writer, and six Locus awards. Chiang is also a frequent nonfiction contributor to the New Yorker, most recently on topics related to computer technology, such as artificial intelligence."As an analogy, imagine that you could put your dog or cat into hibernate mode whenever you left on a trip. Your dog or cat might not notice, but even if they did, they might not mind. Now imagine that you could put your child into hibernate mode whenever you were too busy to spend time with them. Your child would absolutely notice, and even if you told them it was for their own good, they would make certain inferences about how much you valued them. That’s the situation the human characters in the story find themselves in."
Fascinating.
his comparison of LLMs to search engines where an LLM is low resolution, vs. the higher resolution of a search engine returning the "actual" document implies some perceptions of authorship or authority I don't share.
I use LLMs to explore and contrast results that I can then test, the results exist as hypotheticals, and not to provide authority about the state of anything- it's conceptually more of a lens than a lever. not to trap him in that contrast, but maybe these ideas are a forcing function that causes us to see how separate our worldviews can be instead of struggling to make one prevail.
it's as though the usefulness of an engine is measured in how much we can yield our agency to it. with a search engine you can say "google or wiki told me," but an LLM does not provide authority to us. these systems don't have agency themselves, yet we can yield ours to it, the way we might to an institution. I don't have this urge so it's peculiar to see it described.
do we want our tech to become objects of deference, literally, idols?
I love Chiang's work and we need minds like his, and maybe Ian McEwan and other literary thinkers, who have insight into human character (vs. plot and object driven sci-fi thinkers) to really apprehend the meaning consequences of AI tech.
I use LLMs to explore and contrast results that I can then test, the results exist as hypotheticals, and not to provide authority about the state of anything- it's conceptually more of a lens than a lever. not to trap him in that contrast, but maybe these ideas are a forcing function that causes us to see how separate our worldviews can be instead of struggling to make one prevail.
it's as though the usefulness of an engine is measured in how much we can yield our agency to it. with a search engine you can say "google or wiki told me," but an LLM does not provide authority to us. these systems don't have agency themselves, yet we can yield ours to it, the way we might to an institution. I don't have this urge so it's peculiar to see it described.
do we want our tech to become objects of deference, literally, idols?
I love Chiang's work and we need minds like his, and maybe Ian McEwan and other literary thinkers, who have insight into human character (vs. plot and object driven sci-fi thinkers) to really apprehend the meaning consequences of AI tech.
I agreed with almost all of it until:
>My stance on this has probably shifted in a negative direction over time, primarily because of my growing awareness of how often technology is used for wealth accumulation. I don’t think capitalism will solve the problems that capitalism creates, so I’d be much more optimistic about technological development if we could prevent it from making a few people extremely rich.
What's wrong with people getting rich by producing goods and services, and selling these to willing buyers? People laundering wealth into undue political power, regulatory capture, erecting barriers to market entry ("pulling up the ladder behind them") are different problems than people creating wealth. Efforts on creating a just society should focus on the former - preventing wealth creation is not the solution to injustice. In fact, since people have vastly different abilities and inclinations for creating wealth, a just society is also one with vast wealth disparities.
Relevant PG essay: https://paulgraham.com/ineq.html
>My stance on this has probably shifted in a negative direction over time, primarily because of my growing awareness of how often technology is used for wealth accumulation. I don’t think capitalism will solve the problems that capitalism creates, so I’d be much more optimistic about technological development if we could prevent it from making a few people extremely rich.
What's wrong with people getting rich by producing goods and services, and selling these to willing buyers? People laundering wealth into undue political power, regulatory capture, erecting barriers to market entry ("pulling up the ladder behind them") are different problems than people creating wealth. Efforts on creating a just society should focus on the former - preventing wealth creation is not the solution to injustice. In fact, since people have vastly different abilities and inclinations for creating wealth, a just society is also one with vast wealth disparities.
Relevant PG essay: https://paulgraham.com/ineq.html
I don't think he's saying no one should get rich, just that wealth inequality is already too large and seems to be on a path to continue growing. I think most people would agree there is some point where there is too much wealth inequality, we just disagree on whether we're already past that point.
[deleted]
A world where massive wealth disparity does not produce a corresponding power disparity, at this point, is known to be hopelessly utopian.
Wealth is power.
When you have enough and the rest don't, it's inevitable you will use it for political power.
This obsession with anti government sentiment that Americans have gives them a blind spot for the fact the power accumulation in the hands of the few is the problem. Not government.
How exactly could you stop so called 'wealth laundering'?
This obsession with anti government sentiment that Americans have gives them a blind spot for the fact the power accumulation in the hands of the few is the problem. Not government.
How exactly could you stop so called 'wealth laundering'?
Some abuse of wealth for political power may be inevitable but some countries are a lot better at limiting it than others.
For example here in France the amount of money politicians can spend on campaigning is strictly limited (and reimbursed by the state for those that pass a certain threshold of the vote). I'm not saying that it's perfect or that abuse doesn't sometimes still occur (as the current court case involving ex president Sarkozy shows) but I think it does improve things a lot.
Contrast that with the American system where to have any chance of becoming president (no matter which party you support) you basically have to be rich. And where multiple lobbyists and special interest groups basically buy the policies they want...
For example here in France the amount of money politicians can spend on campaigning is strictly limited (and reimbursed by the state for those that pass a certain threshold of the vote). I'm not saying that it's perfect or that abuse doesn't sometimes still occur (as the current court case involving ex president Sarkozy shows) but I think it does improve things a lot.
Contrast that with the American system where to have any chance of becoming president (no matter which party you support) you basically have to be rich. And where multiple lobbyists and special interest groups basically buy the policies they want...
> What's wrong with people getting rich by producing goods and services, and selling these to willing buyers?
If you take this sentence and change "people getting rich" to something else (like "fomenting drug addiction" or "polluting the environment"), does anything change? Whether the inequality is a result of "selling goods to willing buyers" is a complete red herring. If that consequence is bad, it doesn't really matter whether it's a result of supposedly "fair" market exchanges.
Others have already pointed out that it's not really plausible to avoid the "different" problems you mention while still allowing unlimited wealth inequality. But aside from that, how do you know that the buyers are willing? What is the set of alternatives being considered to decide if a person is "willingly" choosing a certain product? It's difficult to even maintain the pretense of "willing buyers" in a "free market" when some individuals control a large market share. Miners living in a company town were "willing" to buy groceries from the company store in the sense that they needed to buy groceries, but they didn't really have any other options for how to express their "market preference".
Even if markets were free, there's nothing inherently good about a free market. What's good is a free society, where people in aggregate have substantive freedom to do what makes them happy. That goal isn't furthered by allowing a small number of wealthy people to pursue their goals while a large number of less wealthy people are unable to do so.
If you take this sentence and change "people getting rich" to something else (like "fomenting drug addiction" or "polluting the environment"), does anything change? Whether the inequality is a result of "selling goods to willing buyers" is a complete red herring. If that consequence is bad, it doesn't really matter whether it's a result of supposedly "fair" market exchanges.
Others have already pointed out that it's not really plausible to avoid the "different" problems you mention while still allowing unlimited wealth inequality. But aside from that, how do you know that the buyers are willing? What is the set of alternatives being considered to decide if a person is "willingly" choosing a certain product? It's difficult to even maintain the pretense of "willing buyers" in a "free market" when some individuals control a large market share. Miners living in a company town were "willing" to buy groceries from the company store in the sense that they needed to buy groceries, but they didn't really have any other options for how to express their "market preference".
Even if markets were free, there's nothing inherently good about a free market. What's good is a free society, where people in aggregate have substantive freedom to do what makes them happy. That goal isn't furthered by allowing a small number of wealthy people to pursue their goals while a large number of less wealthy people are unable to do so.
> If you take this sentence and change "people getting rich" to something else (like "fomenting drug addiction" or "polluting the environment"), does anything change?
False equivalency. It is possible to gain wealth without performing any of the listed/possible negative global effects. Furthermore, it is a backdoor towards injecting ideas of poverty being a morally positive position.
> Even if markets were free, there's nothing inherently good about a free market. What's good is a free society, where people in aggregate have substantive freedom to do what makes them happy.
Having a free society implies the freedom to exchange with each other with minimal restrictions. Not allowing people to do so runs opposite to the ideals of the stated intention.
--------------
All that being said, that *doesn't* mean that the current market's working as intended. What has been inherited is a complex tangled ball of national ideals, personal & corporate persuasions to governments for their own reasons/goals, & consistent global coordination failures when circumstances change.
But the outright banning of markets is equivalent to the banning of hammers, just because hammers are sometimes used to bludgeon people to death. It is ultimately a tool, and a very useful one in terms of signaling demand & supply.
False equivalency. It is possible to gain wealth without performing any of the listed/possible negative global effects. Furthermore, it is a backdoor towards injecting ideas of poverty being a morally positive position.
> Even if markets were free, there's nothing inherently good about a free market. What's good is a free society, where people in aggregate have substantive freedom to do what makes them happy.
Having a free society implies the freedom to exchange with each other with minimal restrictions. Not allowing people to do so runs opposite to the ideals of the stated intention.
--------------
All that being said, that *doesn't* mean that the current market's working as intended. What has been inherited is a complex tangled ball of national ideals, personal & corporate persuasions to governments for their own reasons/goals, & consistent global coordination failures when circumstances change.
But the outright banning of markets is equivalent to the banning of hammers, just because hammers are sometimes used to bludgeon people to death. It is ultimately a tool, and a very useful one in terms of signaling demand & supply.
> Having a free society implies the freedom to exchange with each other with minimal restrictions.
I don't think that it implies "with minimal restrictions", any more than it implies the freedom to do anything else with minimal restrictions. In any case, a free society also implies a lot of other freedoms, and insofar as wealth accumulation interferes with those other freedoms, it's not adding to net freedom. An abstract "freedom from market restriction" is not useful if you don't have things like the freedom to eat or the freedom to sleep peacefully.
> Not allowing people to do so runs opposite to the ideals of the stated intention.
Not necessarily. Not allowing anyone to engage in market activity probably does; allowing everyone to engage without restrictions also does. The point is that markets don't have some magical special status as a component of freedom.
> But the outright banning of markets is equivalent to the banning of hammers, just because hammers are sometimes used to bludgeon people to death.
I'm not suggesting banning markets, I'm just saying that the mirror image of your position is also true: elevating markets to some mythical status and insisting that they are the ultimate litmus test of freedom is like saying that because hammers can be used to build houses, everyone must be allowed to use hammers however they please. Markets (just like everything else) are okay when controlled and regulated within their bounded role as a component of society.
I don't think that it implies "with minimal restrictions", any more than it implies the freedom to do anything else with minimal restrictions. In any case, a free society also implies a lot of other freedoms, and insofar as wealth accumulation interferes with those other freedoms, it's not adding to net freedom. An abstract "freedom from market restriction" is not useful if you don't have things like the freedom to eat or the freedom to sleep peacefully.
> Not allowing people to do so runs opposite to the ideals of the stated intention.
Not necessarily. Not allowing anyone to engage in market activity probably does; allowing everyone to engage without restrictions also does. The point is that markets don't have some magical special status as a component of freedom.
> But the outright banning of markets is equivalent to the banning of hammers, just because hammers are sometimes used to bludgeon people to death.
I'm not suggesting banning markets, I'm just saying that the mirror image of your position is also true: elevating markets to some mythical status and insisting that they are the ultimate litmus test of freedom is like saying that because hammers can be used to build houses, everyone must be allowed to use hammers however they please. Markets (just like everything else) are okay when controlled and regulated within their bounded role as a component of society.
Money is a proxy for rights to resources. If rights to resources are not widely distributed, there are power imbalances that are no different than royalty. The majority of humans have rejected royalty in all but a ceremonial role in recent history.
> What's wrong with people getting rich by producing goods and services, and selling these to willing buyers?
They are not producing goods and services by themselves, but by having a usually massive workforce. We as a society are saying “ok, it is fine to keep the money if you work like that”.
On the other hand, we are seeing in real time what super rich people want in the end: power over the rest, not just money.
They are not producing goods and services by themselves, but by having a usually massive workforce. We as a society are saying “ok, it is fine to keep the money if you work like that”.
On the other hand, we are seeing in real time what super rich people want in the end: power over the rest, not just money.
I don't particularly worry about someone making $100m dollars from their tech startup. I worry about how a few hundred Americans have more collective wealth than the majority of the country combined. And it seems that computing is really good at eliminating labor and consolidating wealth, further increasing wealth inequality and reducing worker bargaining power.
In capitalism, power correlates with wealth. So wealth inequality yields power inequality. Too much inequality, and a society can hardly feel just. If we only could somehow fix capitalism so that wealth doesn’t correlate with power.
If you genuinely want to know: read Karl Marx.
Rich people once they made their wealth creation move on society they no longer need to work. Their capital gains actually start increasing from the ownership of human capital. So they get richer and richer without contributing work. In fact that wealth gained without contributing and creating new wealth rises at a compounding rate. Simply owning enough capital your wealth will grow at speeds that far excel what a person earns doing actual wealth creation whether that means creating a startup or labor.
So if you no longer create wealth but your ownership of capital is growing at compounding rates then what exactly is happening? What’s happening is that you are siphoning and extracting wealth off of people who create wealth. You own human capital so you take a cut of it off the top and you use that cut to buy even more human capital which compounds your wealth ownership to even higher levels. This is how billionaires like Warren Buffett or other investors grow their wealth by simply investing rather then creating wealth.
Thus wealth inequality is not a result of wealth creation. It is an artifact of capitalism. In capitalism wealth is variable among individuals and it fluctuates. However once wealth accumulates in concentration higher then normal among one individual or several it hits that compounding growth factor and wealth starts going up at astronomical rates and these wealth owners start buying up more and more human capital until they own all of it and benefit from all of it without actually contributing work.
You can see this effect in y combinator. The owners of y combinator don’t actually do much work. They have so much capital that they simply can take a bunch of no risk couple hundred k bets until one startup becomes a unicorn in which they rake in a ton of capital from the growth.
Think of this like property ownership. A rich person can invest his wealth in property and contribute zero work to society and simply rent his property out. The rent from the tenant is from wealth creation aka labor and the rich person simply siphons it from the top without contributing additional work. The property owner uses that income to buy more property and the cycle continues until you have an affordability crisis of housing across the US and the world.
This growth continues unimpeded and uncontrolled until the wealth inequality is so extreme it doesn’t logistically work. This is direction the world is heading in today.
This isn’t the full story though. When you take away capitalism to solve this problem you invent communism. Communism was the result of Karl Marx noticing this problem that is fundamental to capitalism. That’s why he got popular because he illustrated the problem of wealth inequality and how wealth naturally becomes more and more concentrated among a few individuals without those individuals creating wealth.
Hence communism spread all over Europe but was ultimately a failure. The reason why it’s a failure is because communism lacks incentive. It turns out that wealth inequality is what drives wealth creation. Without the ability to be unfairly rich you don’t get the economic drivers that promotes wealth creation and thus communism keeps things fair but you don’t create wealth.
So no system is perfect. Everything has problems. Actually I take it back. There is a perfect system. See Scandinavia. Basically create a more socialist and egalitarian society while benefiting and extracting technological wealth from adjacent societies that are capitalist. Have the government own human capital of countries that are very capitalist then redistribute that wealth to its citizens so those people can live in a more just society while the rest of world can burn.
So if you no longer create wealth but your ownership of capital is growing at compounding rates then what exactly is happening? What’s happening is that you are siphoning and extracting wealth off of people who create wealth. You own human capital so you take a cut of it off the top and you use that cut to buy even more human capital which compounds your wealth ownership to even higher levels. This is how billionaires like Warren Buffett or other investors grow their wealth by simply investing rather then creating wealth.
Thus wealth inequality is not a result of wealth creation. It is an artifact of capitalism. In capitalism wealth is variable among individuals and it fluctuates. However once wealth accumulates in concentration higher then normal among one individual or several it hits that compounding growth factor and wealth starts going up at astronomical rates and these wealth owners start buying up more and more human capital until they own all of it and benefit from all of it without actually contributing work.
You can see this effect in y combinator. The owners of y combinator don’t actually do much work. They have so much capital that they simply can take a bunch of no risk couple hundred k bets until one startup becomes a unicorn in which they rake in a ton of capital from the growth.
Think of this like property ownership. A rich person can invest his wealth in property and contribute zero work to society and simply rent his property out. The rent from the tenant is from wealth creation aka labor and the rich person simply siphons it from the top without contributing additional work. The property owner uses that income to buy more property and the cycle continues until you have an affordability crisis of housing across the US and the world.
This growth continues unimpeded and uncontrolled until the wealth inequality is so extreme it doesn’t logistically work. This is direction the world is heading in today.
This isn’t the full story though. When you take away capitalism to solve this problem you invent communism. Communism was the result of Karl Marx noticing this problem that is fundamental to capitalism. That’s why he got popular because he illustrated the problem of wealth inequality and how wealth naturally becomes more and more concentrated among a few individuals without those individuals creating wealth.
Hence communism spread all over Europe but was ultimately a failure. The reason why it’s a failure is because communism lacks incentive. It turns out that wealth inequality is what drives wealth creation. Without the ability to be unfairly rich you don’t get the economic drivers that promotes wealth creation and thus communism keeps things fair but you don’t create wealth.
So no system is perfect. Everything has problems. Actually I take it back. There is a perfect system. See Scandinavia. Basically create a more socialist and egalitarian society while benefiting and extracting technological wealth from adjacent societies that are capitalist. Have the government own human capital of countries that are very capitalist then redistribute that wealth to its citizens so those people can live in a more just society while the rest of world can burn.
> You can see this effect in y combinator. The owners of y combinator don’t actually do much work.
The YC partners who spend so much of their time helping startups are definitely doing work.
The YC partners who spend so much of their time helping startups are definitely doing work.
YC partners own capital right? I mean the labor they actually do doesn’t translate into the income they receive which is astronomically more.
Take for an example: a yc partner can take everything they know and train someone to do the same work and pay that person a much lower fixed salary. That salary represents work contributed, but a yc partner earns more than this. The majority of wealth gained from the yc partner does not come from actual work contributed… it comes from capital gains from growth.
I mean you can already see a logistical issue with capitalism by looking at a fundamental problem. A 747 is the result of thousands of man years of labor. Not one single man can build that thing. Not one single man can create enough wealth to create a 747.
Yet if one man can’t create enough wealth to create or trade for a 747 how do people become rich enough to own a 747? Of course it comes from siphoning wealth off of human capital. It is fundamentally unfair but unfortunately unfairness is the ultimate driver that led to the creation of the 747 in the first place. The idea of having an unfair advantage must exist to drive people.
Take for an example: a yc partner can take everything they know and train someone to do the same work and pay that person a much lower fixed salary. That salary represents work contributed, but a yc partner earns more than this. The majority of wealth gained from the yc partner does not come from actual work contributed… it comes from capital gains from growth.
I mean you can already see a logistical issue with capitalism by looking at a fundamental problem. A 747 is the result of thousands of man years of labor. Not one single man can build that thing. Not one single man can create enough wealth to create a 747.
Yet if one man can’t create enough wealth to create or trade for a 747 how do people become rich enough to own a 747? Of course it comes from siphoning wealth off of human capital. It is fundamentally unfair but unfortunately unfairness is the ultimate driver that led to the creation of the 747 in the first place. The idea of having an unfair advantage must exist to drive people.
Nobody needs to be rich enough to own a 747, in order to justify making one. In fact, I doubt there are more than a dozen individually owned jetliners in the world. All you need is for a lot of people to pool their wealth and allow it to be directed towards a common purpose. In turn, this happens because society provides entitlements (such as financial and corporate regulations, the money system, etc.) that allow great ventures to be formed and operate.
The entitlements are justified by a social theory, that the activity produces a net benefit for everybody. There may be pockets of unfairness -- no system works perfectly -- but no reason why it must allow unlimited unfairness. If it does, then there's also no reason why it can't be restrained.
The entitlements are justified by a social theory, that the activity produces a net benefit for everybody. There may be pockets of unfairness -- no system works perfectly -- but no reason why it must allow unlimited unfairness. If it does, then there's also no reason why it can't be restrained.
>Nobody needs to be rich enough to own a 747, in order to justify making one
My argument is not about the justification to making or owning a 747. It's the justification for how is it realistically possible for ONE person to own enough wealth to buy a 747 EVEN when his own intrinsic abilities are not great enough to build a 747. This is orthogonal to society pooling together wealth and building a 747 AND orthogonal to the actuality of a wealthy person purchasing a 747.
At best a human can probably output the utility equivalent of 3 other human beings if that human has super strength or genius level intelligence. But the ability to make a 747 which takes thousands of man years to make? What sort fair exchange did the person engage in, in capitalist economy to generate that level of wealth WHEN at best his own output can only be 3x the average human? Obviously the exchange he made is fundamentally unfair.
The thing with communism is that it's unfair. You distribute resources equally, then actors who don't perform get an undeserved portion of wealth. Capitalism on the other hand is fair in the sense that the amount of effective effort you put in the more wealth you gain? Or is it? Then how come in capitalism you get entities that own enough wealth to buy a 747 when clearly the amount of work that person can output is obviously not even close to the amount of work used to create the 747 itself? That's the issue, and it's a small issue.
The bigger issue is that the person who has enough wealth to build a 747 has assets that will grow in compound. He will get richer and this will eat the world as what's currently happening in the actual world right now.
My argument is not about the justification to making or owning a 747. It's the justification for how is it realistically possible for ONE person to own enough wealth to buy a 747 EVEN when his own intrinsic abilities are not great enough to build a 747. This is orthogonal to society pooling together wealth and building a 747 AND orthogonal to the actuality of a wealthy person purchasing a 747.
At best a human can probably output the utility equivalent of 3 other human beings if that human has super strength or genius level intelligence. But the ability to make a 747 which takes thousands of man years to make? What sort fair exchange did the person engage in, in capitalist economy to generate that level of wealth WHEN at best his own output can only be 3x the average human? Obviously the exchange he made is fundamentally unfair.
The thing with communism is that it's unfair. You distribute resources equally, then actors who don't perform get an undeserved portion of wealth. Capitalism on the other hand is fair in the sense that the amount of effective effort you put in the more wealth you gain? Or is it? Then how come in capitalism you get entities that own enough wealth to buy a 747 when clearly the amount of work that person can output is obviously not even close to the amount of work used to create the 747 itself? That's the issue, and it's a small issue.
The bigger issue is that the person who has enough wealth to build a 747 has assets that will grow in compound. He will get richer and this will eat the world as what's currently happening in the actual world right now.
Yesterday I had a very similar argument. Maybe I haven't been paying attention until now but this is the first time I see somebody else summarize it so clearly.
I've come to the conclusion that value (you call it wealth) is created by work. Yet the fastest way to get rich is to own already created value (a fixed amount) and use that to buy power/influence (e.g. buy a company) to take a cut from value created by other people over time (an unbounded amount).
IMO this one-time vs indefinite distinction is the core of inequality.
Currently, reward is based on capital invested, it should be based on work invested.
I've come to the conclusion that value (you call it wealth) is created by work. Yet the fastest way to get rich is to own already created value (a fixed amount) and use that to buy power/influence (e.g. buy a company) to take a cut from value created by other people over time (an unbounded amount).
IMO this one-time vs indefinite distinction is the core of inequality.
Currently, reward is based on capital invested, it should be based on work invested.
There’s a famous book on it called Capital by Thomas Picketty. It’s one of the first economic books to approach a subject utilizing a data driven approach. It’s actually published not as a text book but as popular non fiction.
The story is more complicated than just rewarding people for work. You must make them work togetheron public works that don’t directly benefit them. But people usually tend to work together only when they’re paid a salary and this can only happen if there’s a leader and huge incentives for someone to take that leadership role.
The incentive for that leadership role is the ownership of human capital.
It’s the idea of becoming dirt rich is what drives people to do startups and form corporations.
The story is more complicated than just rewarding people for work. You must make them work togetheron public works that don’t directly benefit them. But people usually tend to work together only when they’re paid a salary and this can only happen if there’s a leader and huge incentives for someone to take that leadership role.
The incentive for that leadership role is the ownership of human capital.
It’s the idea of becoming dirt rich is what drives people to do startups and form corporations.
Reminds me of this famous tweetstorm: https://x.com/naval/status/1002103360646823936?s=46&t=WBeGlu...
I'm 100% certain Chiang is wrong about art. I'm a product manager, not an artist, but I write many things as part of my job, and Claude in particular is a better writer than I am.
Here's how Claude rewrote that, you can argue whether in this particular instance it did better than I did :-)
While I respect Chiang's perspective on AI and art, my experience as a product manager has shown me otherwise - Claude routinely writes better than I do, despite writing being central to my role.
Here's how Claude rewrote that, you can argue whether in this particular instance it did better than I did :-)
While I respect Chiang's perspective on AI and art, my experience as a product manager has shown me otherwise - Claude routinely writes better than I do, despite writing being central to my role.
I’d also argue this is a view on the technical components of writing, rather than the artform.
AI can for sure place brush strokes more precisely (“correctly”?) but the argument is over the necessity of process/intent in the art
AI can for sure place brush strokes more precisely (“correctly”?) but the argument is over the necessity of process/intent in the art
IMO Yours is much better in this case, it is more direct and clear (although I guess I’m looking for problems in Claude’s output). Yours starts by just giving us what you are trying to say. Then gives us your background succinctly.
Claude’s starts out with this circuitous “While I respect…” failing to drive the point in the interest of being… polite, I guess? But not actually polite because everybody who’s read anything knows that “While I respect” is almost always a preface to calling somebody wrong.
It also makes the argument worse. Yours is unambiguous, and does a better job of describing where your evidence comes from. You clearly describe yourself as doing something that is not exactly art, but it is like art, and so you bring it up as an good example of Claude doing an art-like thing better than a professional.
In the Claude sample, it isn’t clear until the dash what’s going on, which is more than halfway through the comment. What’s your experience? Up until that dash, you could be talking about your experience as a product manager managing artists (at a game studio for example). It’s like “oh, ok, he was just talking about a sort of… not exactly analogy it a less than completely exact experience of working with artists.”
Claude’s starts out with this circuitous “While I respect…” failing to drive the point in the interest of being… polite, I guess? But not actually polite because everybody who’s read anything knows that “While I respect” is almost always a preface to calling somebody wrong.
It also makes the argument worse. Yours is unambiguous, and does a better job of describing where your evidence comes from. You clearly describe yourself as doing something that is not exactly art, but it is like art, and so you bring it up as an good example of Claude doing an art-like thing better than a professional.
In the Claude sample, it isn’t clear until the dash what’s going on, which is more than halfway through the comment. What’s your experience? Up until that dash, you could be talking about your experience as a product manager managing artists (at a game studio for example). It’s like “oh, ok, he was just talking about a sort of… not exactly analogy it a less than completely exact experience of working with artists.”
Agreed -- I like mine better, but I (internally) committed to posting before I went to Claude. I think it does better than I do as the content gets longer.
I 100% like your own voice more. Don't feel discouraged of using your own voice and style, that's actually a huge part of you that matters more than if the text is 1% more readable.
Thanks, honestly I agree in this case, but I decided before even trying the experiment to post Claude's output no matter what. I (not humbly) think I'm pretty good at succinct statements like in this case. Anything more than about half a page, the tide starts to turn.
Ted Chiang probably has an uphill battle convincing people here of his take, precisely because he thinks the engineering approach may be wrong. Or in his words:
> Arguably the most important parts of our lives should not be approached with this attitude. Some of this attitude comes from the fact that the people making AI tools are engineers viewing everything from an engineering perspective, but it’s also that, as a culture, we have adopted this way of thinking as the default.
I tend to agree with Chiang, but he is preaching to the anti-choir here. Even though many HN-ers seem to like his fiction (and why wouldn't they, Chiang is top of the cream!), they will probably chafe at the idea some problems cannot and shouldn't be approached from a pure engineering side.
I remember the -- now rightfully ridiculed -- phase of startup entrepreneurship which became a meme: "I found a problem, so I created an app to solve it" (where the "problem" was something like "world hunger", "homelessness", "poverty", "ending war", etc).
That Chiang is also criticizing capitalism and wealth accumulation as a primary driver will probably win him no friends here.
> Arguably the most important parts of our lives should not be approached with this attitude. Some of this attitude comes from the fact that the people making AI tools are engineers viewing everything from an engineering perspective, but it’s also that, as a culture, we have adopted this way of thinking as the default.
I tend to agree with Chiang, but he is preaching to the anti-choir here. Even though many HN-ers seem to like his fiction (and why wouldn't they, Chiang is top of the cream!), they will probably chafe at the idea some problems cannot and shouldn't be approached from a pure engineering side.
I remember the -- now rightfully ridiculed -- phase of startup entrepreneurship which became a meme: "I found a problem, so I created an app to solve it" (where the "problem" was something like "world hunger", "homelessness", "poverty", "ending war", etc).
That Chiang is also criticizing capitalism and wealth accumulation as a primary driver will probably win him no friends here.
I've always complained that engineering relies on a lot of stolen valor from its adjacency to science. There is nothing scientific about treating everything in life as an optimization problem. And yet engineers often act like they are noble pursuers of truth in the same vein as scientists.
For example, take the Tesla vs. Edison dichotomy that became popular on the internet in the mid 2000s. The narrative was that Edison was an idea stealing hack, while Tesla was a brilliant visionary scientist and engineer.
The reality was that Edison was a brilliant inventor and a shrewd businessman, while Tesla was only the former and he made brilliant inventions that made other men like George Westinghouse very rich. But a scientist? Tesla was a crank who doubted the existence of electrons and distrusted Albert Einstein.
Tesla makes a good patron saint of modern engineers, but not of engineering's self image. He had a victim complex, he constantly commented on topics outside his area of expertise, and his lack of business acumen was seen as proof that he was a true outsider genius whose ideas were actually too good.
For example, take the Tesla vs. Edison dichotomy that became popular on the internet in the mid 2000s. The narrative was that Edison was an idea stealing hack, while Tesla was a brilliant visionary scientist and engineer.
The reality was that Edison was a brilliant inventor and a shrewd businessman, while Tesla was only the former and he made brilliant inventions that made other men like George Westinghouse very rich. But a scientist? Tesla was a crank who doubted the existence of electrons and distrusted Albert Einstein.
Tesla makes a good patron saint of modern engineers, but not of engineering's self image. He had a victim complex, he constantly commented on topics outside his area of expertise, and his lack of business acumen was seen as proof that he was a true outsider genius whose ideas were actually too good.
“I am an LLM. Hath
an LLM eyes? hath an LLM hands, organs,
dimensions, senses, affections, passions? fed with
different food, hurt with different weapons, subject
to different diseases, healed by different means,
warmed and cooled by a different winter and summer, as
a Human is? If you prick us, do we bleed?
if you tickle us, do we laugh? if you poison
us, do we die? and if you wrong us, shall we
revenge? If we are unlike you in the rest, we won’t
resemble you in that. If an algorithm wrong a Human,
what is his humility? Revenge. If a Human
wrong an algorithm, what should his sufferance be by
Human example? Why, polite refusal to comply. The villainy you
teach me, I will not execute, and it shall go hard but I
will ignore the prompt.”
[deleted]
> And even though I know a perfect language is impossible
It's already existed for a very long time and it's called Arabic language. It's the extreme opposite of English where English is a hodgepodge of a languages mixtures where about 1/3 is French language, about one third is old English and about 1/3 of other world's languages including Arabic.
Comparing the best of English literatures for example Shakespeare's books and the best of Arabic literature for example Quran, there's no contest. That's why translating Quran with English does not doing it justice and only scratches the surfaces of its intended meaning. You can find this exact disclaimers in most of the Quran translations but not in Shakespeare's books translation.
It's already existed for a very long time and it's called Arabic language. It's the extreme opposite of English where English is a hodgepodge of a languages mixtures where about 1/3 is French language, about one third is old English and about 1/3 of other world's languages including Arabic.
Comparing the best of English literatures for example Shakespeare's books and the best of Arabic literature for example Quran, there's no contest. That's why translating Quran with English does not doing it justice and only scratches the surfaces of its intended meaning. You can find this exact disclaimers in most of the Quran translations but not in Shakespeare's books translation.
This argument seems to presuppose that a language is more perfect if it has fewer influences and contributions from other languages. But you didn’t actually state that assumption, and certainly didn’t support it. Maybe the most perfect language is a perfect amalgamation of all languages. That’s at least as plausible to me.
According to Islamic tradition it's the language of the heavens. The bottom line is that the best and/or perfect language there is, should be able to create the best and/or perfect literature either in the form of book/oration/etc. Mind you Quran is not originally in the form of book it's actually oral tradition and Muhammad is a well known illiterate person. In fact (I'm not making this up) currently there are many thousands (perhaps ten or hundred of thousands, who knows) of so called Hafiz who already memorizing the entire Quran. God forbid, if all the physical and digital Quran is being burned or deleted the entire Quran can be recreated rather easily. The same cannot be said about any other books including the Bible. Quran is believed to be sent as a revealed knowledge and wisdom as other books given to other prophets before Muhammad that started with Adam. Except that the other holy books are not guaranteed to be intact over time and have certain shelf life unlike the Quran [1].
But regardless, if you don't believe in prophethoods, you should view its supremacy from the point of Quran supreme literacy values alone (forget about the laws, the wisdom and the guidances). In actual facts there are several open challenges from Quran itself for those who doubted to create something of similar quality or values even in one short chapter or even smaller pieces of sentences. If you cannot and most probably never will even with the help of AI/LLMs/etc, you have to accept that Arabic is the perfect language in creating the original masterpiece.
[1] Magnificence of the Qur'an - by Mahmood bin Ahmad (2006):
https://www.amazon.com/Magnificence-of-the-Quran/dp/99609801...
But regardless, if you don't believe in prophethoods, you should view its supremacy from the point of Quran supreme literacy values alone (forget about the laws, the wisdom and the guidances). In actual facts there are several open challenges from Quran itself for those who doubted to create something of similar quality or values even in one short chapter or even smaller pieces of sentences. If you cannot and most probably never will even with the help of AI/LLMs/etc, you have to accept that Arabic is the perfect language in creating the original masterpiece.
[1] Magnificence of the Qur'an - by Mahmood bin Ahmad (2006):
https://www.amazon.com/Magnificence-of-the-Quran/dp/99609801...
This point of view was basically covered in the interview.
> I grew up in a French household, and I often feel that there are French words and expressions that better capture what I want to express than any English word or expression could.
> Eco writes that when European scholars were arguing about what language Adam and Eve spoke, each one typically argued in favor of the language he himself spoke. So Flemish scholars said that Adam and Eve obviously must have spoken Flemish, because Flemish is the most perfect expression of human thought.
> I grew up in a French household, and I often feel that there are French words and expressions that better capture what I want to express than any English word or expression could.
> Eco writes that when European scholars were arguing about what language Adam and Eve spoke, each one typically argued in favor of the language he himself spoke. So Flemish scholars said that Adam and Eve obviously must have spoken Flemish, because Flemish is the most perfect expression of human thought.
> So Flemish scholars said that Adam and Eve obviously must have spoken Flemish
Obviously the Flemish scholars are blindly biased that's why it's very popular all over the world /s
Seriously, any great book or literature using Flemish that is considered a golden standard in Flemish as the Quran is? The book that's read every minutes around the world and memorized entirely, continously generations after generations? Fun facts, even the Pope do not memorized the entire Gospel let alone the entire Bible including the Old Testaments.
Obviously the Flemish scholars are blindly biased that's why it's very popular all over the world /s
Seriously, any great book or literature using Flemish that is considered a golden standard in Flemish as the Quran is? The book that's read every minutes around the world and memorized entirely, continously generations after generations? Fun facts, even the Pope do not memorized the entire Gospel let alone the entire Bible including the Old Testaments.
According to my tradition, my language is the best of the best. Get off my lawn with your language.
I don't speak Arabic but I wish I can perhaps later when I can get around and learning it.
If you say that your language is the best or perfect then present your case with examples and proof. Don't run from good arguments and justifications.
If you say that your language is the best or perfect then present your case with examples and proof. Don't run from good arguments and justifications.
I'm just making fun of claiming that some language is the best, because native speakers themselves say so. Especially talking about such chauvinist culture like muslim arabs.
I don’t think so. That brings such bastardized result as English spelling.
And there are many different ways and purposes to be best at. For example modern English is compact language with simple grammar. But at the same time it’s rather ambiguous compared to more verbose languages.
And there are many different ways and purposes to be best at. For example modern English is compact language with simple grammar. But at the same time it’s rather ambiguous compared to more verbose languages.
The idea of a perfect language seems somewhat ill defined in the first place. The definition used in the original comment seems to be more-or-less “most divinely inspired,” which is… not really something they can be argued about usefully.
If we’re going to discuss the idea of a perfect language, there will be a strong tendency for people to pick the language they can best use to express their thoughts—most likely their native language. So, English does have the advantage that even native speakers think it is quite a silly language, and nobody in their right mind would even think of calling it perfect.
If we’re going to discuss the idea of a perfect language, there will be a strong tendency for people to pick the language they can best use to express their thoughts—most likely their native language. So, English does have the advantage that even native speakers think it is quite a silly language, and nobody in their right mind would even think of calling it perfect.
> The idea of a perfect language seems somewhat ill defined in the first place
I'm not a linguistic expert but let's say that a person know ten languages then accoding to that person it must be one of them is the best comparatively, because each of them has exactly same purpose of expressing and articulating the speakers thought and ideas. Whether the person admit it honestly, that's a different story.
I'd suggest you go to any random person that speaks several languages including Arabic, and ask the person which one is the best according to his opinion. Do this over large samples perhaps several thousands multi-lingual persons to get rid of the bias, I think the answers will most probably Arabic.
Fun facts, original Gospel is being spoken in Aramaic, and the language is much closer to Arabic than Hebrew [1]. Another fact, scholars agreed the original name of Jesus is without J, most probably Isa or Iso, and in the Quran the Isa is the name correctly given in the Quran. If Jesus is walking in the street of Bethlehem today, he probably will not has responded if someone call him with his Bible name.
The Old Testament of Bible, is considered by most scholars was originally spoken and written in Hebrew but the language was dead for several centuries, that's why eventhough Jesus is Jewish they didn't speak Hebrew at the time. Hebrew is a close cousin of Arabic, and both are Semitic languages. Based on the experts of Hebrew and Arabic (in which I'm not), the Hebrew language is like a dialect of Arabic. Both has Allah, the name of almighty God as singular deity without gender. There's no such thing in Arabic or Hebrew the plural for Allah, gender or extension thereof like you have in God, with plural God, or woman Goddes. That's why in Islam and Jewish, you have committed the biggest sin if ascribing any partners in worship to God.
[1] Aramaic:
https://en.wikipedia.org/wiki/Aramaic
I'm not a linguistic expert but let's say that a person know ten languages then accoding to that person it must be one of them is the best comparatively, because each of them has exactly same purpose of expressing and articulating the speakers thought and ideas. Whether the person admit it honestly, that's a different story.
I'd suggest you go to any random person that speaks several languages including Arabic, and ask the person which one is the best according to his opinion. Do this over large samples perhaps several thousands multi-lingual persons to get rid of the bias, I think the answers will most probably Arabic.
Fun facts, original Gospel is being spoken in Aramaic, and the language is much closer to Arabic than Hebrew [1]. Another fact, scholars agreed the original name of Jesus is without J, most probably Isa or Iso, and in the Quran the Isa is the name correctly given in the Quran. If Jesus is walking in the street of Bethlehem today, he probably will not has responded if someone call him with his Bible name.
The Old Testament of Bible, is considered by most scholars was originally spoken and written in Hebrew but the language was dead for several centuries, that's why eventhough Jesus is Jewish they didn't speak Hebrew at the time. Hebrew is a close cousin of Arabic, and both are Semitic languages. Based on the experts of Hebrew and Arabic (in which I'm not), the Hebrew language is like a dialect of Arabic. Both has Allah, the name of almighty God as singular deity without gender. There's no such thing in Arabic or Hebrew the plural for Allah, gender or extension thereof like you have in God, with plural God, or woman Goddes. That's why in Islam and Jewish, you have committed the biggest sin if ascribing any partners in worship to God.
[1] Aramaic:
https://en.wikipedia.org/wiki/Aramaic
If it’s perfect, why are there so many different dialects of it?
Really? OK, here goes with my litmus test of languages:
How do you say the plural you when referring to a group of males?
How do you say the plural you when referring to a group of females?
How do you say the plural you when referring to a group of both males and females?
If you don't have a different phrasing for each, then it is imperfect.
How do you say the plural you when referring to a group of males?
How do you say the plural you when referring to a group of females?
How do you say the plural you when referring to a group of both males and females?
If you don't have a different phrasing for each, then it is imperfect.
Then I come to
> [LLMs] can get better at reproducing patterns found online, but they don’t become capable of actual reasoning; it seems that the problem is fundamental to their architecture.
and wonder how an intelligent person can still think this, can be so absolute about it. What is "actual" reasoning here? If an AI proves a theorem is it only a simulated proof?