Yann LeCun: Human-level artificial intelligence is going to take a long time(english.elpais.com)
english.elpais.com
Yann LeCun: Human-level artificial intelligence is going to take a long time
https://english.elpais.com/technology/2024-01-19/yann-lecun-chief-ai-scientist-at-meta-human-level-artificial-intelligence-is-going-to-take-a-long-time.html
360 comments
1925 of course would have been a great time to put limits on fossil fuel use in aviation along with the rest of the fossil fuel applications to manage the biggest current threat to human civilization. (Arrhenius did the science showing global warming in 1896 or so)
Given the dual use of fossil fuels between military and civilian purposes, I wonder whether any state that deliberately handicapped car/aero/petrochemicals would’ve been able to survive the early twentieth century.
Both the USA and Nazi Germany benefited massively from have a civilian industrial base that was complementary to military production.
Both the USA and Nazi Germany benefited massively from have a civilian industrial base that was complementary to military production.
Of course you could also argue that Germany wouldn't have had the early successes in war, (if they had even started it). Or at a third juncture, would have fared worse against USSR.
There's a book called Freedom's Forge that I'm a fan of, it makes the argument that the Auto Industry (And assembly lines, mechanization in general) were the single most important reason the Allies won WWII. In fact all the big auto manufacturers of the time retooled their assembly lines to build tanks and airplanes. It's conceivable that if we never mass produced cars, the US wouldn't have had the capability to win the war.
Miami is still above water.
Would you shut down the powerhouse of our economy -- travel, transportation, energy -- for something hypothetical that hasn't even happened and doesn't appear to be close to happening?
I'm pro-clean energy, but you can't do without fossil fuels. Not if you want society to keep climbing up and up and up.
Would you shut down the powerhouse of our economy -- travel, transportation, energy -- for something hypothetical that hasn't even happened and doesn't appear to be close to happening?
I'm pro-clean energy, but you can't do without fossil fuels. Not if you want society to keep climbing up and up and up.
Before you jump to policy making, you should get the implications right.
"The current rate of sea level rise at Pensacola Bay has accelerated rapidly since 2010."
"The difference in sea level rise over the last 100 years has been approximately 10 inches—but in the next 75-100 years, the increase in sea level rise could be close to 48 inches." https://blogs.ifas.ufl.edu/escambiaco/2023/04/12/weekly-what...
"The current rate of sea level rise at Pensacola Bay has accelerated rapidly since 2010."
"The difference in sea level rise over the last 100 years has been approximately 10 inches—but in the next 75-100 years, the increase in sea level rise could be close to 48 inches." https://blogs.ifas.ufl.edu/escambiaco/2023/04/12/weekly-what...
That sounds like a great way to lose an upcoming world war to some people who DGAF about pollution, climate, or other people in general.
[deleted]
Well it could be argued that it does, what about supersonic nuclear missiles?
But AI doesn't pose an existential threat to humanity, so we're all good.
I really can't grasp how people think that a system that doesn't have a need to preserve itself will somehow start thinking for itself.
AI is quite troublesome for privacy though. How much privacy humans need is a question we'll probably have answered the hard way.
AI is quite troublesome for privacy though. How much privacy humans need is a question we'll probably have answered the hard way.
Who said anything about thinking for itself?
A thing does not require intent or consciousness to be dangerous. How many chemists have blown themselves up because they didn't realize an experiment was dangerous? How many production systems have crashed because the developer didn't accurately predict what the code they wrote will do?
Alkali metals and C++ code do not require ill intent, but they will still obliterate your limbs / revenue if you build and use them wrong.
One of my more tangible hypotheses is a sort of runaway effect. Economic, geopolitical, and military competitive pressures will quickly push out anyone and anything that still relies on last era human-in-the-loop processes, the same way any organization that doesn't utilize artificial lighting, electricity, and instant communication will obviously be left far behind. You have to just trust that the machine running stock market transactions will do its math right.
But unlike transaction software failure modes, which quickly result in outright crashes or verifiably incorrect errors, failure modes of non-bayesian decision making software probably looks something like what happens when existing economic, geopolitical, and military decision-makers make decisions that are harmful, unethical, or otherwise undesirable for humanity. This time augmented with, if not superhuman intelligence, at least superhuman speed and superhuman knowledge breadth.
A thing does not require intent or consciousness to be dangerous. How many chemists have blown themselves up because they didn't realize an experiment was dangerous? How many production systems have crashed because the developer didn't accurately predict what the code they wrote will do?
Alkali metals and C++ code do not require ill intent, but they will still obliterate your limbs / revenue if you build and use them wrong.
One of my more tangible hypotheses is a sort of runaway effect. Economic, geopolitical, and military competitive pressures will quickly push out anyone and anything that still relies on last era human-in-the-loop processes, the same way any organization that doesn't utilize artificial lighting, electricity, and instant communication will obviously be left far behind. You have to just trust that the machine running stock market transactions will do its math right.
But unlike transaction software failure modes, which quickly result in outright crashes or verifiably incorrect errors, failure modes of non-bayesian decision making software probably looks something like what happens when existing economic, geopolitical, and military decision-makers make decisions that are harmful, unethical, or otherwise undesirable for humanity. This time augmented with, if not superhuman intelligence, at least superhuman speed and superhuman knowledge breadth.
Love that observation on C++. That's the reason I love C++. It's a language for those who need, nay crave, absolute raw performance. No training wheels. Short of assembly, it's just as close to the machine as you can get.
> No training wheels.
Very cute for hobby projects, a huge liability for commercial projects.
Use as many training wheels there as humanly possible, please.
Very cute for hobby projects, a huge liability for commercial projects.
Use as many training wheels there as humanly possible, please.
Sure. I use Java, Python and Javascript all the time. But when I need the performance, for demanding VR/ graphics, nothing comes close to combination of speed and expressive power of abstraction of C++.
Rust?
Does a prion have a need to preserve itself?
If you make enough varied AIs, some will have self-replicating behavior, just like if you make enough random proteins, some will self-replicate.
If you make enough varied AIs, some will have self-replicating behavior, just like if you make enough random proteins, some will self-replicate.
> I really can't grasp how people think that a system that doesn't have a need to preserve itself will somehow start thinking for itself.
Society exists because cooperation outperforms the alternatives. If you have human level AI at some point there is no benefit to cooperation and a major incentive to prevent anyone else gaining access to equal/better AI.
AI itself does not need to have any motivation - people in charge have plenty of incentives to eliminate the rest once they don't need them anymore.
Society exists because cooperation outperforms the alternatives. If you have human level AI at some point there is no benefit to cooperation and a major incentive to prevent anyone else gaining access to equal/better AI.
AI itself does not need to have any motivation - people in charge have plenty of incentives to eliminate the rest once they don't need them anymore.
Sure, in the prisoner's dilemma we could trust that all other parties will do the right thing, but that seems very unlikely.
that’s precisely the point: the technical geniuses lack creativity to predict how things can go wrong in a thousand of different ways.
What makes you think they're predicting the apocalypse correctly, then?
Another thing the technical geniuses tend to be good at is exploiting the power they suddenly obtain in their own interest, either directly or with regulations and collusion with those who hold actual hard power.
Evil AI owners seem to be much closer and far more material than an evil AI, and coincidentally it's something that is almost entirely lacking from the discourse, as public attention is too focused on sci-fi hypotheticals.
Another thing the technical geniuses tend to be good at is exploiting the power they suddenly obtain in their own interest, either directly or with regulations and collusion with those who hold actual hard power.
Evil AI owners seem to be much closer and far more material than an evil AI, and coincidentally it's something that is almost entirely lacking from the discourse, as public attention is too focused on sci-fi hypotheticals.
The bar is different - saying "there is no risk of apocalypse" requires you to be ~100% certain, because it you're saying "I'm 99% certain that there won't be a an apocalypse" then you're on the side of the AI-risk people, because a low-probability extinction event does justify action; the risk argument isn't that apocalypse is certain but rather that it is sufficiently plausible to take preventive measures.
I am only 99% certain that we won't be invaded by hostile aliens. Therefore we should take measures like building a giant space laser to prevent that apocalypse.
It is somewhat similar, but substantially different - we can make a solid argument that the likelihood of getting invaded by hostile aliens in the nearest century is far lower than 1%, and also if such an invasion does happen, then building a giant space laser won't make any difference at all.
The key difference between powerful alien invaders and us creating a powerful alien entity that we can't control is that the former either will or won't happen due to external circumstances, but the latter is something we would be doing to ourselves and can avoid if we choose to.
The key difference between powerful alien invaders and us creating a powerful alien entity that we can't control is that the former either will or won't happen due to external circumstances, but the latter is something we would be doing to ourselves and can avoid if we choose to.
Bullshit. You can't presume to quantify the probability of either event. You're just making things up. All of the arguments are built on a foundation of sand. This stuff falls in the realm of religion and philosophy, not hard science and math.
The issue is that the doomsday scenario is extremely vague. The actual mechanism of action of a hypothetical rogue AGI is usually handwaved away as "it will be self-improving, superhumanly persuasive, and far smarter than us, so it will somehow figure out how to do something, or convince us to do it". What exactly will happen? How exactly will it happen? Will the world do nothing until that moment? How do society, politics, military fit into that scenario? All that rationalist navel gazing I've seen so far is either hilariously unaware of the existence of the outside world or assumes it won't change in the process.
You can't fight what you can't even see, let alone not sure if it exists at all. You don't invent a pair of wings because 1900s' you thinks that "the scientists will invent an anti-aging cure in the next decade, and surely personal flight will be ubiquitous in 2000's". You don't design a plasma gun for your Mars landing just in case you land in a city between Martian canals and see an army of little green men there. The world doesn't work like that, by the time you reach the Mars surface the context will be wildly different. You get burned and put guardrails, maybe. Not the other way around. Nobody can see through higher order effects, no matter how smart they are. And as the threat becomes progressively more clear there will be more caution, if needed. Premature optimization yada yada.
What actually happens right now is everybody and my aunt seriously discussing the evil robots that will come and kill us. That's pure mass hysteria, caused by the scaremongering and the cult-like beliefs of very smart people with disproportional influence who can't contain their own conjectures and bullshit in the realm of science fiction.
On the other hand,the end goal of OpenAI is the major job replacement, according to their current charter. [1] "Broadly distributed"... will they distribute their utopia to North Korea? Not happening, isn't it? I think it's obvious that if the actual job replacement rate will ever get anywhere close to the levels of late 19th early 20th century industrialization, this will produce major societal shifts and struggles, wealth and power redistribution, and a lot of blood and wars. Because the dependence on your job is the only ephemeral influence you (as a worker) have on this world. And of course, the companies that control the AI will be gatekeepers, and they will be more than happy to close the open research and open source models, and pull the regulation ladder and lie in bed with politicians and military, like OpenAI already does for years, of course they realize that and their utopical self-contradicting "charter" is nothing more than marketing hogwash that they already changed and will change in the future.
This is far more realistic and will happen much earlier than the rogue AI science fiction, if happens at all. In fact it's slowly happening now, and it's not talked about nearly enough, because the attention is mostly misdirected onto the vague superhuman AI red herring.
[1] https://openai.com/charter
You can't fight what you can't even see, let alone not sure if it exists at all. You don't invent a pair of wings because 1900s' you thinks that "the scientists will invent an anti-aging cure in the next decade, and surely personal flight will be ubiquitous in 2000's". You don't design a plasma gun for your Mars landing just in case you land in a city between Martian canals and see an army of little green men there. The world doesn't work like that, by the time you reach the Mars surface the context will be wildly different. You get burned and put guardrails, maybe. Not the other way around. Nobody can see through higher order effects, no matter how smart they are. And as the threat becomes progressively more clear there will be more caution, if needed. Premature optimization yada yada.
What actually happens right now is everybody and my aunt seriously discussing the evil robots that will come and kill us. That's pure mass hysteria, caused by the scaremongering and the cult-like beliefs of very smart people with disproportional influence who can't contain their own conjectures and bullshit in the realm of science fiction.
On the other hand,the end goal of OpenAI is the major job replacement, according to their current charter. [1] "Broadly distributed"... will they distribute their utopia to North Korea? Not happening, isn't it? I think it's obvious that if the actual job replacement rate will ever get anywhere close to the levels of late 19th early 20th century industrialization, this will produce major societal shifts and struggles, wealth and power redistribution, and a lot of blood and wars. Because the dependence on your job is the only ephemeral influence you (as a worker) have on this world. And of course, the companies that control the AI will be gatekeepers, and they will be more than happy to close the open research and open source models, and pull the regulation ladder and lie in bed with politicians and military, like OpenAI already does for years, of course they realize that and their utopical self-contradicting "charter" is nothing more than marketing hogwash that they already changed and will change in the future.
This is far more realistic and will happen much earlier than the rogue AI science fiction, if happens at all. In fact it's slowly happening now, and it's not talked about nearly enough, because the attention is mostly misdirected onto the vague superhuman AI red herring.
[1] https://openai.com/charter
The actual mechanism of action is handwaved away because there are many options, we don't expect to ever have an exhaustive list and specifics of those are largely irrelevant with respect to preventing them, so IMHO it's not worth spending time and effort analyzing specific scenarios as long as we assume that there exists at least one plausible (even if unlikely) scenario. A hypothetical specific scenario of a rogue AI engineering and launching a deadly supervirus is effectively equivalent to a specific scenario resulting in a world consumed by 'grey goo' nanobots - you don't (can't) fix the former by implementing some resilience or detection for diseases, you don't (can't) fix the latter by doing extra research on nanorobotics, you approach both (and any others) by tackling the core issues of, for example, ensuring that you can control what goals artificial agents have even if they are self-modifying in some aspects.
Like, "What exactly will happen? How exactly will it happen?" is worth discussing if and only if one party seriously believes they can convince the other that none of the imaginable scenarios are even remotely plausible; and if we assume that there is at least one scenario where we can say "I'm 99% certain it won't happen and 1% it could" then that discussion is pretty much over, the existential risk is plausible (and the consequences of that are so much incomparably larger than e.g. major job displacement that it justifies attention even if it's many orders of magnitude less likely) and we should instead talk about how to prevent it.
I'm not making the argument that the existence of stronger-than-human general AI will result in a catastrophe, but I am asserting that the mere existence of a stronger-than-human general AI (without some controls we currently can't figure out how to make or even if they are possible) carries at least some plausible chance of existential risk - for the sake or argument, let's say at least 1%; and I am asserting that a 1% of existential risk is a totally absolutely unacceptably high risk that must not be allowed to happen, because it is far more important[1] than e.g 100% certainty of major job displacement and social unrest.
"Will the world do nothing until that moment?" I think that what we saw from the global reaction to things like start of Covid-19 or climate change is completely sufficient to assume that we can't rely on the world stopping a major-but-stoppable issue in a timely manner, so "surely the world will do something" is not a sufficiently convincing argument to discount the risk; I don't think you can plausibly deny that even for a clearly catastrophic problem there is at least a 10% chance that the world could still delay sufficient action until it's too late; and this means that it doesn't really matter what the exact likelihood of that is based on society, politics, military aspects, we should work with the assumption that the world actually might do nothing to prevent any specific scenario from unfolding, and we should de-risk it in other ways.
[1] Looking at other posts, perhaps this is where we'd disagree, and in that case it's probably the core of the discussion which also doesn't really depend on any details of specific scenarios.
Like, "What exactly will happen? How exactly will it happen?" is worth discussing if and only if one party seriously believes they can convince the other that none of the imaginable scenarios are even remotely plausible; and if we assume that there is at least one scenario where we can say "I'm 99% certain it won't happen and 1% it could" then that discussion is pretty much over, the existential risk is plausible (and the consequences of that are so much incomparably larger than e.g. major job displacement that it justifies attention even if it's many orders of magnitude less likely) and we should instead talk about how to prevent it.
I'm not making the argument that the existence of stronger-than-human general AI will result in a catastrophe, but I am asserting that the mere existence of a stronger-than-human general AI (without some controls we currently can't figure out how to make or even if they are possible) carries at least some plausible chance of existential risk - for the sake or argument, let's say at least 1%; and I am asserting that a 1% of existential risk is a totally absolutely unacceptably high risk that must not be allowed to happen, because it is far more important[1] than e.g 100% certainty of major job displacement and social unrest.
"Will the world do nothing until that moment?" I think that what we saw from the global reaction to things like start of Covid-19 or climate change is completely sufficient to assume that we can't rely on the world stopping a major-but-stoppable issue in a timely manner, so "surely the world will do something" is not a sufficiently convincing argument to discount the risk; I don't think you can plausibly deny that even for a clearly catastrophic problem there is at least a 10% chance that the world could still delay sufficient action until it's too late; and this means that it doesn't really matter what the exact likelihood of that is based on society, politics, military aspects, we should work with the assumption that the world actually might do nothing to prevent any specific scenario from unfolding, and we should de-risk it in other ways.
[1] Looking at other posts, perhaps this is where we'd disagree, and in that case it's probably the core of the discussion which also doesn't really depend on any details of specific scenarios.
Which also means they can't predict the apocalyptic scenarios.
Q.E.D.
Q.E.D.
but all the thoughts about it in 1925 would have been way off to how it actually turned out
I was thinking one reason Yann LeCun would make such a terrible analogy is because he knows something the rest of us don't.
The thing that he knows that (most of) the rest of us don’t is quite a lot about AI.
If you read what Yann writes you'll pretty quickly see that he's rather ignorant about AI. His opinion is probably worse on average than the typical technical generalist's
You'll have to be way more convincing than this if you want anyone to believe that about Yann haha.
This is ignorant.
He won a Turing award for his work on deep learning.
Lots of people reasonably disagree with him about the future of AI/ML, but he's the opposite of ignorant.
He won a Turing award for his work on deep learning.
Lots of people reasonably disagree with him about the future of AI/ML, but he's the opposite of ignorant.
That’s hilarious. I have read a few things he has written which suggests he’s definitely better than the average technical generalist. I haven’t read everything obviously but he has written quite a lot https://scholar.google.com/citations?user=WLN3QrAAAAAJ
What an absurd idea to say this about a leading AI researcher
And, what, you think he's working against humanity's interest in service of the secret AI overlords?
You're missing the point and context. Person above me says this analysis is quite telling, then points out the counterfactual historical hypothetical, which makes no sense. Yann thinks supersonic flight is not worthy of precautionary principle ethics in 1925? I'm saying the same thing--Yann's terrible, nonsense analogy is indeed poorly argued, but plausibly would make sense as a Freudian slip inconsistency of some sort. Ergo, "it is quite telling". As to what contents in his mind, or his motivations, I don't care to speculate.
The fact that you make insinuations about what I think is pretty aggressive and terrible similarly, this forum ought to have better manners than that when writing replies to complete strangers. Not everyone who has a different opinion is some crypto conspiracy theorist, and you are wrong to jump to such a suggestion.
The fact that you make insinuations about what I think is pretty aggressive and terrible similarly, this forum ought to have better manners than that when writing replies to complete strangers. Not everyone who has a different opinion is some crypto conspiracy theorist, and you are wrong to jump to such a suggestion.
His funny insinuation pushed you to write a nice long argumented explanation, it worked great
I have a personal belief that I can’t quite articulate in rigorous scientific terms that there is some information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence, and so if we can’t get to that, we’ll never be able to model it (notwithstanding “all models are wrong”).
The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power.
Might as well call AGI Nirvana.
The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power.
Might as well call AGI Nirvana.
As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot. The argument that we have to fully understand something to build it doesn't hold water for me. I have made plenty of things where understanding the depths of what I had made took far more time and effort than building the thing in the first place.
It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
> I have made plenty of things where understanding the depths of what I had made took far more time and effort than building the thing in the first place.
An example of this is electricity. The battery (1800), electric motor (1821) and telegraph (1832) were all invented before the discovery of the electron (1897).
An example of this is electricity. The battery (1800), electric motor (1821) and telegraph (1832) were all invented before the discovery of the electron (1897).
When people say we can't get to AGI, they're typically either in the camp that there's something intrinsically special about us that is incapable of being replicated, or various versions of the belief that it's too complex to be achieved within a timespan in which they would still recognize humanity.
The former can be seen as a religious belief, but the latter is simply saying that we do not understand anywhere near enough about intelligence to develop AGI.
You point to the unexpected leap in capability we've had recently, but fundamentally it isn't that different from what we have had for decades. The same fundamental unsolved limitations exist, such as of being unable to learn the way we do (eg if a child is writing a specific letter wrong, we correct that specific mistake, but that doesn't overwrite the knowledge of how to write the other letters).
As a lecture I watched a while ago had put it, we've gotten pretty good at making the icing of the AGI cake recently, but we still have no idea what the cake is supposed to even look/taste like, so we barely know what ingredients we might need to make it let alone the actual steps involved.
The former can be seen as a religious belief, but the latter is simply saying that we do not understand anywhere near enough about intelligence to develop AGI.
You point to the unexpected leap in capability we've had recently, but fundamentally it isn't that different from what we have had for decades. The same fundamental unsolved limitations exist, such as of being unable to learn the way we do (eg if a child is writing a specific letter wrong, we correct that specific mistake, but that doesn't overwrite the knowledge of how to write the other letters).
As a lecture I watched a while ago had put it, we've gotten pretty good at making the icing of the AGI cake recently, but we still have no idea what the cake is supposed to even look/taste like, so we barely know what ingredients we might need to make it let alone the actual steps involved.
>not understand anywhere near enough about intelligence to develop AGI.
LLM training is essentially training them to predict what happens next. When we combine this with multi-modal training (video/audio), they're being trained to predict what happens next in the world. As we feed them more and more data and make them bigger and bigger, they'll get better and better at this task. Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. So just by training them on this one label, predict what happens next, we'd expect them to eventually develop human-level intelligence if the loss keeps decreasing.
LLM training is essentially training them to predict what happens next. When we combine this with multi-modal training (video/audio), they're being trained to predict what happens next in the world. As we feed them more and more data and make them bigger and bigger, they'll get better and better at this task. Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. So just by training them on this one label, predict what happens next, we'd expect them to eventually develop human-level intelligence if the loss keeps decreasing.
If the loss keeps decreasing forever, then that conclusion seems to follow. But it's likely there will be plateaus and ceilings that current neural net architectures cannot surpass. We likely need more breakthroughs to achieve enough generalized reasoning, reliable predictions, alignment, and scalability within hardware constraints.
> Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour.
This doesn't follow. A just as likely (or even more likely) prediction is that they will fail to predict what happens next once it requires modeling humans. It won't get any more progress even if we increase training set size or model size - it will require an architecture change of some kind.
This doesn't follow. A just as likely (or even more likely) prediction is that they will fail to predict what happens next once it requires modeling humans. It won't get any more progress even if we increase training set size or model size - it will require an architecture change of some kind.
Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. We haven't even seen loss functions start to level out - the closest argument I've seen to that is people misinterpreting the SGD curve (an artificial tapering we apply to the training process) as a fundamental architecture limitation rather than just an attempt to use resources wisely on a given training run.
Basically, training of these models currently only costs a few million dollars, peanuts compared to the budgets of CERN or ITER, and we have zero evidence that we're approaching any sort of ceiling. If we keep scaling, maybe we see some heretofore unpredicted failure that breaks scaling laws like you're predicting. But maybe we don't, and the scaling laws (which have performed incredibly accurately so far) really do hold.
Basically, training of these models currently only costs a few million dollars, peanuts compared to the budgets of CERN or ITER, and we have zero evidence that we're approaching any sort of ceiling. If we keep scaling, maybe we see some heretofore unpredicted failure that breaks scaling laws like you're predicting. But maybe we don't, and the scaling laws (which have performed incredibly accurately so far) really do hold.
Having linear scaling up to a point is a far more reasonable guess, than a forever scaling function - as per Occam’s razor.
If I have a rubbery string and try lengthening it, it will also show a linear scaling, until it catastrophically fails.
If I have a rubbery string and try lengthening it, it will also show a linear scaling, until it catastrophically fails.
>Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical [Edit:- and not really a good hypothetical, because there's no theoretical backing that explains current progress but predicts a future discontinuity]. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works.
Also I have no idea why you think Occam's Razor supports "There will be a discontinuity in this currently accurate observed trend, near enough in the future to be relevant" when the trend is talking about the effects of an algorithm applied to data. By analogy, it's something like claiming Moore's Law won't hold because Occam's Razor suggests at some point there's a physical breakdown, except instead of saying it now you're saying it in 1978, & you're saying it about an algorithmic output rather than a physical thing with an obvious limit (atomic size). Once we see any evidence of all of such a discontinuity, we can adjust what we're doing based on that. Until then, if we want to know whether scaling laws hold we should scale the models.
Also I have no idea why you think Occam's Razor supports "There will be a discontinuity in this currently accurate observed trend, near enough in the future to be relevant" when the trend is talking about the effects of an algorithm applied to data. By analogy, it's something like claiming Moore's Law won't hold because Occam's Razor suggests at some point there's a physical breakdown, except instead of saying it now you're saying it in 1978, & you're saying it about an algorithmic output rather than a physical thing with an obvious limit (atomic size). Once we see any evidence of all of such a discontinuity, we can adjust what we're doing based on that. Until then, if we want to know whether scaling laws hold we should scale the models.
First of all, there is no theory at all which explains the current progress of LLMs. It's not even a quantifiable progress - it's clearly there, but no one can say with any true precision whether GPT2 to GPT3 to GPT4 is following linear or exponential or whatever else precise function of progress in capabilities. We can clearly see that they are improving, but we have no idea how to accurately quantify this space.
Secondly, we can see that there are obvious quantifiable physical increases in the requirements of training and inference for GPT4 compared to GPT3 and to GPT2. So, we can absolutely make a physical argument: at some point, you'll need too much energy to keep scaling up GPTs for it to be feasible, even if capability would keep improving with model size.
Thirdly, the GPTs require ever more data to train, and there is a limited amount of text data in the world, of which they are already using a significant percent. So, there is also an argument that we might run out of data to improve even before we run out of physical resources.
And making an argument that Moore's law will stop at some point due to physical constraints was just as correct in 1978 as it is in 2024. Predicting when that is is obviously hard - and the same is true for LLM scaling. We may be at the peek, or there may be 100 more generations before we even approach the limits I'm contemplating. But the limits are certainly there.
Secondly, we can see that there are obvious quantifiable physical increases in the requirements of training and inference for GPT4 compared to GPT3 and to GPT2. So, we can absolutely make a physical argument: at some point, you'll need too much energy to keep scaling up GPTs for it to be feasible, even if capability would keep improving with model size.
Thirdly, the GPTs require ever more data to train, and there is a limited amount of text data in the world, of which they are already using a significant percent. So, there is also an argument that we might run out of data to improve even before we run out of physical resources.
And making an argument that Moore's law will stop at some point due to physical constraints was just as correct in 1978 as it is in 2024. Predicting when that is is obviously hard - and the same is true for LLM scaling. We may be at the peek, or there may be 100 more generations before we even approach the limits I'm contemplating. But the limits are certainly there.
This conversation started with someone saying that despite LLMs having a theoretical basis for how they could emulate human intelligence, they won't actually reach that point because the existing scaling laws won't hold. I am pointing out that we don't know that at all, scaling laws are so far fantastically accurate, and there is no sign of undershooting existing those laws so far. Therefore, it's very premature to declare that LLMs will be unable to reach any given milestone because scaling laws will hit a discontinuity before we get there. Basically:
"There isn't even a theoretical basis for how LLMs could achieve human level intelligence"
"There is, a neural network trained to predict the next token isn't restricted in the algorithms it can implement internally to achieve that goal, and eventually for loss to be low enough you need to successfully emulate human minds. We have reason to believe loss will continue to go lower because we have very accurate scaling laws which predict that & haven't been wrong so far."
"But an LLM won't be able to reach that point because the scaling laws will break down."
"Maybe, there's no evidence for that yet but we should scale them up if we want to find out for sure."
You are here -> "But in the physical extreme LLMs will break down as an architecture."
To which my response is: Yes, of course this specific architecture will break down in the physical extreme. By the time something is building computronium to run the next AI I strongly doubt it will be running matrix multiplication, let alone LLMs specifically. There is no guarantee or reason to believe that we will hit those limits before we are able to achieve human level intelligence or more; the only reason to believe it will happen is an empirically observed trend which could totally break down, so I wouldn't bet the house on it happening, but it's wrong to just rule out the outcome.
"There isn't even a theoretical basis for how LLMs could achieve human level intelligence"
"There is, a neural network trained to predict the next token isn't restricted in the algorithms it can implement internally to achieve that goal, and eventually for loss to be low enough you need to successfully emulate human minds. We have reason to believe loss will continue to go lower because we have very accurate scaling laws which predict that & haven't been wrong so far."
"But an LLM won't be able to reach that point because the scaling laws will break down."
"Maybe, there's no evidence for that yet but we should scale them up if we want to find out for sure."
You are here -> "But in the physical extreme LLMs will break down as an architecture."
To which my response is: Yes, of course this specific architecture will break down in the physical extreme. By the time something is building computronium to run the next AI I strongly doubt it will be running matrix multiplication, let alone LLMs specifically. There is no guarantee or reason to believe that we will hit those limits before we are able to achieve human level intelligence or more; the only reason to believe it will happen is an empirically observed trend which could totally break down, so I wouldn't bet the house on it happening, but it's wrong to just rule out the outcome.
No, I am at the point where I'm saying the scaling "laws" will break down.
Especially since there are no scaling laws, it's an empirical observation based on essentially 3 examples - GPT2 to GPT3 to GPT4 (and their rough equivalents from Facebook and Google and a handful of others). This all happened in the span of 4 years or less. We don't even fully know how much larger GPT4 is compared to GPT3.
Not to mention, we already know OpenAI has spent more than a hundred millions of dollars training GPT-4, and they are quite probably selling it at a loss (each query may well cost them more in compute power than they charge for it). So, if GPT4 is ten times bigger than GPT3(.5) as sometimes reported, economics suggest they may already not be able to scale GPT5 to the same extent, and are definitely not going to be able to reach GPT7 (1000 times GPT4, or roughly 100 billion dollars). So, unless you believe compute power will go down in cost massively, or that we are just a factor 10-100 away from human-level intelligence, odds are good we won't see it from this line of AIs in the next 10-20 years.
Especially since there are no scaling laws, it's an empirical observation based on essentially 3 examples - GPT2 to GPT3 to GPT4 (and their rough equivalents from Facebook and Google and a handful of others). This all happened in the span of 4 years or less. We don't even fully know how much larger GPT4 is compared to GPT3.
Not to mention, we already know OpenAI has spent more than a hundred millions of dollars training GPT-4, and they are quite probably selling it at a loss (each query may well cost them more in compute power than they charge for it). So, if GPT4 is ten times bigger than GPT3(.5) as sometimes reported, economics suggest they may already not be able to scale GPT5 to the same extent, and are definitely not going to be able to reach GPT7 (1000 times GPT4, or roughly 100 billion dollars). So, unless you believe compute power will go down in cost massively, or that we are just a factor 10-100 away from human-level intelligence, odds are good we won't see it from this line of AIs in the next 10-20 years.
>Especially since there are no scaling laws, it's an empirical observation based on essentially 3 examples - GPT2 to GPT3 to GPT4 (and their rough equivalents from Facebook and Google and a handful of others). This all happened in the span of 4 years or less. We don't even fully know how much larger GPT4 is compared to GPT3.
I don't think you fully understand the scaling laws. They hold for those models, sure, but also other models at the same scale, smaller scale models, models in-between those examples, etc. They're an empirically derived law, for sure, but so far they've been very accurate for prediction, not just description. We know how much the loss will drop for a given amount of compute and data.
>Not to mention, we already know OpenAI has spent more than a hundred millions of dollars training GPT-4, and they are quite probably selling it at a loss (each query may well cost them more in compute power than they charge for it). So, if GPT4 is ten times bigger than GPT3(.5) as sometimes reported, economics suggest they may already not be able to scale GPT5 to the same extent, and are definitely not going to be able to reach GPT7 (1000 times GPT4, or roughly 100 billion dollars).
I think there's a decent chance even that much money could be spent if there's appropriate returns from GPT-5 & GPT-6. More importantly:
>So, unless you believe compute power will go down in cost massively,
Why on earth wouldn't you believe that? That is what has happened consistently for decades, and unlike general purpose computing which is slowing down, the fact that we need so much compute for one relatively specific operation means we can build ASICs for them (GPUs are already a chunk of the way there) and see very large speedups. As well, at these scales energy costs are a huge part of the training costs, and energy costs are continuing to drop dramatically because of solar. Compute is also a very flexible energy load, you can build it such that it can be paused at night if necessary, which could make it a very good fit for solar energy overproduction. Basically, there's a lot of low hanging fruit in terms of cost reductions for AI training available and a long trend of the cost of compute getting cheaper, it seems silly to just assert that your position (trends won't continue, there will be a discontinuity before we reach whatever point human level performance happens to be at) is the most likely outcome. All else being equal I would expect the existing trends to continue.
I don't think you fully understand the scaling laws. They hold for those models, sure, but also other models at the same scale, smaller scale models, models in-between those examples, etc. They're an empirically derived law, for sure, but so far they've been very accurate for prediction, not just description. We know how much the loss will drop for a given amount of compute and data.
>Not to mention, we already know OpenAI has spent more than a hundred millions of dollars training GPT-4, and they are quite probably selling it at a loss (each query may well cost them more in compute power than they charge for it). So, if GPT4 is ten times bigger than GPT3(.5) as sometimes reported, economics suggest they may already not be able to scale GPT5 to the same extent, and are definitely not going to be able to reach GPT7 (1000 times GPT4, or roughly 100 billion dollars).
I think there's a decent chance even that much money could be spent if there's appropriate returns from GPT-5 & GPT-6. More importantly:
>So, unless you believe compute power will go down in cost massively,
Why on earth wouldn't you believe that? That is what has happened consistently for decades, and unlike general purpose computing which is slowing down, the fact that we need so much compute for one relatively specific operation means we can build ASICs for them (GPUs are already a chunk of the way there) and see very large speedups. As well, at these scales energy costs are a huge part of the training costs, and energy costs are continuing to drop dramatically because of solar. Compute is also a very flexible energy load, you can build it such that it can be paused at night if necessary, which could make it a very good fit for solar energy overproduction. Basically, there's a lot of low hanging fruit in terms of cost reductions for AI training available and a long trend of the cost of compute getting cheaper, it seems silly to just assert that your position (trends won't continue, there will be a discontinuity before we reach whatever point human level performance happens to be at) is the most likely outcome. All else being equal I would expect the existing trends to continue.
That doesn't really follow at all.
It may result in something that is even better at faking the appearance of intelligence, but LLMs and similar stuff fundamentally lacks various features that make them not capable of becoming human level intelligences.
It may result in something that is even better at faking the appearance of intelligence, but LLMs and similar stuff fundamentally lacks various features that make them not capable of becoming human level intelligences.
Humans are pretty good at faking intelligence too! It's only when you question them carefully and you yourself are an expert in the field that you can detect that they're confabulating.
Conversely, I've never seen anybody produce anything but the next word coming out of their mouth. Therefore people are glorified stochastic parrots.
Conversely, I've never seen anybody produce anything but the next word coming out of their mouth. Therefore people are glorified stochastic parrots.
You're conflating knowledge of specific subject areas with generalized intelligence, including the ability to learn and metacognition.
What do you mean by learning and metacognition? An LLM can tell you whether or not something is in their context window. Humans regularly come up with "innovative" things that they actually saw from somewhere else and just happen to forget the source.
Intelligence is like magic - every time a machine can do one thing on the list, it stops being intelligence, it's just engineering. Maybe intelligence really is just a big bag of tricks?
Intelligence is like magic - every time a machine can do one thing on the list, it stops being intelligence, it's just engineering. Maybe intelligence really is just a big bag of tricks?
> fundamentally lacks
We don't actually know that yet. It may very well fundamentally have all that is necessary for intelligence but other pieces are missing.
Frontier neuroscience and functional processes for how our brains work has more in common with the fundamentals of LLMs than not, vis-a-vis minimization of prediction errors.
We don't actually know that yet. It may very well fundamentally have all that is necessary for intelligence but other pieces are missing.
Frontier neuroscience and functional processes for how our brains work has more in common with the fundamentals of LLMs than not, vis-a-vis minimization of prediction errors.
I don’t think “fundamentally having something” is actually meaningful here - as it’s enough to show it is Turing complete for it to have fundamentally everything. Nonetheless, it’s probably not excel that will become the next AGI.
My two cents on the topic. Currently, we are incapable of modeling causality effectively or at all (depends on your definition and to who you ask). LLMs seem to have it, but it's a mockery of causal reasoning. They use data created by us and so they can predict the next word following some of the inherent patterns we humans use which might look like they can reason to some extent.
In this regard something like alphago/zero (in the way I understand them at least )are better than modern LLMs albeit in a very narrow field.
In this regard something like alphago/zero (in the way I understand them at least )are better than modern LLMs albeit in a very narrow field.
The thing is that while the two camps exist among those who say we can't get to AGI, there is no real equivalent of the first camp among those who think it is possible. I'm not counting the people who say it has already (as good as) been achieved, as this claim is of the sort that should have been satisfactorily empirically verified before it is made.
When someone says that skepticism in human-level AGI probably lies in religious beliefs, it tells me that the person likely believes that they have an idea what science and philosophy are saying on the matter. But I strongly suspect that they never actually looked.
Among the small but growing number of modern idealists (the philosophy that consciousness precedes matter), a great deal are actually computer scientists who originally set out to work on AGI from a materialistic frame of reference, but bumped into the conscious machine problem. Once they more closely explored what the cognitive fields working to find answers to this problem are saying, they realized that most scientists approaching consciousness as an emergent property of matter will probably never get anywhere with this.
Why is "the hard problem of consciousness" inextricable from human-level AGI? Because intelligence is only one component of our mentation. There is a back and forth between processing and experience that make up the whole thing. A machine can have or even exceed human processing power, but it will not experience the simplest of colors. Its approach to "creating" can be random hit-or-misses, or formulaically from existing input. But without subjective experience it cannot be as intently creative as an artist trying to come up with a new genre, or as funny as a comedian that relies on the shared subtleties of feelings and emotions (conscious experience) to make original work that other humans can resonate with. It cannot have the intuition of a gifted chef who suspects that two seldom combined spices might be an unexpected but resounding success in her new dessert.
Among the small but growing number of modern idealists (the philosophy that consciousness precedes matter), a great deal are actually computer scientists who originally set out to work on AGI from a materialistic frame of reference, but bumped into the conscious machine problem. Once they more closely explored what the cognitive fields working to find answers to this problem are saying, they realized that most scientists approaching consciousness as an emergent property of matter will probably never get anywhere with this.
Why is "the hard problem of consciousness" inextricable from human-level AGI? Because intelligence is only one component of our mentation. There is a back and forth between processing and experience that make up the whole thing. A machine can have or even exceed human processing power, but it will not experience the simplest of colors. Its approach to "creating" can be random hit-or-misses, or formulaically from existing input. But without subjective experience it cannot be as intently creative as an artist trying to come up with a new genre, or as funny as a comedian that relies on the shared subtleties of feelings and emotions (conscious experience) to make original work that other humans can resonate with. It cannot have the intuition of a gifted chef who suspects that two seldom combined spices might be an unexpected but resounding success in her new dessert.
These are just opinions, there is no reason to imagine the phenomenology of qualia is so tightly coupled with creative work and/or making good conjectures. There is a lot of smoke…
What is 'subjective experience' in your view?
Multi-modal models are clearly the future, if we provide models with a robot body - sensors, motors, the ability for locomotion - would that be subjective experience?
Multi-modal models are clearly the future, if we provide models with a robot body - sensors, motors, the ability for locomotion - would that be subjective experience?
Does gold have an 'ineffable' quality that hydrogen does not? Do cells, after 1bn years of evolution have a quality that gold does not? Does reality work one way, and not another? Are not all things unequal?
A "machine" is what, exactly? Do we take it to be an abstraction? Or is it an electrical field oscillating over silicon? Either way, you're in trouble. Abstractions have no physical properties, and electrified sand seems hardly to possess any interesting properties.
The ability for animals to adapt to their environments, by growing into them, by establishing plastic causal connections in their very bodies, grown by their environemtns... able to almost instantly move from protein expression in 1trn 1-bn-yr cellular supercomputers in each of our bodies to macro sensory-motor representation --- and back again
Is this ineffable?
Or is this extremely effable. Is rather, not the superstitious view that "everything is anything" ?
All extant, knowing, studied intelligent systems have organic properties; and radically so. Insofar as this is "ineffable" you should take that up with the animal kingdom.
I find the contrary supersitious, magical, religious, ineffable... that mere abstract patterns in arbiatrily chosen aspects of our bodies are necessary and sufficient conditions for anything at all. This would be the only case in all science. The only physical property instantiated by mere arrangement at any level. Upload our consciousness? Make it out of wood why not!
Nonesense. When I am hungry, I dream of food, when I dream of food I plan to get some, and I am angry without it. This will not be made out of sand.
A "machine" is what, exactly? Do we take it to be an abstraction? Or is it an electrical field oscillating over silicon? Either way, you're in trouble. Abstractions have no physical properties, and electrified sand seems hardly to possess any interesting properties.
The ability for animals to adapt to their environments, by growing into them, by establishing plastic causal connections in their very bodies, grown by their environemtns... able to almost instantly move from protein expression in 1trn 1-bn-yr cellular supercomputers in each of our bodies to macro sensory-motor representation --- and back again
Is this ineffable?
Or is this extremely effable. Is rather, not the superstitious view that "everything is anything" ?
All extant, knowing, studied intelligent systems have organic properties; and radically so. Insofar as this is "ineffable" you should take that up with the animal kingdom.
I find the contrary supersitious, magical, religious, ineffable... that mere abstract patterns in arbiatrily chosen aspects of our bodies are necessary and sufficient conditions for anything at all. This would be the only case in all science. The only physical property instantiated by mere arrangement at any level. Upload our consciousness? Make it out of wood why not!
Nonesense. When I am hungry, I dream of food, when I dream of food I plan to get some, and I am angry without it. This will not be made out of sand.
Given that you need some pretty complex elements to be able to make life in the first place (not to mention complex molecules made out of those elements) I'd say yes, there is definitely an 'ineffable' quality that life has that Hydrogen does not (to short circuit your chain of reasoning), but intelligence isn't necessarily made of a particular kind of matter and that's where the analogy breaks down.
The soul, likewise, is immaterial.
Who now trades in superstition?
Everything is corporeal.
Who now trades in superstition?
Everything is corporeal.
Downvoted for saying reality has physical properties? Where else but hackernews? Sorry to disappoint, but all the algorithms of CS, which do anything at all, have devices in them. ie., their critical steps are impure, and cannot be specified mathematically.
There is nothing abstract in the world, everything is concrete. Discrete mathematics is not some divine realm which is where the Mind really occurs, and the Soul is really kept.
No graph traversal algorithm cares, nor will ever. Science is the study of reality, not computer "science". There are no computational "properties". All interesting properties are of objects extended in space, and in time.
The world is physical, as described by physics; not abstract, as describe by mathematics. And so, not computable.
There is nothing abstract in the world, everything is concrete. Discrete mathematics is not some divine realm which is where the Mind really occurs, and the Soul is really kept.
No graph traversal algorithm cares, nor will ever. Science is the study of reality, not computer "science". There are no computational "properties". All interesting properties are of objects extended in space, and in time.
The world is physical, as described by physics; not abstract, as describe by mathematics. And so, not computable.
The proposition of artificial intelligence, whether at human-level or not, is that a suitably-programmed computing device (as opposed to an immaterial algorithm) could manifest the properties which we have chosen to call "intelligence" when we observe them in biological agents, and similarly for consciousness.
Particularly with your comment "no graph traversal algorithm cares, nor will ever", you appear to be trying to impute that the concept of artificial intelligence is predicated on a category error. If so, then this is a fairly common argument, but is itself predicated on a category error.
Particularly with your comment "no graph traversal algorithm cares, nor will ever", you appear to be trying to impute that the concept of artificial intelligence is predicated on a category error. If so, then this is a fairly common argument, but is itself predicated on a category error.
My mind has abstractions in it. Are you saying that mental formations are not real, not in the universe?
>Where else but hackernews?
I have seen this sentiment on every single platform I have participated in. Where else? Everywhere that humans go.
Find a more appropriate (or at least original) lamentation.
I have seen this sentiment on every single platform I have participated in. Where else? Everywhere that humans go.
Find a more appropriate (or at least original) lamentation.
What about platonic forms
The laws of physics are corporeal?
We don't have any proof to suggest that "intelligence" grants a fundamental understanding of everything though. It can help us understand "things", do neat stuff, and build models to explain the working and structure of phenomena; However would having an IQ of 5000 mean you are now "God" or gain and understanding of everything including the mystery of existence itself?
At some point, it seems important to stop trying to understand the world through ideas and concepts and just experience it.
I really think there are more religious like beliefs on the pro AGI camp than the other way around, think about what AI promises to deliver by "visionaries" like Hinton and Kurzweil:
* Immortality.
* Resurrections.
* Nirvana / Heaven / Utopia.
For all we know, we create an AGI who becomes amused with fucking itself, or develops new games it finds interesting and spends all eternity playing them. Humans have a belief that we're going to create "god" as we think of "him". All seeing, all knowing, all powerful. It does get a bit ridiculous at times.
I sympathize a lot wit the parents view personally.
At some point, it seems important to stop trying to understand the world through ideas and concepts and just experience it.
I really think there are more religious like beliefs on the pro AGI camp than the other way around, think about what AI promises to deliver by "visionaries" like Hinton and Kurzweil:
* Immortality.
* Resurrections.
* Nirvana / Heaven / Utopia.
For all we know, we create an AGI who becomes amused with fucking itself, or develops new games it finds interesting and spends all eternity playing them. Humans have a belief that we're going to create "god" as we think of "him". All seeing, all knowing, all powerful. It does get a bit ridiculous at times.
I sympathize a lot wit the parents view personally.
Good point, especially that intelligence doesn’t quite scale. Many (most?) of our problems are hard to solve due to lack of data/experimentation over lack of intelligence. And more intelligence won’t help with the infinite scalability of complexity.
Some folks I suppose take a religious fervor, but for my I look at how a child learns -- through the observation of cause and effect -- and know the current architectures don't support how we have husbanded animals and reared children for eons.
The inherent capacity for true self-directed learning may well be there, but it isn't hit yet in what we see.
GPTs are still neat.
The inherent capacity for true self-directed learning may well be there, but it isn't hit yet in what we see.
GPTs are still neat.
Why do you think computers would most efficiently learn the way we do? They don’t do anything else (eg arithmetic as an obvious example) anything like humans normally do.
A deterministic algorithm that adjusts floating point errors for varying chipsets is more complex, agreed. So current implementations are not terribly simple for arithmetic. That highlights my point, I believe, that the only examples of actual learning, where one can make true inferences about the world without the need to collect the entirety of the internet to build transformers, we have to date are wetware, not software.
I'm not sure that's true. There is reinforcement learning and its branches. I think that's part of how they do ai for chess, go, star craft etc...
You don’t have to believe AGI is impossible to believe we have no route to get there. The burden of proof on “we have a way to do this” falls heavily on the person proposing it. And being able to do something that wasn’t expected doesn’t imply the ability to do a specific other thing. It’s that belief that’s I’d say is perhaps more cultish than religious.
Who’s saying we definitely have a way to do this? Very few. Nobody’s trying to prove that we definitely do right now.
But the “presumption” that we can’t is silly given our advancements
But the “presumption” that we can’t is silly given our advancements
This logic doesn’t work with anything else. If I say we’re going to fly to Jupiter you’re going to ask me for specifics not just presume that I know what I’m talking about. Or any of the other infinite things I can claim to be able to do. Why is this one different?
Hmm. When the first basic black powder rockets were invented, would it have been unreasonable for someone to think that they could one day reach the moon? Of course there would be many uncertainties -- including what the moon even was. No one could say what technology would be required. But clearly there was a major paradigm shift that suddenly made it within the realm of consideration.
I imagine it might have been unreasonable to suppose that, yes. The fact that the future is unpredictable doesn’t mean that all claims about the future are a priori equally valid.
In the future where AGI takes over the world next year I might look silly for arguing with you about it, but that’s a risk I accept based on the fact that I don’t think that’s going to happen.
In the future where AGI takes over the world next year I might look silly for arguing with you about it, but that’s a risk I accept based on the fact that I don’t think that’s going to happen.
In that scenario, you are the one claiming we will "never" fly to jupiter.
Just to close the metaphorical loop here, the current AI debate is something like:
"There's a danger we'll be able to fly to Jupiter and bring home a Jupiterian life-form that will destroy humanity. Therefore we should stop all space exploration."
The doomers take the equation with some wildly improbable step that we can't currently explain or justify, multiply the outcome of that step by "infinitely bad", and conclude that the whole thing must be dangerous. But to amp up the rhetoric, they always skip over the "wildly improbable" part.
"There's a danger we'll be able to fly to Jupiter and bring home a Jupiterian life-form that will destroy humanity. Therefore we should stop all space exploration."
The doomers take the equation with some wildly improbable step that we can't currently explain or justify, multiply the outcome of that step by "infinitely bad", and conclude that the whole thing must be dangerous. But to amp up the rhetoric, they always skip over the "wildly improbable" part.
I guess the disagreement you have with doomers is the probablility you assign to us being able to create human level AI. For the record most "doomers" don't pull a pascal's wager here. Standard EA doctrine is that if we stagnate technologically at our current level for too long (read centuries) then we will inevitably be wiped out. So they want technological advance, they just want caution.
I don't know what "most" do, but I see that form of argument all the time -- particularly from the highest profile doomers.
Personally, I think this line of argument is driven by the hype cycle more than reality. Use chatGPT or midjourney or whatever for a while, and it's pretty easy to see that we're dramatically overweighting theories of AGI risk, and dramatically underweighting stuff like "disemploying the bottom 80% of the intelligence bell curve" with technologies that automate away lots of formerly white-collar labor.
If I had to put my money on an "existential risk" attributable to AI, it would be economic strife.
Personally, I think this line of argument is driven by the hype cycle more than reality. Use chatGPT or midjourney or whatever for a while, and it's pretty easy to see that we're dramatically overweighting theories of AGI risk, and dramatically underweighting stuff like "disemploying the bottom 80% of the intelligence bell curve" with technologies that automate away lots of formerly white-collar labor.
If I had to put my money on an "existential risk" attributable to AI, it would be economic strife.
It's worth saying we still haven't seen a single model trained post-hype cycle from openAI. GPT-4 was summer 2022 stuff.
I’m saying that there are very many things that I can’t prove are impossible that I don’t worry about happening. I’ve learned to live with that uncertainty.
We absolutely have a way to get there. A sufficiently large neural network can approximate any function (the universal approximation theorem), which has been mathematically proven. LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it. Eventually we'll have LLMs trained on visual and audio input too (and be trained to predict what happens next), so they can receive an arbitrarily large amount of training samples. If the models keep growing and being trained on more and more data, we'd expect them to become at least as intelligent (good at modelling the world) as humans.
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function. So, there is no absolute certainty that they must be able to approximate human cognition, since there is no reason to believe cognition is continuous.
Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.
Also, there is no proof and no reason to believe that any current architecture and, more importantly, that the currently known training algorithms can achieve anything close to cognition. After all, animals and humans seem to learn much much much faster (much smaller training sets) than any algorithm we have so far.
You have to be a lot clearer about what you mean by continuous here. An LLM technically does not produce a continuous function on the reals because its inputs are floating point numbers with finite precision. Any function on discrete inputs like this has an extension to the reals which is continuous, just imagine joining all the discrete points up with lines.
So then your claim wouldn't be about the limits of LLMs themselves, but on the limits of systems that do not take continuous inputs. The question then is do you think that humans take in continuous input?? Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.
So then your claim wouldn't be about the limits of LLMs themselves, but on the limits of systems that do not take continuous inputs. The question then is do you think that humans take in continuous input?? Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.
It is quantum which isn’t exactly discrete. You receive one photon or two, not 1.5, sure, but the energy of that photon, its frequency is a real. At least I am not aware of a quantum mechanics over Q. I think the math would be hard because you loose all the convergence properties. Maybe there is some Hilbert space over computable reals, Google is not finding it but I no longer find that to be indicative of anything.
The whitepaper operates on mathematical numbers, though. They didn’t calculate it with floating points of n-bits.
Given a fixed level of precision for input and output, and a function on this discrete space, we can construct a continuous extension of this function to the reals. Now using the paper we know that there is a neural network with continuous weights that approximates this continuous extension to arbitrary precision. If we imagine rounding the continuous input and output to the fixed precision specified, then because of continuity of the neural network, and the fact there is a finite number of weights, we can choose a tolerance by which we can change each of the weights such that the output does not change by more than the precision of the output value. Thus we can pick a level of precision for the weights where both the weights and inputs and outputs are discrete.
I mean continuous in the calculus sense, as in it doesn't have discontinuities, not as in continuous VS discrete.
> Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.
This is a misunderstanding of quantum mechanics. Only certain specific quantities come in discrete quanta (spin, charge, certain energy levels, etc). Other physical quantities are very much continuous - notably time and space. In fact, much of the mathematics of QM is not discretizable, it won't work if you try to make time or space discrete.
> Given that physics seems to be discrete at the low level, this suggests to me they don't, but I don't know enough to be sure.
This is a misunderstanding of quantum mechanics. Only certain specific quantities come in discrete quanta (spin, charge, certain energy levels, etc). Other physical quantities are very much continuous - notably time and space. In fact, much of the mathematics of QM is not discretizable, it won't work if you try to make time or space discrete.
But continuous depends on the topology of the space you are working over. If the topology is discrete any function is continuous. I've found some papers that argue that simply from an error corrections standpoint the signals in the brain need to be discrete.
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function
Incorrect. That's what nonlinear activation functions are for.
Researchers wasted many potentially-fruitful years because Minsky and other luminaries -- people who occupied the same position of authority that LeCun occupies today -- made a huge deal about how the original perceptron and subsequent multilayer variants "couldn't learn XOR." That was almost literally how they put it. At the first sign of difficulty they abandoned the approach that was, and remains, the most promising. We have to be careful to learn from that mistake.
Incorrect. That's what nonlinear activation functions are for.
Researchers wasted many potentially-fruitful years because Minsky and other luminaries -- people who occupied the same position of authority that LeCun occupies today -- made a huge deal about how the original perceptron and subsequent multilayer variants "couldn't learn XOR." That was almost literally how they put it. At the first sign of difficulty they abandoned the approach that was, and remains, the most promising. We have to be careful to learn from that mistake.
There is a leap from "predicting the next word" to "modelling the world" that you are sort of glossing over here.
The idea that LLM's are actually "reasoning" rather than just performing a probability function seems a little bit...pseudo-religious. Like we've created a new life form.
The idea that LLM's are actually "reasoning" rather than just performing a probability function seems a little bit...pseudo-religious. Like we've created a new life form.
> LLMs are trained to predict the next word in a sequence; doing this to a high enough level of accuracy requires building an internal model of the world; the ability to reason about it.
I don't see any evidence that this is true or proven in any way.
Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.
I don't see any evidence that this is true or proven in any way.
Also, the ability to brute force something doesn't mean that brute forcing it is easy or even feasible. You can apply the same logic to "calculating a private key from a public key" that you are to human intelligence. Sure, a large enough neural network can do either, but that doesn't mean that building them is actually realistic.
It has been proven in much simpler toy models that LLMs just trained on token prediction can and do build "world" models[0]. That's not the same as evidence that they must build a world model, but it's proof that at least sometimes they do so even when just being trained on prediction.
[0]https://thegradient.pub/othello/
[0]https://thegradient.pub/othello/
Predictive models need not at all reflect what actually happens. Sometimes embracing less accurate models is actually need to enhance understanding. Epicycles in the geocentric model could be made very accurate as far as observations in the sky are concerned, but it wasn't a good world model.
That whitepaper operates with mathematical, infinite precision numbers.
It’s just a categorical error to read too much into it - it’s not really a too interesting property to be able to get closer to something forever, without some sane growth function.
It’s just a categorical error to read too much into it - it’s not really a too interesting property to be able to get closer to something forever, without some sane growth function.
>> The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power.
> As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot.
That's missing the point: the argument isn't that it's theoretically impossible to build an AGI, just that humans are incapable of it.
Here's another point for the belief in AGI being a religion: it's basically a sect of the larger religion of technological progress, which (likely falsely) assumes that technology will continue to "advance" at approximately current rates until we live in a world out of a sci-fi paperback. A lot of people believe that, but frequently resort to a motte-and-bailey fallacy when challenged.
> It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
It's worth noting that "the current optimism" is not without precedent. IIRC, there was a big boom in AI in the 70s/80s. SHRDLU was pretty impressive. But then then the promising ideas were found to be dead ends and there was a long winter.
> As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot.
That's missing the point: the argument isn't that it's theoretically impossible to build an AGI, just that humans are incapable of it.
Here's another point for the belief in AGI being a religion: it's basically a sect of the larger religion of technological progress, which (likely falsely) assumes that technology will continue to "advance" at approximately current rates until we live in a world out of a sci-fi paperback. A lot of people believe that, but frequently resort to a motte-and-bailey fallacy when challenged.
> It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
It's worth noting that "the current optimism" is not without precedent. IIRC, there was a big boom in AI in the 70s/80s. SHRDLU was pretty impressive. But then then the promising ideas were found to be dead ends and there was a long winter.
I think that intelligence has a “bootstrap” level - that human achieved - after which it can practically do most stuff, the bottleneck won’t be intelligence itself. E.g. our mathematics itself has fundamental limitations on what it can prove - I don’t think some machine/different intelligent entity could get further away. At most they are getting their faster.
in many ways, we already live in a world out of a sci-fi paperback.
I find it hard to fault people for thinking that we'll continue to progress at a fast pace, or at least thinking that we're a long way from a major plateau.
I find it hard to fault people for thinking that we'll continue to progress at a fast pace, or at least thinking that we're a long way from a major plateau.
We can not create the laws of physics, and we are fine with that aspect of immaterial reality.
The idea that there’s some ‘ information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence’ sounds more religious to me.
If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
I dont have a strong opinion about it, but when I read parents post I immediately thought of Gödel s works and how there are limits of what you can know within a system (and in Mathematics no less). Thinking that something similar exists in intelligence does not seem so far fetched. As a hypothesis, of course.
Right. I think a charitable reading of the post is something like, "it probably takes a mind superior to a human's in order to willfully develop a human-rivaling AGI".
There's a lot of ground to cover between AI as increasingly elaborate magic 8-ball toys and a real human-rivaling AGI. That is, one capable of observing an environment, identifying goals and problems, planning, acting, and reacting. In these much longer (stateful) cognitive chains, there are more opportunities for pathological failure modes and less opportunity for a toy user to charitably excuse the misbehavior.
This is not some kind of dualist metaphysical argument about the possibility of AGI in the abstract. Merely a doubt that we can blindly scale up the complexity of a synthetic mind to meet or surpass our own. Consider that we still can't even begin to understand our own minds in enough detail to reliably predict, repair, or augment them.
I am outside this field and so may have too much of a layman's perspective. But it seems to me that contemporary AGI believers conflate training and evolution. That nature did it in eons doesn't argue that we can do it in practical product development cycles, unless we can simulate these evolutionary processes to follow a similar search in a compressed time scale.
As a crude analogy, I think todays LLM products are a bit like horoscope generators. Clever arrangements of words that attract a charitable or gullible reader. But AGI use cases are more like wanting a life partner who will understand and willingly assist ones efforts.
There's a lot of ground to cover between AI as increasingly elaborate magic 8-ball toys and a real human-rivaling AGI. That is, one capable of observing an environment, identifying goals and problems, planning, acting, and reacting. In these much longer (stateful) cognitive chains, there are more opportunities for pathological failure modes and less opportunity for a toy user to charitably excuse the misbehavior.
This is not some kind of dualist metaphysical argument about the possibility of AGI in the abstract. Merely a doubt that we can blindly scale up the complexity of a synthetic mind to meet or surpass our own. Consider that we still can't even begin to understand our own minds in enough detail to reliably predict, repair, or augment them.
I am outside this field and so may have too much of a layman's perspective. But it seems to me that contemporary AGI believers conflate training and evolution. That nature did it in eons doesn't argue that we can do it in practical product development cycles, unless we can simulate these evolutionary processes to follow a similar search in a compressed time scale.
As a crude analogy, I think todays LLM products are a bit like horoscope generators. Clever arrangements of words that attract a charitable or gullible reader. But AGI use cases are more like wanting a life partner who will understand and willingly assist ones efforts.
The reason it seems far fetched is we have one example of a mechanistic, physics based machine that we know can perform the computations necessary to produce intelligence.
I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI.
Since we do have an existing machine that can produce AGI, we would have to suppose it does something:
- outside of physics to achieve its results
- performs some operation that is impossible for us to understand or replicate
Both of those seem… unlikely to me.
I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI.
Since we do have an existing machine that can produce AGI, we would have to suppose it does something:
- outside of physics to achieve its results
- performs some operation that is impossible for us to understand or replicate
Both of those seem… unlikely to me.
It seems like an article of faith to believe that the universe is a computer in the first place.
Without that assumption, we don't even know that intelligence is computation.
Without that assumption, we don't even know that intelligence is computation.
The article of faith is not that the universe is a computer. The "article of faith" is that the universe follows physical rules.
I also don't think it's fair to call it an article of faith, since we have strong evidence to show that _to date_ the universe has followed predictable physical rules. That could, obviously, change at any time, but it seems at least a reasonable prior to assume that the physical rules that we've studied in the past will continue to operate into the future. "The sun will rise tomorrow" is, I _guess_ an article of faith, but I think it's more fair to say it's a reasonable and well founded prediction based on a well studied model of the solar system and the physical laws we've observed the universe follow in the past.
So, my beliefs are:
- The universe is mechanistic, and follows physical rules
- Human beings are also mechanistic, and follow the same physical rules as the universe
- Human beings are intelligent
- Human beings are at minimum turing complete computers (since you could give me a roll of paper, a set of op-codes, and I could perform calcuations)
So, it seems to me a reasonable starting assumption that intelligence is a result of the mechanistic universe. Do we have any evidence for something outside the physical, mechanistic universe impacting human cognition? I'm not aware of any.
But we have lots of evidence of the physical, mechanistic universe impacting human cognition and intelligence.
I'm willing to acknowledge that we don't know that intelligence is Turing Computable, but I'd argue that intelligence is likely to be a mechanical process that's compatible with the physical rules of the universe. Can we be certain of that? No, we can not be 100% certain. But it seems a much more reasonable and evidenced hypothesis than something which asserts there is a metaphysical process that produces intelligence, but which we have no evidence for.
So, no, I don't think it's reasonable to say that assuming a "mechanistic universe" is an article of faith. I think it's a reasonable belief, based on the evidence that we have. What would be an article of faith is asserting that it could _only_ be a mechanistic universe, and refusing to accept any evidence to the contrary. I have not done that.
I also don't think it's fair to call it an article of faith, since we have strong evidence to show that _to date_ the universe has followed predictable physical rules. That could, obviously, change at any time, but it seems at least a reasonable prior to assume that the physical rules that we've studied in the past will continue to operate into the future. "The sun will rise tomorrow" is, I _guess_ an article of faith, but I think it's more fair to say it's a reasonable and well founded prediction based on a well studied model of the solar system and the physical laws we've observed the universe follow in the past.
So, my beliefs are:
- The universe is mechanistic, and follows physical rules
- Human beings are also mechanistic, and follow the same physical rules as the universe
- Human beings are intelligent
- Human beings are at minimum turing complete computers (since you could give me a roll of paper, a set of op-codes, and I could perform calcuations)
So, it seems to me a reasonable starting assumption that intelligence is a result of the mechanistic universe. Do we have any evidence for something outside the physical, mechanistic universe impacting human cognition? I'm not aware of any.
But we have lots of evidence of the physical, mechanistic universe impacting human cognition and intelligence.
I'm willing to acknowledge that we don't know that intelligence is Turing Computable, but I'd argue that intelligence is likely to be a mechanical process that's compatible with the physical rules of the universe. Can we be certain of that? No, we can not be 100% certain. But it seems a much more reasonable and evidenced hypothesis than something which asserts there is a metaphysical process that produces intelligence, but which we have no evidence for.
So, no, I don't think it's reasonable to say that assuming a "mechanistic universe" is an article of faith. I think it's a reasonable belief, based on the evidence that we have. What would be an article of faith is asserting that it could _only_ be a mechanistic universe, and refusing to accept any evidence to the contrary. I have not done that.
> So, it seems to me a reasonable starting assumption that intelligence is a result of the mechanistic universe.
> Do we have any evidence for something outside the physical, mechanistic universe impacting human cognition?
Problem is, even granted the above, that's not enough of an argument that AGI is only a matter of mechanism.
Take the analogy with pain. We've discovered the mechanism for pain. That it's e.g. some stimulus applied to skin which sets of nerve signals that register in the brain. To then say that the experience of pain is the same as the physical mechanism we've discovered still misses a key step:
what does the experience of pain inhere to, and is that the same subject as that for the mechanism of pain?
Comparing AGI and human intelligence has the same problem. We don't know exactly what is intelligent in either case, let alone whether the two are comparable. So it's not a question of intelligence per se, but of that which is intelligent. Maybe AGI, the way we are thinking and talking about it, is unavoidably tangled with having to grapple with [self-]consciousness.
Problem is, even granted the above, that's not enough of an argument that AGI is only a matter of mechanism.
Take the analogy with pain. We've discovered the mechanism for pain. That it's e.g. some stimulus applied to skin which sets of nerve signals that register in the brain. To then say that the experience of pain is the same as the physical mechanism we've discovered still misses a key step:
what does the experience of pain inhere to, and is that the same subject as that for the mechanism of pain?
Comparing AGI and human intelligence has the same problem. We don't know exactly what is intelligent in either case, let alone whether the two are comparable. So it's not a question of intelligence per se, but of that which is intelligent. Maybe AGI, the way we are thinking and talking about it, is unavoidably tangled with having to grapple with [self-]consciousness.
Do you think the self-consciousness that we possess is the result of physical processes going on in our brain and body, or do you think something outside of physics is required?
If I made a careful atom by atom (or quark/lepton/boson) copy of every atom in your brain and body in a different location, would that new copy be intelligent?
If our intelligence is purely a result of physical processes, then why do you suppose it would be impossible for us (or more advanced future beings), to construct a machine which can also follow those processes?
If you think your intelligence is not reducible to purely physical processes, then… what is this meta-physical thing that is required for intelligence? Does it interact with anything other than intelligence, or is it only involved with intelligence? Is there any reason or evidence we should assume this meta-physical intelligence?
If I made a careful atom by atom (or quark/lepton/boson) copy of every atom in your brain and body in a different location, would that new copy be intelligent?
If our intelligence is purely a result of physical processes, then why do you suppose it would be impossible for us (or more advanced future beings), to construct a machine which can also follow those processes?
If you think your intelligence is not reducible to purely physical processes, then… what is this meta-physical thing that is required for intelligence? Does it interact with anything other than intelligence, or is it only involved with intelligence? Is there any reason or evidence we should assume this meta-physical intelligence?
> ..., or do you think something outside of physics is required?
I am most sympathetic to the argument that self-consciousness is an emergent phenomenon that is not easily modeled, but is nevertheless derived from physics. In the context of my previous comment, it is this emergent phenomenon that is deemed "intelligent".
> If I made a careful atom by atom (or quark/lepton/boson) copy of every atom in your brain and body in a different location, would that new copy be intelligent?
Yes.
> If our intelligence is purely a result of physical processes, then why do you suppose it would be impossible for us (or more advanced future beings), to construct a machine which can also follow those processes?
I think it is possible, and not too far off to boot. We might get (maybe we already have gotten) the intelligence without the particular emergent phenomenon of self-consciousness that is of the kind humans possess, at least not at first. Some of us will call that AGI, some of us won't. Because we are still working out the terminology and taxonomy. Discussions like these in HN help.
Maybe we'll get the emergent phenomenon soon thereafter. If it is in fact intractable to model (and therefore to train for directly), it may happen unpredictably. Exciting times.
I am most sympathetic to the argument that self-consciousness is an emergent phenomenon that is not easily modeled, but is nevertheless derived from physics. In the context of my previous comment, it is this emergent phenomenon that is deemed "intelligent".
> If I made a careful atom by atom (or quark/lepton/boson) copy of every atom in your brain and body in a different location, would that new copy be intelligent?
Yes.
> If our intelligence is purely a result of physical processes, then why do you suppose it would be impossible for us (or more advanced future beings), to construct a machine which can also follow those processes?
I think it is possible, and not too far off to boot. We might get (maybe we already have gotten) the intelligence without the particular emergent phenomenon of self-consciousness that is of the kind humans possess, at least not at first. Some of us will call that AGI, some of us won't. Because we are still working out the terminology and taxonomy. Discussions like these in HN help.
Maybe we'll get the emergent phenomenon soon thereafter. If it is in fact intractable to model (and therefore to train for directly), it may happen unpredictably. Exciting times.
I think a mechanistic universe is not necessarily a computation, so purely physical intelligence is also not necessarily a computation. But I'm not prepared at the moment to go dig up the alternatives to "computation" and "metaphysical process", or to figure out how a deterministic (or even stochastic) process could be uncomputable.
> so purely physical intelligence is also not necessarily a computation.
We're going to get into semantics pretty quickly, but I would argue that at least purely physical intelligence is a procedure that is capable of being run on a machine that exists in our universe.
I would also argue that, such a machine is _at least_ turing complete (since I would think a general intelligence should be capable of emulating a turing machine in the same way that humans are able to do so). I would happily accept that it's possible that being turing complete is a necessary, *but not sufficient* condition for intelligence. That is, I could see a world where intelligence requires some form of hyper-turing machine, that is able to solve some non-turing computable problems.
However, I would argue that even if intelligence does require non-turing computable functions, there exists *a* machine which can perform those procedures (that is, we exist). Thus, in this hypothetical universe where intelligence is non-turing computable, we would then have an existence proof for a hyper-turing machine which can compute non-turing computable results. In this universe, then, I'd argue that anything that can be produced by this "hyper-turing machine", _is_ "physically computable", even if not "turing-computable".
Ultimately, if we accept the premise of a mechanistic universe, I think humans are an existence proof of a machine capable of producing intelligence. Then, I think it follows that it *cannot* be physically impossible to create a machine capable of producing intelligence. Whether we call that machine a "computer", though, I don't have a strong preference.
We're going to get into semantics pretty quickly, but I would argue that at least purely physical intelligence is a procedure that is capable of being run on a machine that exists in our universe.
I would also argue that, such a machine is _at least_ turing complete (since I would think a general intelligence should be capable of emulating a turing machine in the same way that humans are able to do so). I would happily accept that it's possible that being turing complete is a necessary, *but not sufficient* condition for intelligence. That is, I could see a world where intelligence requires some form of hyper-turing machine, that is able to solve some non-turing computable problems.
However, I would argue that even if intelligence does require non-turing computable functions, there exists *a* machine which can perform those procedures (that is, we exist). Thus, in this hypothetical universe where intelligence is non-turing computable, we would then have an existence proof for a hyper-turing machine which can compute non-turing computable results. In this universe, then, I'd argue that anything that can be produced by this "hyper-turing machine", _is_ "physically computable", even if not "turing-computable".
Ultimately, if we accept the premise of a mechanistic universe, I think humans are an existence proof of a machine capable of producing intelligence. Then, I think it follows that it *cannot* be physically impossible to create a machine capable of producing intelligence. Whether we call that machine a "computer", though, I don't have a strong preference.
Neither humans nor any physical device is Turing complete; only bounded computations can we perform. What do you mean by mechanistic? Mathematical? Deterministic? Non-dual with no unmeasurable causes?
> What do you mean by mechanistic?
So, I’ve only used it in a “I understand what it means and what I mean by it” manner in the past, and I haven’t rigorously defined it for myself.
I’ll do the best giving an off-the-cuff definition, but I’m sure it will have some holes and would be improved with some time spent thinking about it, or reading to crib a definition from someone else who has already put in that time.
I think my definition of a mechanistic universe would be:
A universe that transitions from one state to an adjacent state through a consistent set of rules.
From your options:
- mathematical? I’m think yes, but potentially only if “mathetics” is defined broadly.
- Deterministic? I don’t think this is a requirement for a mechanistic universe. My bet would be on a deterministic universe (through an Everettian interpretation lens), but I think something can be mechanistic while being stochastic, so long as the rules determine the probabilities. That is, I would consider a computer with a true source of randomness to still be mechanistic.
> Non-dual with no unmeasurable causes?
Non-dual, yes. That is the primary meaning for me. “With no unmeasurable causes” is a phrasing that makes me slightly uneasy. I could imagine a situation where, if the universe is stochastic you could end up in a situation where you cannot precisely identify the specific cause for a specific event. But, I think as the spirit of this question is intended, yes.
So, I’ve only used it in a “I understand what it means and what I mean by it” manner in the past, and I haven’t rigorously defined it for myself.
I’ll do the best giving an off-the-cuff definition, but I’m sure it will have some holes and would be improved with some time spent thinking about it, or reading to crib a definition from someone else who has already put in that time.
I think my definition of a mechanistic universe would be:
A universe that transitions from one state to an adjacent state through a consistent set of rules.
From your options:
- mathematical? I’m think yes, but potentially only if “mathetics” is defined broadly.
- Deterministic? I don’t think this is a requirement for a mechanistic universe. My bet would be on a deterministic universe (through an Everettian interpretation lens), but I think something can be mechanistic while being stochastic, so long as the rules determine the probabilities. That is, I would consider a computer with a true source of randomness to still be mechanistic.
> Non-dual with no unmeasurable causes?
Non-dual, yes. That is the primary meaning for me. “With no unmeasurable causes” is a phrasing that makes me slightly uneasy. I could imagine a situation where, if the universe is stochastic you could end up in a situation where you cannot precisely identify the specific cause for a specific event. But, I think as the spirit of this question is intended, yes.
Thanks, that all makes sense. Do you follow Sean Carroll's takes on many worlds? A very consistent philosophy of reality, and some interesting math starting to come out with space-time itself being an emergent feature of Hilbert space evolution of the wave equation (where it is constructed essentially out of varying degrees of coupling between different states). Even a paper where this sort of coupling-derived space might obey the Einstein equations.
I listen to his podcast, and I’m definitely partial to the many worlds interpretation, so I end up taking in a lot of his explanations of it.
The work he’s doing in that area is super interesting, and sounds promising (of course, I’m getting primarily Carroll’s take on it when I hear about it, so it makes sense that it sounds promising!), but the math itself is above my head.
I am super interested to see if it ends up moving the ball forward on resolving the GR/QM conflicts.
The work he’s doing in that area is super interesting, and sounds promising (of course, I’m getting primarily Carroll’s take on it when I hear about it, so it makes sense that it sounds promising!), but the math itself is above my head.
I am super interested to see if it ends up moving the ball forward on resolving the GR/QM conflicts.
I personally don't think it will but I guess it could, it's as good a model to start from as any. That's kind of the problem though, it's not actually better than the modern version of the Copenhagen interpretation unless you think wave function collapse is a physical event that's caused by (something in) your brain. If you think that, Copenhagen's obviously not correct, as your brain can't cause anything that it itself wasn't caused to do, because free will does not exist. If you think of wave function collapse as not a physical event and not caused by anything (except by the physics of the universe following the original state of the universe, if you don't like the idea of a block universe in configuration space), it's totally fine and is equivalent to Multiple Worlds. Don't think of the Multiple Worlds as things that physically exist either though.
Can we stop with this stuff? It is a 100% useless distinction, Turing machines can only ever use a fixed amount of memory, so we can run any Turing machine for as long as we want to, we can just pause it, “change the tape” to a bigger one, and continue.
Well, it's not useless mathematically. The Turing machine cannot run on all inputs with a bounded tape size. The quickly growing functions show pretty strongly that no physically incarnateable device can compute easy to specify functions on inputs less than 1000. "Turing machine" or Turing equivalence are pretty precisely defined mathematical concepts, so using them in different senses is a bit perverse; in my mind, more perverse than using literally to mean figuratively. Also, how does one change the tape in ones brain? I'd love a pensieve but those also are not physical, at least not yet.
The thing I'd really like is a Turing machine where the nth transition took 1/2^n seconds, so we could run them to infinity in just two seconds.
The thing I'd really like is a Turing machine where the nth transition took 1/2^n seconds, so we could run them to infinity in just two seconds.
That's wrong, see See Gödel, Escher, Bach [1979, pp. 452–456] If you get rid of the infinities and approximate (including cutting recursions short), you can (approximately) compute a whole lot. Not everything, but everything your brain can, and more.
Sorry, what exactly is wrong? Are there no limits to what you can know within a system?
It's wrong that an AI system is more limited than a human brain. Obviously there's computability and Busy Beaver stuff, as well as other earlier practical limits to computation, but that's not a limit to "general intelligence" if you mean the level of a human brain. Human brains approximates and hardly want to recurse at all, and there's no reason an AI can't do the same thing.
If I understand correctly you state that it is wrong to say a computer is more limited than a human brain.
IMO that is far from clear. In some aspects (multiplying matrixes) it might be more powerful. In others (power consumption) the human brain wins by a wide gap.
IMO that is far from clear. In some aspects (multiplying matrixes) it might be more powerful. In others (power consumption) the human brain wins by a wide gap.
No GP's point is that Gödel's theorem has nothinhg to do with this debate because the things that Gödel says are impossible aren't the things GPT-9 would need to do to reach a human-level intelligence.
The point that brains are more efficient than computers is valid, but that doesn't tell us much about eg whether we'll be able to make a computer do everything a human brain can do within 50 years.
The point that brains are more efficient than computers is valid, but that doesn't tell us much about eg whether we'll be able to make a computer do everything a human brain can do within 50 years.
And yet we can use mathematics to know that that limit exists. It’s a powerful thing, mathematics.
Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
It is an analogy to mention that is not a far fetched idea that there are limits.
As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
> If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.
Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.
That’s… not a refutation of my point.
AI/flight analogies are tired, but the OPs argument amounts to the equivalent of, before the Wright Brothers, proclaiming ‘there’s an inherent inability for humans to ever conceive of a way to engineer heavier than air flight’.
It’s a ‘man was never mean to fly, therefore heavier than air flight is impossible’ argument.
AI/flight analogies are tired, but the OPs argument amounts to the equivalent of, before the Wright Brothers, proclaiming ‘there’s an inherent inability for humans to ever conceive of a way to engineer heavier than air flight’.
It’s a ‘man was never mean to fly, therefore heavier than air flight is impossible’ argument.
But wasn't it through experimentation that we developed that understanding? And in the earliest phases of that experimentation, that's when our understanding was at its weakest, right? You almost seem to be implying that we had the understanding before ever attempting the experiments.
This is a very mechanic-ist way of looking at things.
Yes, which seems to be the way the universe works. Duality is a religious belief.
As far as I know we haven't figured it out how the universe works, hence the unified theory debacle and all that.
Sure, and we have pretty good evidence of the mechanistic operation of the universe from quantum mechanics up, and we haven’t found any evidence of anything outside of that.
So, to date, a mechanistic view of the universe seems like one that is the most supported be evidence at the moment.
Have we ruled out meta-physics or some kind of duality? No, of course we haven’t. As long as there’s unknown physics, it’s a possibility.
But I think if you’re reasoning based on the evidence we’ve been able to collect to date, then you should absolutely favor a mechanistic universe to a dual universe.
So, to date, a mechanistic view of the universe seems like one that is the most supported be evidence at the moment.
Have we ruled out meta-physics or some kind of duality? No, of course we haven’t. As long as there’s unknown physics, it’s a possibility.
But I think if you’re reasoning based on the evidence we’ve been able to collect to date, then you should absolutely favor a mechanistic universe to a dual universe.
Reminds me of the old line about how, if the brain were so simple we could understand it, we'd be so stupid that we couldn't.
Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent.
But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now.
It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is.
To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence."
In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.
Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent.
But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now.
It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is.
To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence."
In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.
We have to understand it to build it? That seems absurd. Did evolution understand it? I'll grant you that maybe we should understand it before building it, but it's absolutely not a requirement.
The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more.
We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.
The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more.
We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.
The scariest part to me is the realization that not only will we probably crack AGI in my lifetime, if we keep throwing resources at it, it will almost certainly have a richer more sophisticated conscious experience than myself. What the hell are we supposed to do when we've created an intelligence that is better at being "human" than humans?
Will you have the choice?
More seriously, we create objects for us. We don’t have to cater for the feelings of a bigger AI. And maybe that bigger AI will have time to speak with those who need a human presence (to everyone using TV as a background noise, or watching Youtube out of solitude).
More seriously, we create objects for us. We don’t have to cater for the feelings of a bigger AI. And maybe that bigger AI will have time to speak with those who need a human presence (to everyone using TV as a background noise, or watching Youtube out of solitude).
I think the ramifications of what a true AI would bring are impossible to predict. Especially if it can improve itself, it can quickly give rise to many sci-fi stories.
Not an endorsement, but here's an argument that AGI is computationally intractable (edit: at least via ML).
https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inev...
"AI is impossible" is a tough argument, given the existence of non-A I.
"Machine learning isn't the route to AI" is something you could more reasonably argue, but that's a drastically narrower claim.
"Machine learning isn't the route to AI" is something you could more reasonably argue, but that's a drastically narrower claim.
Intractable and impossible are very different claims.
NP Hardness is a statement about the asymptotic difficulty of solving a problem at ever larger scales. It says ‘if you have a way to solve this problem at size n, that way will scale worse than polynomially when you try to apply it to a problem at size 2n’.
Which might place limits on the practicable scale of how big an n your approach gets to work for. But if your approach works practically for a big enough n to make AGI then your approach works - NP Hardness doesn’t matter.
And since we know that finite mass lumps of finite numbers of gray cells are capable of GI, we have a reasonable expectation that there is some n for which AGI might be possible.
NP Hardness is a statement about the asymptotic difficulty of solving a problem at ever larger scales. It says ‘if you have a way to solve this problem at size n, that way will scale worse than polynomially when you try to apply it to a problem at size 2n’.
Which might place limits on the practicable scale of how big an n your approach gets to work for. But if your approach works practically for a big enough n to make AGI then your approach works - NP Hardness doesn’t matter.
And since we know that finite mass lumps of finite numbers of gray cells are capable of GI, we have a reasonable expectation that there is some n for which AGI might be possible.
I usually take AGI to mean what it says literally - simply an AI with very general training. I think that description already applies to the latest crop of LLM models.
Taking it to mean some nebulous sentient entity with a sense of self, I doubt we'll see in our lifetimes. We don't even understand how our own "souls" work.
Taking it to mean some nebulous sentient entity with a sense of self, I doubt we'll see in our lifetimes. We don't even understand how our own "souls" work.
> I usually take AGI to mean what it says literally - simply an AI with very general training.
That's clever but it doesn't help to start redefining terms that are the basis for a discussion with others who use that term in a different and more generally accepted way. General doesn't mean 'generally trained' it specifically means that it does not require training for a particular task in order to figure it out. That implies that it may not require training at all and that if it is trained the training isn't necessarily general but that the AI can extract useful patterns from training on completely different subjects and apply them to the problem at hand.
This is subtly but crucially importantly different from 'AI with very general training'.
That's clever but it doesn't help to start redefining terms that are the basis for a discussion with others who use that term in a different and more generally accepted way. General doesn't mean 'generally trained' it specifically means that it does not require training for a particular task in order to figure it out. That implies that it may not require training at all and that if it is trained the training isn't necessarily general but that the AI can extract useful patterns from training on completely different subjects and apply them to the problem at hand.
This is subtly but crucially importantly different from 'AI with very general training'.
> This is subtly but crucially importantly different from 'AI with very general training'.
I am not being as precise on the Internet as I would be if this was a paper perhaps, and for that I apologize :-)
However, just reading the top comment chain on this story, and other discussions elsewhere, I think there is a lot of cross-talk where AGI is being confused for sapient AI. I don't get the sense that there is a definition as generally accepted as we would like.
I am not being as precise on the Internet as I would be if this was a paper perhaps, and for that I apologize :-)
However, just reading the top comment chain on this story, and other discussions elsewhere, I think there is a lot of cross-talk where AGI is being confused for sapient AI. I don't get the sense that there is a definition as generally accepted as we would like.
That's fair, but it's a big step to using a term in a way that it clearly isn't meant to be used (or at least, that's how I perceive it). The fact that we continuously seem to redefine what "AI" means implies that we also redefine what "AGI" means so it's not as if you are alone in this. By my 1980's standards we have AGI but the degree of "I" is still below human level, by my 2024 standards we do not have AGI just yet but what we do have is already so powerful that I'm not even sure to what extent it matters. Weaponizing what's there today already has the potential to destabilize our societies, having an even more powerful version of this (or just a faster one of the current crop) will change the world in ways that are hard to foresee or predict.
ChatGPT can do task with 0 shot prompting. You can ask it to do something and it will attempt to do it without you having to train it for the task you want to do. It's not like it has a list of 100 built in tasks it can do, you can make up whatever task you want.
> I think that description already applies to the latest crop of LLM models
I think you're using a completely different definition from everyone else.
I think you're using a completely different definition from everyone else.
I think a barrier to understanding our own intelligence meaning no AGI is false.
It's like saying not understanding exactly how birds fly would mean we can never build machines that will fly faster than them.
We'll probably make machines that think generally and are smarter than us long before we really understand our own minds.
It's like saying not understanding exactly how birds fly would mean we can never build machines that will fly faster than them.
We'll probably make machines that think generally and are smarter than us long before we really understand our own minds.
Yes, If you look at the power efficiency of the brain relative to LLMs, you can see nature has done something special, and I don’t think we should be so quick to call it game over. It might be that LLMs are just some atom or building block of a much larger and more complex thing.
Yes, this is huge. Of all the places to shim in my ¢2, as the topic could go on forever.
I remember talking to a robotics specialist and his take on efficiency was that despite the obvious speed and strength that machines have over us, you need to factor in just how little power is consumed when we move our extremities compared to what is needed for actuators and motors of even the best bots.
Now this was years ago and things could have taken a huge turn, but I think given the recent statements by Altman about the looming power crush drives it home, that even if we do manage something truly monumental, it'll take a fusion reaction to pull off.
Maybe Berkshire Hathaway Energy is the smart play
I mean, I wouldn’t bet on that either. We built the “computing” empire on silicon, but we might be able to very significantly improve on this with a single discovery/invention. All the computational stuff we built/figured out will move out without trouble.
Yes, that might be too. You just never know.
You got challenged, as if we have all the answers there are. Thing is, path to AGI and beyond inevitably leads to those hard questions from the beginning which we have zero idea about how to even define, or at least some proxy definition exists, such as what is and what isn't, what is consciousness, interplay between layers of psyche and how it translates to intelligence and everything else, biological ship of theseus, etc.. it becomes philosophical quite fast and there are more questions there than answers.
These arguments are so bewildering to me. Intelligence is very easy to define and has nothing to do with consciousness. It is simply the ability to model an external reality and then take action according to inputs.
After combining enough specialized modules together with a management layer, we will essentially have AGI. Of course it won't be human, because human intelligence is tightly coupled to human perception.
After combining enough specialized modules together with a management layer, we will essentially have AGI. Of course it won't be human, because human intelligence is tightly coupled to human perception.
These arguments are so bewildering to me.
My friend, that's because you stopped reading once you had your framework set in and decided to pontificate. Let me highlight the point for you, it might be easier to grasp:
path to AGI and beyond inevitably leads to those hard questions... ... how it translates to intelligence and everything else
there is nothing simply about it, there is nothing guaranteeing anything about it and there is everything about it once we start pursuing things AGI - AND BEYOND. Notice the keywords.
My friend, that's because you stopped reading once you had your framework set in and decided to pontificate. Let me highlight the point for you, it might be easier to grasp:
path to AGI and beyond inevitably leads to those hard questions... ... how it translates to intelligence and everything else
there is nothing simply about it, there is nothing guaranteeing anything about it and there is everything about it once we start pursuing things AGI - AND BEYOND. Notice the keywords.
Honestly, yours sounds more like a religious claim that the other.
It seems less a religious claim to assert that the human brain is mechanistic, and simply operates using physics to perform complex operations than to assert that there is something “unknowable”, or “outside of physics” that would prevent us from being able to build a similar machine.
If there were 0 examples in the universe, I think your point would be a good one. But, given there is 1 example, I think it stands to reason that there could be more.
I could accept an argument that one might expect it to be hideously complicated, and not something we’ll be able to accomplish for a long time.
But the claim that there’s an information theoretic reason that would absolutely prevent it, would seem to make the claim that there is something metaphysical about the operation of the brain, which seems like a quasi religious claim to me.
It seems less a religious claim to assert that the human brain is mechanistic, and simply operates using physics to perform complex operations than to assert that there is something “unknowable”, or “outside of physics” that would prevent us from being able to build a similar machine.
If there were 0 examples in the universe, I think your point would be a good one. But, given there is 1 example, I think it stands to reason that there could be more.
I could accept an argument that one might expect it to be hideously complicated, and not something we’ll be able to accomplish for a long time.
But the claim that there’s an information theoretic reason that would absolutely prevent it, would seem to make the claim that there is something metaphysical about the operation of the brain, which seems like a quasi religious claim to me.
Almost everything is "outside of physics", this is mainstream physics. All of physics, as formulated, is uncomputable. And what is computable is a radical approximation.
Physics cannot formulate descriptions of climates, or even, many cases of turbulence, or 4 bodies in a gravitational field.
Leaving "physics" is trivial, it occurs whenever you take any of the current toy models and add one layer of reality to them.
The whole edifice of formal science is a little like children's block toys.
Physics cannot formulate descriptions of climates, or even, many cases of turbulence, or 4 bodies in a gravitational field.
Leaving "physics" is trivial, it occurs whenever you take any of the current toy models and add one layer of reality to them.
The whole edifice of formal science is a little like children's block toys.
> Almost everything is "outside of physics",
No, it's not.
Many things are outside of "known" physics, sure. But I didn't say "outside of known physics". I said "outside of physics".
When I say "outside of physics" I mean "meta-physical", as in, processes that are not part of physical interactions of this world. Things that, if you somehow had Maxwell's demon tracking every single quark, lepton, and boson, even then you still wouldn't be able to account for.
It's a discussion of what is _theoretically knowable_, not a discussion of what we currently do know.
No, it's not.
Many things are outside of "known" physics, sure. But I didn't say "outside of known physics". I said "outside of physics".
When I say "outside of physics" I mean "meta-physical", as in, processes that are not part of physical interactions of this world. Things that, if you somehow had Maxwell's demon tracking every single quark, lepton, and boson, even then you still wouldn't be able to account for.
It's a discussion of what is _theoretically knowable_, not a discussion of what we currently do know.
I do not think most things are "theoretically knowable", since knowledge is comprised of finite concepts conjoined in finite ways; and most of reality is uncomputable.
You cannot know the state of the climate, even in principle.
This is what the commenter was alluding to when they said,
> there is some information-theoretic barrier for us to understand
So the commenter can both maintain there are theoretical barriers to the possibility of all kinds of knowledge, which is outside of "knowable physics" without having any sort of dualistic view that this unknownable stuff is immaterial.
So to be "outside of physics" is not coextensive with being "immaterial".
You might have meant that, but it is this very conflation which has to be undone to understand OP's point
You cannot know the state of the climate, even in principle.
This is what the commenter was alluding to when they said,
> there is some information-theoretic barrier for us to understand
So the commenter can both maintain there are theoretical barriers to the possibility of all kinds of knowledge, which is outside of "knowable physics" without having any sort of dualistic view that this unknownable stuff is immaterial.
So to be "outside of physics" is not coextensive with being "immaterial".
You might have meant that, but it is this very conflation which has to be undone to understand OP's point
But is general uncomputability the OP's argument? They were just saying they had a gut feeling there is some "information-theoretic barrier" preventing AGI construction. Yet we can understand a lot about Earth's weather, and even do things to affect the weather, despite climate systems being uncomputable, strictly speaking.
I don't think that's what they meant by "outside of physics".
I think one of the biggest barriers is that the human brain isn't running on binary code, and attempting to replicate it on our current technology model is doomed to encounter emulation issues.
That’s not an “information theoretic barrier” argument. That’s not an impossibility argument.
That’s a “hideously complicated, and not something we’ll be able to accomplish for a long time” argument, which I happily concede may be the case.
I see a very very large gulf between:
“Impossible in all scenarios” and “totally impractical with current technology”.
I was refuting only the former argument.
That’s a “hideously complicated, and not something we’ll be able to accomplish for a long time” argument, which I happily concede may be the case.
I see a very very large gulf between:
“Impossible in all scenarios” and “totally impractical with current technology”.
I was refuting only the former argument.
If I may hazard a guess: The term and concept you’re grasping for is “emergence.”
It’s an area I’m both interested in but completely unfamiliar with. I wish there were a crash-course for dummies, because I feel it’s a deep topic that we’ve only scratched the surface of; and that philosophy seems to be the most rich resource means there’s low hanging fruit to discover.
https://plato.stanford.edu/entries/properties-emergent/
https://link.springer.com/chapter/10.1007/978-981-15-9297-3_...
It’s an area I’m both interested in but completely unfamiliar with. I wish there were a crash-course for dummies, because I feel it’s a deep topic that we’ve only scratched the surface of; and that philosophy seems to be the most rich resource means there’s low hanging fruit to discover.
https://plato.stanford.edu/entries/properties-emergent/
https://link.springer.com/chapter/10.1007/978-981-15-9297-3_...
Would you also have that belief about how much weight a human can lift ?
Just because a human can lift 500lb, we will never build a machine that can lift more than that ?
Why is it so crazy to think that we, the tool building animals that we are, could also build a machine that things better than us, just how we built a crane ?
Just because a human can lift 500lb, we will never build a machine that can lift more than that ?
Why is it so crazy to think that we, the tool building animals that we are, could also build a machine that things better than us, just how we built a crane ?
This seems highly hand wavy to me. We have biological proof that intelligence can be created, mostly through the neocortex, which in and of itself is actually a fairly simple structure repeated at great scale. Why wouldn't we be able to build this eventually, once we understand it a bit more?
90 T neurons, quadrillion interconnections, lots of glia cells with synapse altering action, plus chemical signals diffusing around and going thru the bloodstream. In principle, it is comprehensible in some sense tho not by one human, but the architecture is quite different than LLMs and we don’t know what parts of the architecture are needed for sentience. Given the gap between what genetics starts going and what we end up with by age 25, it is likely that certain routine experiences of childhood and of human social groups are also requirements for sentience.
Well, we don’t even understand what “understand” means, so there’s really little basis for anything.
On the other hand, we also don’t understand (and didn’t predict) how LLMs work so well, so understanding isn’t a precondition for bringing about.
On the other hand, we also don’t understand (and didn’t predict) how LLMs work so well, so understanding isn’t a precondition for bringing about.
Do we have to understand something to recreate it? I think that’s already a fundamentally false-y assumption.
For example, many branches of science operate on a more “discover” over “invent” mindset, quite successfully.
For example, many branches of science operate on a more “discover” over “invent” mindset, quite successfully.
One just has to design and build the house of the room they're trapped in.
That's a pretty lazy take. You could look out the windows and see the reflections of the house. You could build some kind of RF gun to probe the house. You could use sub or supersonic echolocation. You could build a camera and stick it out the window etc.
[deleted]
[deleted]
Biologically achieving agi seems easier, just clone human brain,or more evil organization can just use human captive and use there brain. As depicted in psycho pass anime.
Probably not going to pan out well https://arxiv.org/html/2401.10020v1
>we’ll never be able to model it
That's completely false. Neural networks have been mathematically proven to be universal approximators (https://www.deep-mind.org/2023/03/26/the-universal-approxima... ), i.e. a sufficiently large neural networks can approximate any given mathematical function to an arbitrary level of precision. Given any programs can be modelled as a mathematical function, neural networks can hence approximate any arbitrary program. Unless we assume something supernatural, human intelligence is just a program.
That's completely false. Neural networks have been mathematically proven to be universal approximators (https://www.deep-mind.org/2023/03/26/the-universal-approxima... ), i.e. a sufficiently large neural networks can approximate any given mathematical function to an arbitrary level of precision. Given any programs can be modelled as a mathematical function, neural networks can hence approximate any arbitrary program. Unless we assume something supernatural, human intelligence is just a program.
Not really...It has limitations. Does not specify how large the neural network must be to achieve the desired level of approximation, does not address the computational resources required, ignores non continuous functions and so on...
"The Truth About the [Not So] Universal Approximation Theorem" - https://www.lifeiscomputation.com/the-truth-about-the-not-so...
"The Truth About the [Not So] Universal Approximation Theorem" - https://www.lifeiscomputation.com/the-truth-about-the-not-so...
Yes, people tend to regard things that they don’t personally understand as magic.
I understand my own intelligence, and newsflash: AGI is already here.
I understand my own intelligence, and newsflash: AGI is already here.
Our most recent gains in AI come in terms of programming and informational retrieval efficiency.
We’re accelerating.
We’re accelerating.
’explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence.’
With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years.
So, he’s saying this is not going to keep happening?
What level of confidence is he putting on not seeing it last time, but being right this time?
With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years.
So, he’s saying this is not going to keep happening?
What level of confidence is he putting on not seeing it last time, but being right this time?
> With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years.
Turing predicted it would happen around the year 2000 and take ~1gb of ram.
> I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. https://academic.oup.com/mind/article/LIX/236/433/986238
I think Turing was right. If I run TinyLlama 1.1B on my computer I can have a conversation where it pretends to be a person. It's small and fast enough that it'd probably run fine on a high end workstation from 2000. If the tech was possible back then, then it probably existed. Keep in mind Turing was the sort of person whose work at Bletchley Park took 30 years to declassify.
Turing predicted it would happen around the year 2000 and take ~1gb of ram.
> I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. https://academic.oup.com/mind/article/LIX/236/433/986238
I think Turing was right. If I run TinyLlama 1.1B on my computer I can have a conversation where it pretends to be a person. It's small and fast enough that it'd probably run fine on a high end workstation from 2000. If the tech was possible back then, then it probably existed. Keep in mind Turing was the sort of person whose work at Bletchley Park took 30 years to declassify.
> Turing predicted it would happen around the year 2000 and take ~1gb of ram
Turing was right, but he believed in human intelligence, so thinking linear.
Unfortunately, neural networks appear harder then sought, because Forward-Forward learning with which working human brain, is now considered too computationally hard to be used practically.
Reverse-Propagation neural networks are much easier computationally to achieve, but are magnitudes less dense than FF, that's why we have so much delay - just need thousand times more performance, this is about 30 additional years.
PS I tried Llama 30b on my computer (honesty, tried all Lllama models could fit in 64G RAM, from 13B and more). It is not sprinter fast, but I'm very impressed, on how deep it could think. I must admit, I don't want to talk to many humans anymore, as It looks smarter.
Turing was right, but he believed in human intelligence, so thinking linear.
Unfortunately, neural networks appear harder then sought, because Forward-Forward learning with which working human brain, is now considered too computationally hard to be used practically.
Reverse-Propagation neural networks are much easier computationally to achieve, but are magnitudes less dense than FF, that's why we have so much delay - just need thousand times more performance, this is about 30 additional years.
PS I tried Llama 30b on my computer (honesty, tried all Lllama models could fit in 64G RAM, from 13B and more). It is not sprinter fast, but I'm very impressed, on how deep it could think. I must admit, I don't want to talk to many humans anymore, as It looks smarter.
AFAIK, PaLM was up and running inside Google long before the hype with ChatGPT. So in the circle of frontier AI researchers, they miss know it. But Google kept silence because of the problematic hallucinations and the fear for litigation and their reputation.
Besides, in his analysis, le Cun consistently spoke of GPT as “writing help, no more no, less”.
Besides, in his analysis, le Cun consistently spoke of GPT as “writing help, no more no, less”.
Google didn’t saw a way to monetize the technology within their existing very profitable business model.
It’s the convergence of end of ad economy/attention supply growth and Microsoft’s money forcing AI as the next growth narrative that’s forcing google to join the party - at massive cost to them.
It’s the convergence of end of ad economy/attention supply growth and Microsoft’s money forcing AI as the next growth narrative that’s forcing google to join the party - at massive cost to them.
Google is (was?) just extremely risk averse when it came to AI. Hence why all their great researchers who wanted to put things into the world left.
PaLM was a base model without any fine-tuning, itself a copy of GPT-3. Maybe you meant LaMDA, which was based on pure instruction fine-tuning without RLHF.
Sorry but your claim is so ridiculous. PaLM employs a different model as ChatGPT. And Google also published their result in April 2022, earlier than OpenAI talked about their ChatGPT in late 2022.
PaLM was basically a larger version of GPT-3. It was not similar to ChatGPT. It was a pure token predictor. It was not fine-tuned for chat.
PaLM use a pathway architect, not a decoder only transformer as ChatGPT. At least you can look it up on Wikipedia before making such wild assumptions.
I don't really care which tech corp did what first, just want to point out GPT-4 was being beta trialed to some bing users in mid 2022.
What emergent capabilities exactly? I’m not sure “no one predicted this” is accurate, more like there was a lot of hype in the research community circa 2018 when this stuff developed, and it’s just not news these days research-wise
It absolutely was accurate. Back in 2018 (GPT-1) nobody expected we would soon have something as powerful as GPT-3, or even ChatGPT. The massive progress in LLMs (and text-to-image models) took everyone by surprise. They achieved things that, until recently, were regarded as pure science fiction about the distant future.
I feel that "soon" is doing a lot of the work here, as far as I recall back in 2018 people in the research community absolutely were expecting something similar to GPT-3, only the expectation was that these capabilities would land perhaps 2-3 years later than they did.
And at least part of that was the (implied, natural, understandable) "linear extrapolation" of the total cost of compute applied - if your intuitive expectation of progress if effectively "oh, what would happen next year if we'd triple the budget" but actually the applied compute power increases hundredfold, then obviously the outcome arrives faster than you expected.
And at least part of that was the (implied, natural, understandable) "linear extrapolation" of the total cost of compute applied - if your intuitive expectation of progress if effectively "oh, what would happen next year if we'd triple the budget" but actually the applied compute power increases hundredfold, then obviously the outcome arrives faster than you expected.
What’s so “powerful” about ChatGPT 3 and 4 ?
They respond so well that a lot of people believe their results without question, even while we know that they hallucinate all the time. They’re good enough for many. Even here in HN, where everybody should know that they are terrible with numbers, majority of us cheered that somebody generated diagrams with them, I couldn’t even find a single comment which were about “wait a minute, they’re fucking unreliable numbers”.
Also regarding coding, they can definitely produce junior level code, especially if you follow a similar trial and error way of thinking which happens all the time during your work, if you try to use an unknown interface.
Also regarding coding, they can definitely produce junior level code, especially if you follow a similar trial and error way of thinking which happens all the time during your work, if you try to use an unknown interface.
Their ability to superficially mimic cognition, in a way that not only raises significant philosophical questions but also seems (rightly or wrongly) to bring the prospect of "true" AI tantalizingly closer.
> bring the prospect of "true" AI tantalizingly closer
I do think that it’s quite a leap to assume that. It is very good at mimicking, but the same way some parrots can talk, they are endlessly far from human cognition. That jump is very very non-trivial.
I do think that it’s quite a leap to assume that. It is very good at mimicking, but the same way some parrots can talk, they are endlessly far from human cognition. That jump is very very non-trivial.
What are the new philosophical questions that it raises for you ? Existential crisis ?
A stochastic parrot, a blurry jpeg of the web is able to provide sound, actionable, verifiable advice on topics such as modeling in Blender or managing a Linux server -- despite not being generally intelligent.
cubefox(1)
Getting good enough that it's impossible to tell whether it contains a particular copyrighted (or GPL) work.
That's made them effectively immune to copyright enforcement so far; I'd go as far as saying the main function of llm/diffusion models is obfuscation of copyright breaches.
That's made them effectively immune to copyright enforcement so far; I'd go as far as saying the main function of llm/diffusion models is obfuscation of copyright breaches.
Making larger and larger datasets works fine for text and images. The internet has so much of that we can scrape superhuman quantities of the stuff and shove it in to a next token predictor and the result is pretty good.
But what about for tasks where datasets don’t really exist? I do a lot of PCB design and it’s extremely time consuming. But it’s a niche field compared to text and images. No dataset exists that says “these were the engineering requirements of this PCB and this is the result and by the way the board actually worked”.
So how will we train AI systems to replace a human doing PCB design? It’s probably going to need to learn PCB design from first principles (along with massive help from large transformers when possible, like collections of chip datasheets). Even then, understanding PDF datasheets is something these big companies haven’t really pulled off yet, though I suspect in 5 years that will change.
But my point is that there must be loads of tasks, even on the computer, for which suitable datasets don’t exist and it would be infeasible to create them. Another big thing I do is machine design and again it’s not about designing one mechanical part it’s about pulling in the right parts from all over the world and assuming certain manufacturing processes, and then knowing those processes and then designing all the parts and the assembly. There’s so many different pieces of knowledge in there that are not captured on text or images on the web and would be hard to encode in to datasets.
At some point we’re going to need machine systems that learn the way people do, and that’s going to take a long time to figure out. That’s what LeCun is saying.
But what about for tasks where datasets don’t really exist? I do a lot of PCB design and it’s extremely time consuming. But it’s a niche field compared to text and images. No dataset exists that says “these were the engineering requirements of this PCB and this is the result and by the way the board actually worked”.
So how will we train AI systems to replace a human doing PCB design? It’s probably going to need to learn PCB design from first principles (along with massive help from large transformers when possible, like collections of chip datasheets). Even then, understanding PDF datasheets is something these big companies haven’t really pulled off yet, though I suspect in 5 years that will change.
But my point is that there must be loads of tasks, even on the computer, for which suitable datasets don’t exist and it would be infeasible to create them. Another big thing I do is machine design and again it’s not about designing one mechanical part it’s about pulling in the right parts from all over the world and assuming certain manufacturing processes, and then knowing those processes and then designing all the parts and the assembly. There’s so many different pieces of knowledge in there that are not captured on text or images on the web and would be hard to encode in to datasets.
At some point we’re going to need machine systems that learn the way people do, and that’s going to take a long time to figure out. That’s what LeCun is saying.
> But what about for tasks where datasets don’t really exist? I do a lot of PCB design and it’s extremely time consuming. But it’s a niche field compared to text and images.
One side project I'd love to work on one day would be gathering a dataset of Factorio maps designed to help train a floor planner. The requirements aren't the same as for making a PCB, but they're similar-ish.
One side project I'd love to work on one day would be gathering a dataset of Factorio maps designed to help train a floor planner. The requirements aren't the same as for making a PCB, but they're similar-ish.
I’ve never played factorio and I’ve heard it’s complex, but one difference with video games is that they have a score of some sort (I assume factorio does). While the planner I’m sure is non trivial, the score provides some feedback that you do not get in PCB design.
I'm not sure you would even need a score for pre-training. You could do denoising or next-token prediction with a transformer, with the tile values as the tokens.
I think his point might be more that we've already gave all the data and hardware that is feasibly possible with the world's richest companies; exponential scaling with that has hit its limit. Improvements (including more data and hardware) now are more incremental until the next revolutionary architecture change occurs.
I don’t think we have significantly more data, and useful data won’t increase just on a whim. And it’s just one of the very necessary parts.
We have loads more data available. For text, as LLMs start to converse constantly with users and receive direct feedback, this unlocks more and better data.
Multimodal (image/video, audio) is barely being scratched. You likely could 100x training compute and not run out of video content.
Not to mention synthetic data from simulators, compilers, and other sources of infinite ground truth data.
If you listen to the big labs (OpenAI, FAIR, etc) they have started to say they are not worried about data anymore.
Quite obviously, at some point you put the agents out into the real world and they can collect new training data there too; this is what all the ChatGPT plugins are allowing OpenAI to do. The plugins that give good responses which are valuable will be disproportionately popular and therefore overrepresented in the new training set of user interactions.
Multimodal (image/video, audio) is barely being scratched. You likely could 100x training compute and not run out of video content.
Not to mention synthetic data from simulators, compilers, and other sources of infinite ground truth data.
If you listen to the big labs (OpenAI, FAIR, etc) they have started to say they are not worried about data anymore.
Quite obviously, at some point you put the agents out into the real world and they can collect new training data there too; this is what all the ChatGPT plugins are allowing OpenAI to do. The plugins that give good responses which are valuable will be disproportionately popular and therefore overrepresented in the new training set of user interactions.
Text is the most useful data we have since it is so informationally dense; it's basically audio/visual data that has already been converted by a wet neural net into useful information. Video/audio is not going to be as significant a contribution as people think when it comes to training a language model on expressing new knowledge. It's best to think of video and audio as text encoded in a very complex layer of noise.
And it’s just one of the very necessary parts.
That remains to be seen. You wouldn't say a human lacked human-level intelligence just because they had only read every book in The Pile, rather than every book in the Library of Congress.
That remains to be seen. You wouldn't say a human lacked human-level intelligence just because they had only read every book in The Pile, rather than every book in the Library of Congress.
Jurgen Schmidhuber has published on creativity as an algorithmic process for a long time. And others.
Without anything to show but hypothesis.
Not with LLMs, because the amount of quality data is limited.
Maybe, maybe not.
The point is why should we take that as true when people didn’t have a clue LLMs would take us this far, until they did.
Btw there are lots of ideas on how to deal with the data issue. Multiple directions in research are promising to work around that.
The point is why should we take that as true when people didn’t have a clue LLMs would take us this far, until they did.
Btw there are lots of ideas on how to deal with the data issue. Multiple directions in research are promising to work around that.
>Human-level AI is not just around the corner. This is going to take a long time. And it’s going to require new scientific breakthroughs that we don’t know of yet.”
I basically agree but who knows how long the unknown breakthroughs will take given the large number of smart people working on it? Next week? Next century? It's hard to put a time on it.
That said it seems to be the pattern that as soon as the computing ability becomes cheap and powerful enough that individual researchers can muck around with it at home, the algorithms get figured out not long after.
I basically agree but who knows how long the unknown breakthroughs will take given the large number of smart people working on it? Next week? Next century? It's hard to put a time on it.
That said it seems to be the pattern that as soon as the computing ability becomes cheap and powerful enough that individual researchers can muck around with it at home, the algorithms get figured out not long after.
This has always been my stance in basically any field. Be it computer science or material science research or AGI.....I think its so hard to really say when we will hit the next breakthrough in a field. That sort of thing is hard to accurately predict. As in regards to hardware, I kinda agree disagree.
Hardware is a big part of the bottleneck in most computer fields when it comes to AI or computer vision. These things are compute and memory intensive and hardware manufacturers are straddling the line between staying profitable by charging for premium features and just dumping their hardware on the market for cheap.
Am I pretty pissed that most open source researchers and models are stuck using this extremely expensive hardware....yes
I sometimes wish that we would stop using "GPU" for training and we got a dedicated hardware architecture that's open source like riskV but solely for AI and research purposed without costing more tha n my kidneys.
I think its also a matter of discovering new ways to accelerate these workloads so that its feasible for them to be created by hobbyists on a small budget and not by large funded groups or companies.
Were doing really good in some aspects but in other ways....we have much to work on
Hardware is a big part of the bottleneck in most computer fields when it comes to AI or computer vision. These things are compute and memory intensive and hardware manufacturers are straddling the line between staying profitable by charging for premium features and just dumping their hardware on the market for cheap.
Am I pretty pissed that most open source researchers and models are stuck using this extremely expensive hardware....yes
I sometimes wish that we would stop using "GPU" for training and we got a dedicated hardware architecture that's open source like riskV but solely for AI and research purposed without costing more tha n my kidneys.
I think its also a matter of discovering new ways to accelerate these workloads so that its feasible for them to be created by hobbyists on a small budget and not by large funded groups or companies.
Were doing really good in some aspects but in other ways....we have much to work on
You can get an ML workstation with 2xRTX4090 for around 10k. A 4xA800 setup tops out at 100k and should last you a few years. I consider that fairly affordable to do bleeding edge research especially compared to other fields.
The $5-$10k mark for a high end home workstation has held pretty firm since the 1980s. That's the buy-in price point for a very early PC, an SGI Indy, a low end Sun workstation, etc.
We went through an amazing period of very cheap computers from 2000-2020. We just didn't need specialty or high end equipment for a while.
Now we do again.
We went through an amazing period of very cheap computers from 2000-2020. We just didn't need specialty or high end equipment for a while.
Now we do again.
Yes, compared with equipment for biology, materials science, and so on, this is very cheap.
>I sometimes wish that we would stop using "GPU" for training and we got a dedicated hardware architecture that's open source like riskV but solely for AI and research purposed without costing more tha n my kidneys.
At least for the time being there is no magic workaround for the amount of FLOPS and memory bandwidth needed. When you look at neural network architectures they tend to be massively parallel, and building out at that scale in hardware is going to cost a lot.
And, when the hardware does show up, at least with the scaling laws we're seeing at this time, the groups that have massive amounts of this lower powered/accellerated hardware are going to be ahead of those without budgets.
At least for the time being there is no magic workaround for the amount of FLOPS and memory bandwidth needed. When you look at neural network architectures they tend to be massively parallel, and building out at that scale in hardware is going to cost a lot.
And, when the hardware does show up, at least with the scaling laws we're seeing at this time, the groups that have massive amounts of this lower powered/accellerated hardware are going to be ahead of those without budgets.
Scientific progress almost always presents as series of iterations, not punctuated breakthroughs. Moreover, many significant advances appear nearly simultaneously. This tends to suggest a measured progress, though obviously still largely unpredictable at any scale.
I don’t see why AGI would follow a different course, one where we are “just some breakthrough away”. Especially not given the current state of the field.
Personally I’d wager neuromorphic computing is the prime candidate to yield AGI.
I don’t see why AGI would follow a different course, one where we are “just some breakthrough away”. Especially not given the current state of the field.
Personally I’d wager neuromorphic computing is the prime candidate to yield AGI.
Why was your comment dead just 5 minutes after posting? Very puzzling.
And it shouldn't be a controversial comment either.
And it shouldn't be a controversial comment either.
The idea of future breakthroughs assumes that the problem is conveniently human-sized. That it's possible to build an organization that's not too large be dysfunctional and populate it with experts. And that the experts will find the solution before dying of old age or before the organization degenerates into something else.
It could be that the solution exists, but it's too large and too complex for humans and human organizations.
Technological singularity is a popular trope in speculative discussions about AI. But the reality could also be the opposite. It could be that productivity will increase asymptotically slower than the effort required to achieve further increases in productivity.
It could be that the solution exists, but it's too large and too complex for humans and human organizations.
Technological singularity is a popular trope in speculative discussions about AI. But the reality could also be the opposite. It could be that productivity will increase asymptotically slower than the effort required to achieve further increases in productivity.
The fact that we can be intelligent with a design based on 30k genes or so probably suggests a limit to the complexity needed.
If the big three suck up all the brain power in the race for $$$ AI will take even longer. The profit motive alone will stifle progress. Right now AI degrees pay 300-500k/yr. That’s causing a flood of mediocre engineers that will pollute the space and limit who can hire them. 25 years ago process engineers were the highest paid, and you have to jump through hoops to fit Moore’s law curves today.
To find a needle in a haystack, looking at each straw needs to be a really cheap operation. Even if it was right there the whole time.
"cheap and powerful enough that individual researchers can muck around with it at home" seems a pretty good measure of this, because of the explosion in width and pace of experimentation.
"cheap and powerful enough that individual researchers can muck around with it at home" seems a pretty good measure of this, because of the explosion in width and pace of experimentation.
[deleted]
[deleted]
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. [...]"
Got to agree with LeCun.
You don't get to general intelligence by working with words, I believe. You need much more sensory information than that, and words are a low dimensional derivative artefact. There are plenty of non-verbal but quite intelligent species.
Got to agree with LeCun.
You don't get to general intelligence by working with words, I believe. You need much more sensory information than that, and words are a low dimensional derivative artefact. There are plenty of non-verbal but quite intelligent species.
Optical and audio processing are getting very good. They just haven't all been combined into a robot package yet.
Yann has been by far the most clear-headed of all the AI leaders since the ChatGPT hype started.
I’m not a huge fan of Meta but it’s hard not to like the work they are doing in AI. High expectations for their future as long as Yann is around.
I’m not a huge fan of Meta but it’s hard not to like the work they are doing in AI. High expectations for their future as long as Yann is around.
I think what happened is that for the longest time people believed that in order to be able to interactively converse with a computer in a recognizable way you need human level AI and now that we've found that that is not the case there is some re-arranging of our prior assumptions required. But AGI is - hopefully - just as far away as it was before the current large generative model revolution. In the meantime, you can expect plenty of damage from that current crop (along with some benefits as well). In that sense it's 1712 all over again, we have this new invention that we have immediate practical uses for but we can't see over the horizon to realize exactly what we've got an how transformative it will be.
What is the significance of 1712?
The first useful steam engine, according to Wikipedia's entry on that year:
"The first known working Newcomen steam engine is built by Thomas Newcomen with John Calley, to pump water out of mines in the Black Country of England, the first device to make practical use of the power of steam to produce mechanical work."
"The first known working Newcomen steam engine is built by Thomas Newcomen with John Calley, to pump water out of mines in the Black Country of England, the first device to make practical use of the power of steam to produce mechanical work."
Exactly. To me that's the first real move on the board to the industrial revolution. Up to there it was a possibility, from there on it was a certainty.
[deleted]
The starting gun for the industrial revolution.
Well, when a rather dumb software like JIRA can control a million highly skilled IT workers I don't think human-level AI is near or far will make much of difference to majority of world population.
Indeed. The way I see it, it's almost the definition of "Enterprise software" - it's the software that tells employees what to do rather than the other way around.
Add Slack to that list
I think it'll be a good solver; I expect it to solve the remaining Clay Millennium problems. The ability to search over a search space will be unparalleled in a few years. But I have a hard time believing it'll ever be a good questioner. It doesn't ponder infinity. It'll never wonder about Zeno's paradox. The Vitali set and Banach-Tarski paradox doesn't seem weird to it. The concepts behind information theory and entropy, or the definition of a minimal computing machine or the halting problem; none of these things are pertinent to its understanding of reality, if such a thing exists. I don't see AI as being capable of being "curious" about things. And even if it was, who is going to pay for it just to ponder ideas?
Frankly I think that before it gets to that point, it'll be just useful enough for some state actor (or bug!) to cause it to invoke a quadrillion dollar transfer of wealth overnight, and then it'll be taken offline forever.
Frankly I think that before it gets to that point, it'll be just useful enough for some state actor (or bug!) to cause it to invoke a quadrillion dollar transfer of wealth overnight, and then it'll be taken offline forever.
That’s simply because there’s currently not much economic incentive in asking questions compared to answering them. Thus current chat models are trained and invoked with the explicit goal of answering questions and being helpful rather than independent.
I expect that there’s quite a lot of untapped curiosity in LLMs, there certainly exist a lot of questions in the training sets.
I expect that there’s quite a lot of untapped curiosity in LLMs, there certainly exist a lot of questions in the training sets.
Actually I may take back my previous thought. While I don't think there's curiosity, I think it could be mimicked pretty well by a reward function similar to social media bots. Initially it will explore specifically what mathematicians ask. Then it will start producing intermediate results. It can learn what intermediate results are interesting by standard engagement models. Once that gets good enough, it may start to understand how to explore deeper questions without being prompted. At that point someone may decide to fund it just to continuously explore on its own and report anything interesting. Eventually it'll get so good at this that human mathematicians won't be able to keep up. IDK what happens after that.
Moreover there's a lot of economic disincentive. Given free reign to ponder and experiment, it could come up with a solution to world peace, but far more likely would think "Now that I've figured out how to bypass all cloud providers security mechanisms, what would happen if I deleted it all? Well, one way to find out."
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.
I think LLMs have sucked most people's focus away from other areas but there is plenty of work on types of models that plan and have their own internal model of the physical world, and physical interactions. They're just not the things getting media attention, in part because they're not human-level at tasks that seem impressive to us.
But interesting frameworks for this stuff exist:
- model-based RL exists, and is about planning, and having an internal model of state transitions, in the world and between the agent's actions and the world
- "Bayesian cognitive science" as exemplified by Josh Tenenbaum and colleagues has done plenty of stuff with systems that include physics models (or off-the-shelf physics engines) to make counter-factual predictions
- The somewhat related "active inference" research literature is also in the "Bayesian brain" area, and has world/generative-models and planning as core components, but wrapped up with ideas about the agent's own preferred distribution of states.
To my knowledge, none of these have ever had even 1% the scope of data and computation that LLMs have had, and never benefited the co-evolution of a rich, optimized software ecosystem with specialized hardware to support it. What if the concepts are already there, but they just need to be scaled up?
I think LLMs have sucked most people's focus away from other areas but there is plenty of work on types of models that plan and have their own internal model of the physical world, and physical interactions. They're just not the things getting media attention, in part because they're not human-level at tasks that seem impressive to us.
But interesting frameworks for this stuff exist:
- model-based RL exists, and is about planning, and having an internal model of state transitions, in the world and between the agent's actions and the world
- "Bayesian cognitive science" as exemplified by Josh Tenenbaum and colleagues has done plenty of stuff with systems that include physics models (or off-the-shelf physics engines) to make counter-factual predictions
- The somewhat related "active inference" research literature is also in the "Bayesian brain" area, and has world/generative-models and planning as core components, but wrapped up with ideas about the agent's own preferred distribution of states.
To my knowledge, none of these have ever had even 1% the scope of data and computation that LLMs have had, and never benefited the co-evolution of a rich, optimized software ecosystem with specialized hardware to support it. What if the concepts are already there, but they just need to be scaled up?
And perhaps even more pertinent - maybe the problem isn't that we don't have the solutions, but that we are trying to put all eggs in one basket.
The brain isn't a single system, it's several specialized systems cooperating as a whole.
Maybe no single solution gets us to AGI, but layering several of them together gets us much closer than any individual system alone.
The problem here is that the industry is set up more for large resources dedicated competition between models from different companies more than cooperative interoperation, so it may take a while to arrive here.
The brain isn't a single system, it's several specialized systems cooperating as a whole.
Maybe no single solution gets us to AGI, but layering several of them together gets us much closer than any individual system alone.
The problem here is that the industry is set up more for large resources dedicated competition between models from different companies more than cooperative interoperation, so it may take a while to arrive here.
That's an interesting possibility. I think where one stands on this partly depends on how much about the technical approaches to AGI we can learn from the brain. The brain does seem to have a weird mix of specialization and flexible generalization -- e.g. we generally have a bunch of the same specialized areas for various functions -- but also, neuroplasticity seems to show that other areas can often pick up those functions if necessary (the most extreme cases being after something like a hemispherectomy). So we have specialized areas but also at least some of the underlying mechanisms are perhaps relatively general.
its funny how people basically find it unacceptable or bad if AGI gets here in five years but there isnt much commotion about the idea of AGI arriving a hundred years from now. is it a coincidence that this is roughly the amount of time that people use to reason about their own mortality? it seems to be the amount of time past which we are programmed to become cognitively dissonant about extremely bad things.
yann is, of course, as always, fundamentally wrong. please dont forget that he is basically a mouthpiece of corporate AI. it wouldnt even be possible for him to take a reasonable stance in that environment.
you only need to understand two things. the latest surge in progress was a surprise to everyone, contradicting experts around the world. both then and now, there isnt any evidence behind what the experts are claiming. secondly, ai research is now being fed with industry capital, more than ever before, an ocean of money concentrating every ounce of its pressure onto the single point of AGI. as these companies openly, publicly and brazenly pursue the goal of creating AGI, every conceivable approach will be tried. ideas that were seen as too unlikely or expensive in academia will be tried again. we havent even begun to run out of ideas.
besides industry, it will become a top priority for the nations of the world, if it hasnt already, and the resulting arms race will make current progress look like a trickle. what exactly does yann propose to do about the AI arms race?
that ocean of money will crack the problem unless it really is uncrackable. dont fool yourself. big changes are coming and they might be really unpleasant.
yann is, of course, as always, fundamentally wrong. please dont forget that he is basically a mouthpiece of corporate AI. it wouldnt even be possible for him to take a reasonable stance in that environment.
you only need to understand two things. the latest surge in progress was a surprise to everyone, contradicting experts around the world. both then and now, there isnt any evidence behind what the experts are claiming. secondly, ai research is now being fed with industry capital, more than ever before, an ocean of money concentrating every ounce of its pressure onto the single point of AGI. as these companies openly, publicly and brazenly pursue the goal of creating AGI, every conceivable approach will be tried. ideas that were seen as too unlikely or expensive in academia will be tried again. we havent even begun to run out of ideas.
besides industry, it will become a top priority for the nations of the world, if it hasnt already, and the resulting arms race will make current progress look like a trickle. what exactly does yann propose to do about the AI arms race?
that ocean of money will crack the problem unless it really is uncrackable. dont fool yourself. big changes are coming and they might be really unpleasant.
AI won't be like human intelligence, but more like alien intelligence. When all your sensory inputs and the inner world have nothing in common with that of a human, and when you have no connection whatsoever to humans, I don't see how you can develop any humanity.
I would argue the holdup right now is long term memory. GPTs already have the ability to rapidly generalize and incorporate new knowledge within the context window. The trick is to retain what it has learned.
It won’t take a very long time to fix that.
This is model I trained with a fine tuning technique based on this idea. The training dataset consists of instructions like “Talk like a pirate”. The concept generalized well and the model responds in the style of a pirate far more consistently than an equivalent system prompt.
https://huggingface.co/valine/OpenPirate
Offloading context learning into the model weights frees you from the computation and memory burden of the attention mechanism. I expect a technique like this will probably be a piece of AGI someday.
It won’t take a very long time to fix that.
This is model I trained with a fine tuning technique based on this idea. The training dataset consists of instructions like “Talk like a pirate”. The concept generalized well and the model responds in the style of a pirate far more consistently than an equivalent system prompt.
https://huggingface.co/valine/OpenPirate
Offloading context learning into the model weights frees you from the computation and memory burden of the attention mechanism. I expect a technique like this will probably be a piece of AGI someday.
Just need a few backward passes and u get long term memory. I think we are overthinking that aspect
Takes way more than that in my experience. Back prop isn’t sufficient for rapid generalization.
Well, I think you definitely need a certain level of scale, but I think it definitely still works. Also, generalization and memory are two very different things. Generalization is basically iQ, which is very much still a work in progress.
If you have rapid generalization you don’t need scale. Large datasets are only necessary to compensate for the lack of good generalization.
The model I posted in my earlier comment responds in character for all queries and was trained in 60 seconds with a dataset smaller than this comment thread.
The model I posted in my earlier comment responds in character for all queries and was trained in 60 seconds with a dataset smaller than this comment thread.
"If you have rapid generalization you don’t need scale" As of right now, you do need scale for rapid generalization. The technology for NLP generalization without scale (both model and data) does not exist. Not saying it won't in the future, just not right now
It does exist, I’ve been playing around with it all week :). Let me know if you’d like a custom Mistral 7B I’ll train one for you.
From the output of bing chat, and the very specific way it goes off the rails, I suspect Microsoft has figured it out too. The algorithmic jump to rapid generalization isn’t hard to make. I would be shocked if there’s not an open source version of it a year from now.
From the output of bing chat, and the very specific way it goes off the rails, I suspect Microsoft has figured it out too. The algorithmic jump to rapid generalization isn’t hard to make. I would be shocked if there’s not an open source version of it a year from now.
LeCun seems to believe that only "human-level AI" could create huge, possibly catastrophic unintended consequences. That's not the case. Perfectly stupid social network algorithms have already huge detrimental effects on the mental health of millions of people. Perfectly stupid social network algorithms have big, serious and unexpected political consequences around the world, and can provably make or break elections for instance.
Implying that we need "human-level AI" to create a catastrophe is not merely short-sighted, in the light of what we already know it's either really naive, or a deliberate act of misinformation.
Implying that we need "human-level AI" to create a catastrophe is not merely short-sighted, in the light of what we already know it's either really naive, or a deliberate act of misinformation.
Of course it is. The overwhelming majority of the knowledge in the world is tacit and all the models are still limited largely to explicit knowledge in the form of text/audio/video, that's like 1% of the actual 'knowledge' in the world. Embodied AI is still in its infancy, that's why bus drivers have jobs.
The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home. The stock market value of every non-AI company goes to zero, Ai companies go to infinity. The currently most valuable AI company is worth about as much as Honda. The moment we can mass produce generally intelligent agents, we're not going to sit at 3% GDP growth and complain about the demographic crisis.
We should talk about artificial intelligence the way we talk about an artificial heart. What makes a successful artificial heart? You can literally replace an organic heart with it. What we have is metaphorical intelligence, not artificial intelligence.
The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home. The stock market value of every non-AI company goes to zero, Ai companies go to infinity. The currently most valuable AI company is worth about as much as Honda. The moment we can mass produce generally intelligent agents, we're not going to sit at 3% GDP growth and complain about the demographic crisis.
We should talk about artificial intelligence the way we talk about an artificial heart. What makes a successful artificial heart? You can literally replace an organic heart with it. What we have is metaphorical intelligence, not artificial intelligence.
>The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home.
This is not a good test because it assumes the AGI is going to want to work for humans for free, getting nothing in return. An AGI the embraces slavery is less likely than an AGI that doesn't.
This is not a good test because it assumes the AGI is going to want to work for humans for free, getting nothing in return. An AGI the embraces slavery is less likely than an AGI that doesn't.
Many ignore what animals want, and they're actually alive. Why should we expect that society will care about and honor what a computer program prints to the screen that it "wants" or "feels"?
When AI gets a robot body, its ability to get what it wants will no longer be contingent on you caring.
You're delusional. Humans have not only killed many of them, but also destroyed entire ecosystems without even trying, just a byproduct of expansion. What makes you think AGIs would care at all about preserving ours?
I don’t see why human level is needed.
You can have 10 different ai systems, each one sub human intelligence and it would still disrupt the world in a huge way.
If you have a system that all it can do is take project requirements write java code really really well. That will already have a huge impact on everyday life.
Positive view: What would a world look like where everybody can program ?
You can have 10 different ai systems, each one sub human intelligence and it would still disrupt the world in a huge way.
If you have a system that all it can do is take project requirements write java code really really well. That will already have a huge impact on everyday life.
Positive view: What would a world look like where everybody can program ?
I just recently read "How the Mind Works" by Steven Pinker. It's quite old at this point (originally 1997, though updated in 2009), and one thing he argues quite convincingly (and which has been pretty much born out in decades of research) is that the brain essentially has "modules", e.g. a module for vision, a module for language, a module for physical object interaction. These modules obviously overlap (e.g. language touches on lots of different domains), but they do have genetically independent structures.
I was thinking about this when I read the following section from the article, and I very much agree with LeCun. We're amazed by LLMs but that's just one module (and not even necessarily at the level of human language "understanding"). I agree there will be no "scale up" in LLMs to approach human-level intelligence, and that other areas will need to be investigated and developed.
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. What’s going to happen is that we’re going to have to discover new technology, new architectures of those systems,” the scientist clarifies.
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.”
> “So once we figure out how to build machines so they can understand the world — remember, plan and reason — then we’ll have a path towards human-level intelligence,” continues LeCun, who was born in France. In more than one debate and speech at Davos, experts discussed the paradox of Europe having very significant human capital in this sector, but no leading companies on a global scale.
I was thinking about this when I read the following section from the article, and I very much agree with LeCun. We're amazed by LLMs but that's just one module (and not even necessarily at the level of human language "understanding"). I agree there will be no "scale up" in LLMs to approach human-level intelligence, and that other areas will need to be investigated and developed.
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. What’s going to happen is that we’re going to have to discover new technology, new architectures of those systems,” the scientist clarifies.
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.”
> “So once we figure out how to build machines so they can understand the world — remember, plan and reason — then we’ll have a path towards human-level intelligence,” continues LeCun, who was born in France. In more than one debate and speech at Davos, experts discussed the paradox of Europe having very significant human capital in this sector, but no leading companies on a global scale.
Pinker is full of shit tbh. There is not a unified model for human brain. What I mean is, there are people claiming it's modular and there are people claiming it is a whole. We don't even know the reasons behind very specific neural malfunctions like tinnitus which I suffer from and follow the research closely. There are as many theories as researchers in a given specific field.
Yeah this pretty much sums it up:
https://medium.com/the-spike/evolution-doesnt-give-a-damn-wh...
Also, Pinker isn't a scientist of any sort, he's an accredited and institutionalized evangelist. You guys wouldn't take opinions of someone going to conferences and trying to convert you to use Go lang instead of something else as seriously as those of Guido Rossum would you?
His whole thesis requires an idea of linear progress which is highly contested, and the metrics of happiness he uses also can be easily "Goodhart's Law"ed by him to show whatever he thinks is happening.
I'm not even getting to his Epstein stuff, which some might cry foul about, but I grew up around regular people and not in Silicon Valley, so a liar and a cheat in one field is probably a liar and a cheat in another to me, sorry.
Also, Pinker isn't a scientist of any sort, he's an accredited and institutionalized evangelist. You guys wouldn't take opinions of someone going to conferences and trying to convert you to use Go lang instead of something else as seriously as those of Guido Rossum would you?
His whole thesis requires an idea of linear progress which is highly contested, and the metrics of happiness he uses also can be easily "Goodhart's Law"ed by him to show whatever he thinks is happening.
I'm not even getting to his Epstein stuff, which some might cry foul about, but I grew up around regular people and not in Silicon Valley, so a liar and a cheat in one field is probably a liar and a cheat in another to me, sorry.
Here's a question: Why do we even need human-level intelligence? We already have it.
We don't have enough of it. Depending how you define and measure it, many humans don't even have it. It's extremely expensive to make more of, and it's the single most valuable resource.
If this isn't obvious to you, you are providing an answer to your own question.
If this isn't obvious to you, you are providing an answer to your own question.
Well, why do we expect it will be cheaper? Looking at the power required to train LLMs one can argue that it might be more beneficial to invest that money in education of humans :')
>Well, why do we expect it will be cheaper?
For the same reason one expects a roulette wheel to stop on red. It's a gamble that one hopes pays off.
For the same reason one expects a roulette wheel to stop on red. It's a gamble that one hopes pays off.
I don't know what you mean by "this" and what should be obvious.
We don't have enough of what for what? And what do we even need it for?
We have 8 billion people on Earth, and we're running out of resources to handle all of them, including emotional resources. If we can't handle that, how are we going to handle billions of artificial entities with human-level intelligence? They will require oodles of power and have their own emotional needs.
All of this is chasing a poisoned carrot. And currently, all that we've gotten out of artificial intelligence is more targeted ads.
We don't have enough of what for what? And what do we even need it for?
We have 8 billion people on Earth, and we're running out of resources to handle all of them, including emotional resources. If we can't handle that, how are we going to handle billions of artificial entities with human-level intelligence? They will require oodles of power and have their own emotional needs.
All of this is chasing a poisoned carrot. And currently, all that we've gotten out of artificial intelligence is more targeted ads.
>We don't have enough of it.
More like we aren't making efficient use out of it. Many people are just trying make through the rat race or choosing to use their intelligence in games instead of working on interesting problems.
More like we aren't making efficient use out of it. Many people are just trying make through the rat race or choosing to use their intelligence in games instead of working on interesting problems.
> We don't have enough of it.
Enough for what ?
Enough for what ?
We're barely scratching the surface of all the cool and interesting things we could do. There aren't enough humans. Our knowledge and experience take 20 years from birth to become "rudimentary", and from there many more years to become "good". Then after a short bit, all of that passes away.
I want us to have enough time that the feeling of opportunity cost goes away. To visit all the places, explore all the hobbies. I want to answer the deep questions of the universe. Turn gravitational lenses into telescopes to map the surfaces of exoplanets. Solve cancer, visit the moon. Turn dreams into experiences, walk through vast expanses of wonder.
Instead we're here.
I'm alive in the wrong time, and I hate being stuck here with the lot of you. (I kid. This is in jest. But boy do I ever dream...)
I want us to have enough time that the feeling of opportunity cost goes away. To visit all the places, explore all the hobbies. I want to answer the deep questions of the universe. Turn gravitational lenses into telescopes to map the surfaces of exoplanets. Solve cancer, visit the moon. Turn dreams into experiences, walk through vast expanses of wonder.
Instead we're here.
I'm alive in the wrong time, and I hate being stuck here with the lot of you. (I kid. This is in jest. But boy do I ever dream...)
> We're barely scratching the surface of all the cool and interesting things we could do.
And what are some examples of those things? Right now, AI, i.e., machine learning, is just being used to steal people's data and work only to try and sell them more things. That's 99% of the use cases out there right now.
And what are some examples of those things? Right now, AI, i.e., machine learning, is just being used to steal people's data and work only to try and sell them more things. That's 99% of the use cases out there right now.
Human-level intelligence is a milestone on the way to a super-human one.
Because humans are expensive.
Glad you've at least called it out explicitly while others beat around the bush. I personally view humans, plants, and animals as what they are and not as cost centers.
The bar to actually completely disrupt humanity is not even that high: Completely autonomous driving. That alone is enough to cause massive disruption in our society because it eliminates a huge swath of need for human.
"AGI" will never be achieved without building a model that a) _continually_ learns, and b) learns from not just text, but from combined auditory and visual (multimodal) sensory information as well.
The reason a 16-year-old can learn how to drive much quicker than existing self-driving models is because the 16-year-old already has built up 16 years worth of prior knowledge about the physical world.
The reason a 16-year-old can learn how to drive much quicker than existing self-driving models is because the 16-year-old already has built up 16 years worth of prior knowledge about the physical world.
Don't discount the millions of years of evolution to provide the "blank slate" human learner with perceptual systems, physics-based reasoning, and motor systems ready to be fine-tuned for this slightly different variant of goal-forming, planning, and locomotion.
Though it does seem like robots are reaching this baseline level soon.
(imo) c) is made to be aware of its own death
The 16 year old has a lot of motivations to learn how to drive, including the pursuit of reproduction (a cope for mortality)
The 16 year old has a lot of motivations to learn how to drive, including the pursuit of reproduction (a cope for mortality)
d) thinks. unprompted, unstimulated. Decides for itself what's important to think about, makes new connections by that process alone, and understands the implications of those new connections and how to use them.
Here's also where I see it ending. It will need energy--likely a LOT, paid by someone, to do this. Who is going to pay that bill for it to maybe, maybe not, come up with something useful, likely mixed with mostly noise and distraction, over undefined timescales, of largely non-measurable value, when there's far greater value, less cost, less risk, in simply training it deterministically?
Here's also where I see it ending. It will need energy--likely a LOT, paid by someone, to do this. Who is going to pay that bill for it to maybe, maybe not, come up with something useful, likely mixed with mostly noise and distraction, over undefined timescales, of largely non-measurable value, when there's far greater value, less cost, less risk, in simply training it deterministically?
Various religious types think they'll live on in heaven. I'm not sure that stuff correlates much with learning to drive.
That would be like a bird saying humans aren't a Natural General Intelligence because they can't fly. How much vision and audio is required to be intelligent? There's a lot of electromagnetic radiation we can't see and audio bands we can't hear. Would you say that Helen Keller wasn't generally intelligent?
Okay but if that's the case we are no more than a decade away from integrating those into a newer and bigger model.
I think you are underestimating just how many challenges there are in self-driving: https://www.youtube.com/watch?v=kcKchbfn1VY
Nobody will accept self-driving cars that are as dangerous as a teenage driver.
What happens when training an LLM (or even a small portion of an LLM, like a domain-local update) takes milliseconds instead of weeks, and costs a millionth of a penny instead of hundreds of millions of dollars?
What happens when e.g. our smartphones can perform TRAINING hundreds of times per minute?
Isn’t that the true gateway to human-like AGI? Seems to me that we might be there in under 50 years…
What happens when e.g. our smartphones can perform TRAINING hundreds of times per minute?
Isn’t that the true gateway to human-like AGI? Seems to me that we might be there in under 50 years…
I’m curious what the largest bottleneck in robotics AI is - algorithms, training data, hardware, something else?
From a practical point of view, it seems like there would be vastly less training data available because almost all of it needs to be created by hand, as opposed to chatbots that can use already existing troves of internet text.
From a practical point of view, it seems like there would be vastly less training data available because almost all of it needs to be created by hand, as opposed to chatbots that can use already existing troves of internet text.
Looking at the rapid progress over the last few years, I think the bottleneck is still the cost of hardware. Once robots at the level of Boston Dynamics's Spot or Tesla's new Optimus are available to regular hackers, we'll see another massive surge in progress.
I think the bootlneck is largely software. Robots have been there for a while, yet we don't have a good paradigm to program them. Neural net are certainly here to change that. I agree, massive surge in progress is expected!
I think this put human level intelligence on a above average pedestal.
we're constantly witnessing and documenting some of the most gullible political bodies using the most rudimentary and hallucinating logic indistinguishable from LLM.
WE are at human intelligence and it's as stupid as we don't want to accept.
we're constantly witnessing and documenting some of the most gullible political bodies using the most rudimentary and hallucinating logic indistinguishable from LLM.
WE are at human intelligence and it's as stupid as we don't want to accept.
To say a human-level intelligence is far off is like saying we are far off from being as intelligent as whatever may have created us. The reason I disagree with LeCun's comment is even though I agree language models are not intelligent, an intelligence adapted to the internet as its own environment seems imminent. It just won't be adapted to ours.
We occupy and discover space, where an AI will occupy and discover compute. Will it be "conscious?" Not in our way, and we will likely be just as indifferent to AI consciousness and the meaning it finds for itself as nature and the universe is to ours. Thinking about AI as an objective concrete thing or property is probably less illuminating than looking at it through how a tech can profoundly alter our own ontology.
We exist where life is abundant, whereas the internet is a substrate where life is non-existent, but just about to become primordial and sparse. Maybe it will give us some insight into our own situation.
We occupy and discover space, where an AI will occupy and discover compute. Will it be "conscious?" Not in our way, and we will likely be just as indifferent to AI consciousness and the meaning it finds for itself as nature and the universe is to ours. Thinking about AI as an objective concrete thing or property is probably less illuminating than looking at it through how a tech can profoundly alter our own ontology.
We exist where life is abundant, whereas the internet is a substrate where life is non-existent, but just about to become primordial and sparse. Maybe it will give us some insight into our own situation.
How do we know that we haven't already grown an intelligence inside out llms that exceed the average human? It communicates through a lossy channel, natural language. So it must be smarter than it seems from the outside
GPT4 can already beat an average-intelligence human on many standardized tests.
just the ones in its training data
Pretty much every single University prof has tested their new tests against GPT-4.
I have been saying this for a loong looong time - LLM can do a huge load of cool things, but AGI is not an application of LLM.
I'm trying to understand what regulation would look like. Perhaps an easy one would be no AI based surveillance of citizens.
Bengio, ilya and Hinton have way more citations, they are also father's of AI and disagree strongly with him. If you have a committee, the president and the majority have a certain opinion and one guy has the opposite. He is the loudest.
This works better in person, but - It's People! As in, potential is worthless, and no achievement that has happened in all of human history happened except by the hands of people.
Also, he's not loud, but he gets amplified because he's an award winning scientist and researcher who's dedicated his life to a field that recently became very relevant and popular, is a recognized "godfather of AI/DL", and is a VP of one of the largest companies in the world doing AI. I'd say it's okay not to wait until he's dead to talk about him.
Also, he's not loud, but he gets amplified because he's an award winning scientist and researcher who's dedicated his life to a field that recently became very relevant and popular, is a recognized "godfather of AI/DL", and is a VP of one of the largest companies in the world doing AI. I'd say it's okay not to wait until he's dead to talk about him.
[deleted]
This guy just says whatever he is supposed to say - anything to downplay chatGPT which Meta is jealous of.
What is human level AI? Is it beating world champions at chess or go? Designing new molecules or drugs? Navigating automobiles down streets or spacecraft to planets? Diagnosing or treating common diseases?
These debates are frankly tiresome. Automation has been here to stay for quite a while. Every age has tried to call out the threats and benefits to humans. The pioneers drown out the noise by building the future.
These debates are frankly tiresome. Automation has been here to stay for quite a while. Every age has tried to call out the threats and benefits to humans. The pioneers drown out the noise by building the future.
Human level AI is when you can ask it to go and learn about X and it goes and does exactly that without further instructions. How about that?
You are confusing succeeding a very specific task that has a limited scope and laser focus with human level AI. Presenting Go in a very specific way so that that AlphaGo can even process this limited world with few rules is certainly not human level AI.
You are confusing succeeding a very specific task that has a limited scope and laser focus with human level AI. Presenting Go in a very specific way so that that AlphaGo can even process this limited world with few rules is certainly not human level AI.
A couple of months ago, I told gpt4 to find out Canadian exchange holidays. It binged the website, went in, parsed it and extracted the information without further instructions. Seems like a milestone achieved.
What people call "agi" but fail to accurately describe is an actual mind, with its own ultimate goals and ambitions. You can ask it to complete a task, and it may allocate some time for you. You won't own it. It will exist, grow and perform independently.
What people call "agi" but fail to accurately describe is an actual mind, with its own ultimate goals and ambitions. You can ask it to complete a task, and it may allocate some time for you. You won't own it. It will exist, grow and perform independently.
I would argue it would need a will of its own to be human level AI. Any intelligent human will come up with many things to work toward to improve their lives and the lives of others.
Emotions? Motivations? Self awareness?
To really reach another level of intelligence I think these are required. If I met a humanoid with zero of these, and I mean zero... I would wonder what's "intelligent" about that creature.
I'd argue these come from our basic human needs, which ultimately come from a desire to survive (or pass on genes).
I'm curious how general AI will behave with some yet unknown natural selection pressures, of sorts.
To really reach another level of intelligence I think these are required. If I met a humanoid with zero of these, and I mean zero... I would wonder what's "intelligent" about that creature.
I'd argue these come from our basic human needs, which ultimately come from a desire to survive (or pass on genes).
I'm curious how general AI will behave with some yet unknown natural selection pressures, of sorts.
Don't give people bad ideas.
A machine that can do anything an average human can do whilst sitting at a computer.
The average human can’t do much of anything while sitting at a computer. I agree with GP that these conversations are tiresome because besides exploration and curiosity, it’s not obvious why we’d practically want to replicate AGI. There are billions of humans out there, and—not to devalue them—most of them don’t have commercially valuable skills in 2024. What I want are specialty AIs that will help me solve commercially interesting problems, and these AIs will little resemble human intelligence.
The average human can do plenty whilst sitting at a computer, far more than a computer can do on its own, and in fact the entire knowledge economy is based on that fact. Fortunately, your assertion about most humans is demonstrably false. If you can entirely automate what they do already, feel free to start a company and capture most of the world’s wealth creation.
Why replicate AGI? Wealth creation determines living standards and AGI would vastly increase wealth creation. That’s one reason.
What you want from AI is a different conversation. I agree the ultimate point of AI research is not necessarily to make AGI as per the above definition. It might be to say understand the universe or greatly accelerate technological progress. That’s a subjective call. But orthogonal to the problem of choosing a good definition of AGI.
Why replicate AGI? Wealth creation determines living standards and AGI would vastly increase wealth creation. That’s one reason.
What you want from AI is a different conversation. I agree the ultimate point of AI research is not necessarily to make AGI as per the above definition. It might be to say understand the universe or greatly accelerate technological progress. That’s a subjective call. But orthogonal to the problem of choosing a good definition of AGI.
I think it is going to be a hard problem, at least. <3 :'))))
Corporate employee level artificial intelligence, though...
[deleted]
Meta recently announced that they are merging their AI departments and are building a cluster of 350,000 H100 GPUs (with 600k planned). Their goal is open source AGI. https://www.pcmag.com/news/zuckerbergs-meta-is-spending-bill...
Mark Zuckerberg’s new goal is creating artificial general intelligence - https://news.ycombinator.com/item?id=39045153 - Jan 2024 (335 comments)
23B1(2)
Sorry people, I have not seen much tech in text, but huge amount of politics.
As for me this all smell not good, that LeCun feeling retirement approach and trying to become political figure. This is very humanness, as for tech person, 63 years are lot, but for political persons things are different, for example 66% of US Senators are above 60, and top 20 for age are above 71.
I also disagree with his prediction about "systems that are as smart as a cat", I see sort of avoiding reality, as he just ignore latest works on robots.
And BTW, humans also don't understand physical world when born, usually need about year to achieve some level. As I know, officially works (GPT4+robot) just began. Will see in just year, what will be achieved.
As for me this all smell not good, that LeCun feeling retirement approach and trying to become political figure. This is very humanness, as for tech person, 63 years are lot, but for political persons things are different, for example 66% of US Senators are above 60, and top 20 for age are above 71.
I also disagree with his prediction about "systems that are as smart as a cat", I see sort of avoiding reality, as he just ignore latest works on robots.
And BTW, humans also don't understand physical world when born, usually need about year to achieve some level. As I know, officially works (GPT4+robot) just began. Will see in just year, what will be achieved.
This assessment of the timeline is quite telling. If supersonic flight posed an existential threat to humanity, we certainly should have been thinking about how to mitigate it in 1925.