Ilya Sutskever, Yann LeCun and the End of “Just Add GPUs”(abzglobal.net)
abzglobal.net
Ilya Sutskever, Yann LeCun and the End of “Just Add GPUs”
https://www.abzglobal.net/web-development-blog/ilya-sutskever-yann-lecun-and-the-end-of-just-add-gpus
133 comments
Not to say you are wrong but how do you know it's AI generated rather than written by Sorca Marian? To me phrases like "Models look god-tier on paper" look more human than AI as a) "god-tier" never came up in the interview and b) it's brief and doesn't waffle on.
If it was AI-generated I had no difficulty with it, certainly on par with typical surface level journalist summaries, and vastly better than losing 2 hours of my life to watching some video interviews.. :) AI as we know it may not be real intelligence but it certainly has valid uses
I get a lot from seeing the person talk vs reading a summary. I have gone back and watched a lot of interviews and talks with Ilya. In hindsight, it is easy to hear the future ideas in his words at the time.
That said, I use AI summaries for a lot of stuff that I don't really care about. For me, this topic is important enough to spend two hours of my life on, soaking up every detail.
As for being on par with typical surface level journalism. I think we might be further into the dead internet than most people realize: https://en.wikipedia.org/wiki/Dead_Internet_theory
That said, I use AI summaries for a lot of stuff that I don't really care about. For me, this topic is important enough to spend two hours of my life on, soaking up every detail.
As for being on par with typical surface level journalism. I think we might be further into the dead internet than most people realize: https://en.wikipedia.org/wiki/Dead_Internet_theory
> I get a lot from seeing the person talk vs reading a summary
And some people simply have the opposite preference. There's lots of situations where sound is simply not an option. Some people are hearing impaired. Some people are visually impaired, and will definitely not get much from watching the person speak. Some ESL people have a hard time with spoken English. Even some native English speakers have a hard time with certain accents. Some people only have 5 minutes to spare instead of 50.
All of those problems are solved by the written word, which is why it hasn't gone away yet, even though we have amazing video capabilities.
You can have a preference without randomly label everything you don't like as AI slop.
And some people simply have the opposite preference. There's lots of situations where sound is simply not an option. Some people are hearing impaired. Some people are visually impaired, and will definitely not get much from watching the person speak. Some ESL people have a hard time with spoken English. Even some native English speakers have a hard time with certain accents. Some people only have 5 minutes to spare instead of 50.
All of those problems are solved by the written word, which is why it hasn't gone away yet, even though we have amazing video capabilities.
You can have a preference without randomly label everything you don't like as AI slop.
This may not be AGI, but I think LLMs as is, with no other innovation, are capably enough for gigantic labor replacement with the right scaffolding. Even humans need a lot of scaffolding at scale (e.g. sales reps use CRMs even though they are generally intelligent). LLMs solve a “fuzzy input” problem that traditional software struggles with. I’m guessing something like 80% of current white collar jobs can be automated with LLMs plus scaffolding.
> LLMs solve a “fuzzy input” problem that traditional software struggles with
This is basically all of it.
Kind of how word processors solved the writing is tedious struggle and search solved the "can't curate the internet" struggle.
This is basically all of it.
Kind of how word processors solved the writing is tedious struggle and search solved the "can't curate the internet" struggle.
What's a white collar job than can be automated with LLMs plus scaffolding?
My first job out of uni was in creating automated tests to validate some set top box. It involved using library of "blocks" to operate a remote control. Some of the people I have been working with spent their whole career in this narrow area, building those libraries of block and using them for customer and I have no doubts a LLM can today produce the same tests without any human intervention
Replacing labor doesn't require replacing whole jobs, it's enough to only replace specific tasks within those jobs which will reduce the number of workers needed for the same amount of work.
But then it becomes a competitive advantage for another firm to use the same employees to do more work, leading to the jobs not being replaced.
To pick a rather extreme example, the fraction of the population involved in farming is rather lower than in the past. Due to productivity improvements.
It's not clear why your analogy wouldn't have implied the end of white collar work when computers were first invented or when the internet was invented. Both of those should have been massive productivity boosts which meant the workers would have to go elsewhere to feed themselves. Instead Jevon's paradox kicks in every time.
Counterexample, not analogy.
To pick a rather extreme example, the cotton gin...
Well the question here if LLMs are the cotton gin or if they’re the combine/tractor thing that killed all the farming jobs.
I think it was the GPS, automation (robotics), bioengineered crops, and conglomerates. My point is, I'm pretty sure it's a lot of factors. Even in the cotton gin case. It's probably naïve to give so much credit to one thing
Most QA, most analyst positions, a good chunk of the kludge in intellectually challenging jobs, like medical diagnostics or software engineering, most administrative work, including in education and in healthcare, about 80% of customer success, about 80% of sales, are all within striking distance of automation with current-generation LLMs. And taht's entirely ignoring the 2nd-order effects in robotics and manufacturing.
You don't need LLM to replace QA. Just fire them, push some testing to developers and the rest to the users. Shareholders will be pleased by budget efficiency!
CEO.
Business Intelligence Engineers
You sound like a manager that doesn't understand what your employees are doing.
I see LLMs in a similar way - a new UI paradigm that "clicks the right buttons" when you know what you need, but don't know exact names of the buttons to click.
And from my experience there are lots and lots of jobs that are just "clicking the right buttons".
And from my experience there are lots and lots of jobs that are just "clicking the right buttons".
Not sure it'll really work like that. Company A finds its programers 2x as productive with LLMs and thinks they'll fire half but competitor B has similar effects and uses the 2x to make more features so A has to do that to keep up.
Such a decision merely tips the scale into a brittle structure territory. It introduces critical points of failure (funneling responsibility through fewer "nodes", stronger reliance on compute, electricity, internet, and more) and reduces adaptability (e.g. data bias, data cutoff dates, unaware of minute evolving human needs, etc).
Theyre not deterministic and real wirld work is risk adverse. People aew confusing sinusidal growth with singularity.
I agree, AI image recognition is so good already that it can tell what someone is doing or what is happening in a picture. Just have that run at 30 fps and make the robot's movements align with that understanding and bam, you effectively have "AGI" in some sense no? I mean sure, maybe it doesn't really remember anything like a human would and it doesn't learn on the fly yet but it's definitely some kind of intelligent, autonomous thing that will be able to respond to just about anything in the world. Making it able to learn on demand is something people are working on. Chatgpt remembers some stuff already too after all. It's very small and very spotty, weird memory but hey, it's something. As soon as that becomes a tiny bit better you'll already beat humans at memory.
Same here. Scale existing chips 100-1000x - there's plenty left to do. With that - we'll likely need 100x more power production too.
Try talking to white collar workers outside your bubble. Better yet get a job.
This opinion is not based in reality. The only way to understand that is to go outside and talk to real people who are neither techies nor managers, and, better yet, try to do their jobs better than they do.
All the frontier houses know this too. They also know it will be extremely difficult to raise more capital if their pitch is "we need to go back to research, which might return nothing at all."
Ilya did also acknowledge that these houses will still generate gobs of revenue, despite being at a dead end, so I'm not sure what the criticism is, exactly.
Everyone knows another breakthrough is required for agi to arrive; sama explicitly said this. Do you wait and sit on your hands until that breakthrough arrives? Or make a lot of money while skating to where the puck will be?
Ilya did also acknowledge that these houses will still generate gobs of revenue, despite being at a dead end, so I'm not sure what the criticism is, exactly.
Everyone knows another breakthrough is required for agi to arrive; sama explicitly said this. Do you wait and sit on your hands until that breakthrough arrives? Or make a lot of money while skating to where the puck will be?
I would say the issue is that most of the big AI players are burning a lot more cash than they earn, and the main thesis is that they are doing so because their product will be so huge that they will need 10x-100x infrastructure to support it.
But what we're seeing at the moment, is a deceleration, not an acceleration.
But what we're seeing at the moment, is a deceleration, not an acceleration.
There is no example of a leading company that ships a world-changing product, yet somehow runs out of cash.
Maybe they lose relevance. Maybe they miss the breakthrough. That becomes the reason. So perplexity? Sure. Anthropic, even? Yep. Google? OpenAI? Nah.
Regardless, viewing the unit economics, there are very clear sight lines to profitability if they want it. Just like with Amazon, Tesla, Apple, etc., when you want to grow, hoarding cash is a bad play.
Maybe they lose relevance. Maybe they miss the breakthrough. That becomes the reason. So perplexity? Sure. Anthropic, even? Yep. Google? OpenAI? Nah.
Regardless, viewing the unit economics, there are very clear sight lines to profitability if they want it. Just like with Amazon, Tesla, Apple, etc., when you want to grow, hoarding cash is a bad play.
>no example of a leading company that ships a world-changing product, yet somehow runs out of cash
Concorde, Lotus 1-2-3, Compuserve, AOL, Yahoo, Polaroid...
Concorde, Lotus 1-2-3, Compuserve, AOL, Yahoo, Polaroid...
OpenAI is very unlikely to go bankrupt, but they could be in such a difficult financial position that they would have to make painful compromises with Microsoft and/or Nvidia and lose most of their leverage.
Microsoft, largest software company in the world, has (publicly and privately) admitted that it is in their best interest to ensure that OpenAI remains the leading ai company for at least the next seven years.
As for nvidia, if OpenAI has less leverage, that necessitates a different ai company having more. Who would it be?
As for nvidia, if OpenAI has less leverage, that necessitates a different ai company having more. Who would it be?
Nvidia.
Believe it or not but they have their own robotics and AI plans, fully financed by their GPU division. They don’t intend to sell shovels forever.
Believe it or not but they have their own robotics and AI plans, fully financed by their GPU division. They don’t intend to sell shovels forever.
Revenue isn't profit
Countless examples of a lack of profit not spelling the doom of a company (uber, amazon, tesla).
Countless examples of companies that strive for profit too early, only to die
Countless examples of companies that strive for profit too early, only to die
A distinction that matters if the unit economics are bad, which nobody has full visibility over.
No matter the unit economics, commoditization and advancement of open models and small startups means that there is at almost a year or two to exploit competitive advantage. If scaling stops, the window to make a profit is extremely narrow.
Open models serve as a safeguard, not a competitor, to closed models.
They still cost billions to pre-train
They still cost billions to pre-train
Training costs will come down, at a tremendous pace.
They cost billions to train right now because people are willing to throw billions away to get to the goal first. Given more time, cheaper and more clever training methods will be found.
They cost billions to train right now because people are willing to throw billions away to get to the goal first. Given more time, cheaper and more clever training methods will be found.
Which is an issue with unit economics.
Google made $35 billion in profit last quarter.
Houses?
So we're rehypothecating CDOs like the last bubble?
So we're rehypothecating CDOs like the last bubble?
I think OP means "The Great Houses of AI"
Source?
Everyone at anthropic is saying ASI is imminent…
Everyone at anthropic is saying ASI is imminent…
People at Anthropic have a vested interest in getting you to believe that they are creating new and exciting things. Of course they say that a breakthrough is imminent. Doesn't make it in the least true, though.
[deleted]
Reminds me of this commercial [1]
[1] https://www.youtube.com/watch?v=MzakqMAaHME
[1] https://www.youtube.com/watch?v=MzakqMAaHME
>> Everyone at anthropic is saying ASI is imminent…
Who exactly is saying this, other than C-level people?
Who exactly is saying this, other than C-level people?
Say we discover a new architecture breakthrough like Yann LeCun's proposed JEPA. Won't scaling laws apply to it anyway?
Suppose training is so efficient that you can train state of the art AGI on a few GPUs. If it's better than current LLMs, there will be more demand/inference, which will require more GPUs and we are back at the same "add more gpus".
I find it hard to believe that we, as a humanity, will hit the wall of "we don't need more compute", no matter what the algorithms are.
Suppose training is so efficient that you can train state of the art AGI on a few GPUs. If it's better than current LLMs, there will be more demand/inference, which will require more GPUs and we are back at the same "add more gpus".
I find it hard to believe that we, as a humanity, will hit the wall of "we don't need more compute", no matter what the algorithms are.
> Won't scaling laws apply to it anyway?
The fight was never "Scale is all you need" (SIAYN) vs "scale is irrelevant" it was "SIAYN" vs "Scaling is not enough". I'm not aware of any halfway serious researcher that did not think scaling was going to result in massive improvements. Being a researcher from the SINE camp myself...
Here's the thing:
The SIAYN camp argued that the transformer architecture was essentially good enough. They didn't think scale was all you needed, but that the rest would me minor tweaks and increasing model size and data size would get us there. That those were the major hurdles. In this sense they argued that we should move our efforts away from research and into engineering. That AGI was now essentially a money problem rather than a research problem. They pointed to Sutton's Bitter Lesson narrowly, concentrating on his point about compute.
The SINE (or SINAYN) camp wasn't sold. We read the Bitter Lesson differently. That yes, compute is a key element to modern success, but just as important was the rise of our flexible algorithms. In the past we couldn't work with such algorithms because of lack of computational power, but that the real power was the algorithms. We're definitely a more diverse camp too, with vary arguments. Many of us look at animals and see that we can do so much more with so much less[2]. Clearly even if SIAYN were sufficient, it does not appear to be efficient. Regardless, we all agree that there's still subtle nuances in intelligence that need working out.
The characteristics of the scaling "laws" matter but it isn't enough. In the end what matters is generalization. For that we don't really have measures. Unfortunately, with the SIAYN camp also came benchmark maximization. It was a good strategy in the beginning as it helped give us direction. But we are now at the hard problem with the SINE camp predicted. How do you do things like make a model a good music generator when you have no definition of "good music"? Even in a very narrow sense we don't have a half way decent mathematical definition of any aesthetics. We argued "we should be trying to figure this out so we don't hit a wall" and they argued "it'll emerge with scale".
So now the cards have been dealt. Who has the winning hand? More importantly, which camp will we fund? And will we fund the SIAYN people that converted to SINE or will we fund those who have been SINE when times were tough?
[0] They've been power laws and I expect them to continue to be power laws[1]. But the parameters of those laws do still matter, right?
[1] https://www.youtube.com/watch?v=HBluLfX2F_k
[2] A mouse has on the order of 100M neurons (and 10^12 synapses). Not to mention how little power they operate on! These guys can still our perform LLMs on certain tasks despite the LLMs having like 4 orders of magnitude more parameters and many more in data!
I agree scaling alone is not enough, and transformers itself is a proof of that - it was an iteration on the attention mechanism and a few other changes.
But no matter what the next big thing is, I'm sure it would immediately fill all available compute to maximize its potential. It's not like intelligence has a ceiling beyond which you don't need more intelligence.
But no matter what the next big thing is, I'm sure it would immediately fill all available compute to maximize its potential. It's not like intelligence has a ceiling beyond which you don't need more intelligence.
Was "scale is all you need" actually a real thing said by a real person? Even the most pro scale people like Altman seem to be saying research and algorithms are a thing too. I guess as you say a more important thing is where the money goes. I think Altman's been overdoing it a bit on scaling spend.
Yes, they even made t-shirts.
Which Altman said that AGI is a scaling problem, which is why he was asking for $7T. But he was clearly a lier given this from last year. There's no way he really believed this in late 2024.
Just because we end up with success doesn't mean the path to get there was reasonable nor does it mean it was efficient.
[0] https://www.tomsguide.com/ai/chatgpt/sam-altman-claims-agi-i...
> Even the most pro scale people like Altman seem to be saying research and algorithms are a thing too.
Which Altman said that AGI is a scaling problem, which is why he was asking for $7T. But he was clearly a lier given this from last year. There's no way he really believed this in late 2024.
> Altman claimed that AGI could be achieved in 2025 during an interview for Y Combinator, declaring that it is now simply an engineering problem. He said things were moving faster than expected and that the path to AGI was "basically clear."[0]
Just because we end up with success doesn't mean the path to get there was reasonable nor does it mean it was efficient.
[0] https://www.tomsguide.com/ai/chatgpt/sam-altman-claims-agi-i...
Fair enough although that Altman quote doesn't match what he actually said in the interview. He said:
>...first time ever where I felt like we actually know what to do like I think from here to building an AGI will still take a huge amount of work there are some known unknowns but I think we basically know what to go what to go do and it'll take a while it'll be hard but that's tremendously exciting... https://youtu.be/xXCBz_8hM9w?t=2330
and at the end there was "what are you excited for in 2025?" and Altman says "AGI" but that doesn't specify if that's it arriving or just working on it.
I don't think huge amount of work and known unknowns is the same as we just need to scale.
>...first time ever where I felt like we actually know what to do like I think from here to building an AGI will still take a huge amount of work there are some known unknowns but I think we basically know what to go what to go do and it'll take a while it'll be hard but that's tremendously exciting... https://youtu.be/xXCBz_8hM9w?t=2330
and at the end there was "what are you excited for in 2025?" and Altman says "AGI" but that doesn't specify if that's it arriving or just working on it.
I don't think huge amount of work and known unknowns is the same as we just need to scale.
I disagree with the framing in 2.1 a lot.
I would not say that any model shows a "crazy" score on ARC-AGI.
I broadly have seen incremental improvements in benchmarks since 2020, mostly at a level I would believe to be below average human reasoning, but above average human knowledge. No one would call GPT-3 godlike and it is quite similar to modern models in benchmarks; it is not a difference of like 1% vs 90%. I think most people would consider gpt-3 to be closer to opus 4.5 than opus 4.5 is to a human.
> Models look god-tier on paper:
> they pass exams
> solve benchmark coding tasks
> reach crazy scores on reasoning evals
I would not say that any model shows a "crazy" score on ARC-AGI.
I broadly have seen incremental improvements in benchmarks since 2020, mostly at a level I would believe to be below average human reasoning, but above average human knowledge. No one would call GPT-3 godlike and it is quite similar to modern models in benchmarks; it is not a difference of like 1% vs 90%. I think most people would consider gpt-3 to be closer to opus 4.5 than opus 4.5 is to a human.
Roughly I'd agree, although I don't have hard numbers, and I'd say GPT-4 in 2023 vs GPT-3 as the last major "wow" release from a purely-model perspective. But they've also gotten a lot faster, which has its own value. And the tooling around them has gotten MASSIVELY better - remember the "prompt engineering" craze? Now there are a lot of tools out there that will take your two-sentence prompt and figure out - even asking you questions sometimes - how to best execute that based on local context like in a code repository, and iterate by "re-prompting" itself over and over. In a fraction of the time you could've done that by manual "prompt engineering."
Though I do not fully know where the boundary between "a model prompted to iterate and use tools" and "a model trained to be more iterative by design" is. How meaningful is that distinction?
But the people who don't get this are the less-technical/less-hands-on VPs, CEOs, etc, who are deciding on layoffs, upcoming headcount, "replace our customer service or engineering staffs with AI" things. A lot of those moves are going to look either really silly or really genius depending on exactly how "AGI-like" the plateau turns out to be. And that affects a LOT of people's jobs/livelihood, so it's good to see the hype machine start to slow down and get more realistic about the near-term future.
Though I do not fully know where the boundary between "a model prompted to iterate and use tools" and "a model trained to be more iterative by design" is. How meaningful is that distinction?
But the people who don't get this are the less-technical/less-hands-on VPs, CEOs, etc, who are deciding on layoffs, upcoming headcount, "replace our customer service or engineering staffs with AI" things. A lot of those moves are going to look either really silly or really genius depending on exactly how "AGI-like" the plateau turns out to be. And that affects a LOT of people's jobs/livelihood, so it's good to see the hype machine start to slow down and get more realistic about the near-term future.
> I'd say GPT-4 in 2023 vs GPT-3 as the last major "wow" release from a purely-model perspective. But they've also gotten a lot faster, which has its own value. And the tooling around them has gotten MASSIVELY better
Tooling vs model is a false dichotomy in this case. The massive improvements in tooling are directly traceable back to massive improvements in the models.
If you took the same tooling and scaffolding and stuck GPT-3 or even GPT-4 in it, they would fail miserably and from the outside the tooling would look abysmal, because all of the affordances of current tooling come directly from model capability.
All of the tooling approaches of modern systems were proposed and prototypes were made back in 2020 and 2021 with GPT-3. They just sucked because the models sucked.
The massive leap in tooling quality directly reflects a concomitant leap in model quality.
Tooling vs model is a false dichotomy in this case. The massive improvements in tooling are directly traceable back to massive improvements in the models.
If you took the same tooling and scaffolding and stuck GPT-3 or even GPT-4 in it, they would fail miserably and from the outside the tooling would look abysmal, because all of the affordances of current tooling come directly from model capability.
All of the tooling approaches of modern systems were proposed and prototypes were made back in 2020 and 2021 with GPT-3. They just sucked because the models sucked.
The massive leap in tooling quality directly reflects a concomitant leap in model quality.
I dunno, some of the questions on things like Humanity's Last Exam sure strike me as "godlike." Yes, I'm happy that I can still crush LLMs on ARC-AGI-2 but I see the writing on the wall there, too. Barely over a year ago LLMs were what, single digit percentages on ARC-AGI-1?
I would hope god can do better than 40% on a test. If you select experts from the relevant fields humans, they together would get a passing grade (70%) at least. A group of 20 humans is not godlike.
Modern ML builds on two pillars: GPUs and autodiff. Given that GPUs are running out of steam, I wonder what we should focus on now.
The price, power, and size. Make it cheap, low power, and small enough for mobile. One way to do this is inference in 4, 2, 1 bit. Also GPUs are parallel, most tasks can be split on several GPUs. Just by adding they you can scale up to infinity. In theory. So datacenters aren't going anywhere, they will still dominate.
Another way is CPU+ + fast memory, like Apple does. It's limited but power efficient.
Looks like with ecosystem development we need the whole spectrum from big models+tools running on datacenters to smaller running locally, to even smaller on mobile devices and robots.
Another way is CPU+ + fast memory, like Apple does. It's limited but power efficient.
Looks like with ecosystem development we need the whole spectrum from big models+tools running on datacenters to smaller running locally, to even smaller on mobile devices and robots.
I personally don’t think the scaling hypothesis is wrong, but it is running up against real limits
What high quality data sources are not already tapped?
Where does the next 1000x flops come from?
What high quality data sources are not already tapped?
Where does the next 1000x flops come from?
>What high quality data sources are not already tapped
Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic). Maybe the "AGI needs to be embodied" people will be right, because that's the only way to get enough coherent multimodal data to do things like long-range planning, navigation, game-playing, and visual tasks.
Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic). Maybe the "AGI needs to be embodied" people will be right, because that's the only way to get enough coherent multimodal data to do things like long-range planning, navigation, game-playing, and visual tasks.
> Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic).
Be careful with mistaking data for information.
You are getting a digital (maybe lossy compressed) samples of photons and sound waves. It is not unlimited, a camera pointed at a building at night is going to have very little new information from second to second. A microphone outside is going to have very little new information second to second unless something audible is happening close by.
You can max out your storage capacity by adding twenty ML high megapixel cameras recording frames as tiff file but gain little new useful information for every camera you add.
Be careful with mistaking data for information.
You are getting a digital (maybe lossy compressed) samples of photons and sound waves. It is not unlimited, a camera pointed at a building at night is going to have very little new information from second to second. A microphone outside is going to have very little new information second to second unless something audible is happening close by.
You can max out your storage capacity by adding twenty ML high megapixel cameras recording frames as tiff file but gain little new useful information for every camera you add.
Also self-driving cars which are hoovering this data already. Both alphabet and grok have an unusual advantage with those data sources.
This is where it is a bit confusing for people not familiar with the state of the art.
Some people don't seem to realize how critical the "eval" function is for machine learning.
Raw data is not much more useful than noise for the current recipes of model training.
Human produced data on the internet (text, images, etc.) is highly structured and the eval function can easily be built.
Chess or Go has rules and the eval function is more or less derived or discovered from them.
But the real world?
For driving you can more or less build a computer vision system able to follow a road in a week, because the eval function is so simple. But for all the complex parts, the eval function is basically one bit (you crashed/not crashed) that you have to sip very slowly, and it very inefficient to train such a complex system with such a minimal reward even in simulations.
Some people don't seem to realize how critical the "eval" function is for machine learning.
Raw data is not much more useful than noise for the current recipes of model training.
Human produced data on the internet (text, images, etc.) is highly structured and the eval function can easily be built.
Chess or Go has rules and the eval function is more or less derived or discovered from them.
But the real world?
For driving you can more or less build a computer vision system able to follow a road in a week, because the eval function is so simple. But for all the complex parts, the eval function is basically one bit (you crashed/not crashed) that you have to sip very slowly, and it very inefficient to train such a complex system with such a minimal reward even in simulations.
The real world is governed by physics, so isn't "next state prediction" a sufficient eval function that forces it to internalize a world model? And increasing the timespan over which you predict requires an increasing amount of "intelligence" because it requires modeling the real-world behavior of constituent subsystems that are often black-boxes (e.g. if there is a crow on a road with a car approaching, you can't just treat it as a physics simulation, you need to know that crows are capable of flying to understand what is going to happen).
I don't see how this is any less structured than the CLM objective of LLMs, there's a bunch of rich information there.
I don't see how this is any less structured than the CLM objective of LLMs, there's a bunch of rich information there.
If Transformers were able to learn in such a trivial fashion, we would already have AGI.
There is at least one missing piece to the puzzle, and some say 5-6 more breakthrough are necessary.
There is at least one missing piece to the puzzle, and some say 5-6 more breakthrough are necessary.
> But for all the complex parts, the eval function is basically one bit (you crashed/not crashed)
It is not like crashed/not crashed is the only possible eval function.
It can be easily much more nuanced than that. The driving system should be able to predict how everyone will move next is a good sub-goal. Checking if you were in the positon of an other driver, seeing what they see would our code be driving the same way as them is also a good sub goal. (Obviously total alignment here is neither possible nor is it desireable.)
Other evaluation is to check if you forced anyone to change speed/swerve to avoid you. And then you can have synthetic scenairos for every time you approached a lane which had priority over you. You can add conflicting vehicles approaching (with different timings and speeds) and see if own vehicle notices and handles them correctly. (And “handles them correctly” is not a binary crashed/not crashed either, you can check if the vehicle inconvenienced the simulated vehicle.)
It is not like crashed/not crashed is the only possible eval function.
It can be easily much more nuanced than that. The driving system should be able to predict how everyone will move next is a good sub-goal. Checking if you were in the positon of an other driver, seeing what they see would our code be driving the same way as them is also a good sub goal. (Obviously total alignment here is neither possible nor is it desireable.)
Other evaluation is to check if you forced anyone to change speed/swerve to avoid you. And then you can have synthetic scenairos for every time you approached a lane which had priority over you. You can add conflicting vehicles approaching (with different timings and speeds) and see if own vehicle notices and handles them correctly. (And “handles them correctly” is not a binary crashed/not crashed either, you can check if the vehicle inconvenienced the simulated vehicle.)
> What high quality data sources are not already tapped?
Synthetic data? Video?
> Where does the next 1000x flops come from? Even with Moore's law dead, we can easily build 1,000x more computers. And for arguments about lack of power - we have sun.
> Where does the next 1000x flops come from? Even with Moore's law dead, we can easily build 1,000x more computers. And for arguments about lack of power - we have sun.
A Dyson sphere brain ?
I don't think we need that for 1,000x. We can building more solar, nuclear and there is still room for at least 10x improvement in efficiency for the chips. We are far far away from maxing out our compute capability as civilization before we start shooting satellites into the sun.
NSA datasets. Did your eye catch on the “genesis” project?
I thought about the huge pile of hard drives in Utah this morning. The TLAs in the USA have a metric shit ton of data that _should_ not be used but _could_ be used.
Even still, we need evolutions in model architecture to get to the next level. Data is not enough.
Even still, we need evolutions in model architecture to get to the next level. Data is not enough.
They are going to be used.
A lot of what's on that pile of hard drives is ciphertext waiting for cryptographically relevant quantum computing to arrive.
LLMs can't do jack shit with ciphertext (sans key).
LLMs can't do jack shit with ciphertext (sans key).
Dropbox, O365, Google workspace, AWS s3.
While I think there's obvious merit to their skepticism over the race towards agi, Sutskever's goal doesn't seem practical to me. As Dwarkesh also said, we reach to a safe and eventually perfect system by deploying it in public and iterating over it until optimal convergence dictated by users in a free market. Hence, I trust that Google, OpenAI or Anthropic will reach there, not SSI.
> we reach to a safe and eventually perfect system by deploying it in public and iterating over it until optimal convergence dictated by users in a free market
Possibly... but also a lot of the foundational AI advancements were actually done in skunkworks-like environments and with pure research rather than iterating in front of the public.
It's not 100% clear to me if the ultimate path to the end is iteration or something completely new.
Possibly... but also a lot of the foundational AI advancements were actually done in skunkworks-like environments and with pure research rather than iterating in front of the public.
It's not 100% clear to me if the ultimate path to the end is iteration or something completely new.
I can see where Sutskever is coming from.
We are in a situation where the hardware is probably sufficient for AI to do as well as humans, but in terms of thinking things over, coming to understand the world and developing original insights about it, LLMs aren't good, probably due to the algorithm.
To get something good at thinking and understanding you may be better rebuilding the basic algorithm rather than tinkering with LLMs to meet customer demands.
I mean the basic LLM thing of have an array of a few billion parameters, feed in all the text on the internet using matrix multiplication to adjust the parameters, use it to predict more text and expect the thing to be smart is a bit of a bodge. It's surprising it works as well as it does.
We are in a situation where the hardware is probably sufficient for AI to do as well as humans, but in terms of thinking things over, coming to understand the world and developing original insights about it, LLMs aren't good, probably due to the algorithm.
To get something good at thinking and understanding you may be better rebuilding the basic algorithm rather than tinkering with LLMs to meet customer demands.
I mean the basic LLM thing of have an array of a few billion parameters, feed in all the text on the internet using matrix multiplication to adjust the parameters, use it to predict more text and expect the thing to be smart is a bit of a bodge. It's surprising it works as well as it does.
[dupe]
Earlier:
Ilya Sutskever: We're moving from the age of scaling to the age of research
https://news.ycombinator.com/item?id=46048125
And one of the recent LeCun discussions:
https://news.ycombinator.com/item?id=45897271
Earlier:
Ilya Sutskever: We're moving from the age of scaling to the age of research
https://news.ycombinator.com/item?id=46048125
And one of the recent LeCun discussions:
https://news.ycombinator.com/item?id=45897271
This caught my eye.
> The industry is already operating at insane scale.
Sounds a lot like "640K ought to be enough for anybody", or "the market can stay irrational longer than you can stay solvent".
I don't doubt this person knows how things should go but I also don't doubt this will get bigger before it gets smaller.
> The industry is already operating at insane scale.
Sounds a lot like "640K ought to be enough for anybody", or "the market can stay irrational longer than you can stay solvent".
I don't doubt this person knows how things should go but I also don't doubt this will get bigger before it gets smaller.
[deleted]
Didn’t we just see big pretraining gains from Google and likely Anthropic?
I like Dario’s view on this, we’ve seen this story before with deep learning. Then we progressively got better regularization, initialization, and activations.
I’m sure this will follow the same suit, the graph of improvement is still linear up and to the right
I like Dario’s view on this, we’ve seen this story before with deep learning. Then we progressively got better regularization, initialization, and activations.
I’m sure this will follow the same suit, the graph of improvement is still linear up and to the right
The gains were on benchmarks. Ilya describes why this is a red herring here: https://youtu.be/aR20FWCCjAs?t=286
Gemini 3 is a huge jump. I can't imagine how anyone who uses the models all the time wouldn't feel this.
What does it do that Opus doesn't do?
I like Ilya's points but its also clearly progress, and we can't just write it off because we like another narrative
I thought Ilya said we have more companies than we have ideas. He also noted that our current are resulting in models which are very good at benchmarks but have some problems with generalization (and gave a theory as to why).
But I don't recall him actually saying that the current ideas won't lead to AGI.
But I don't recall him actually saying that the current ideas won't lead to AGI.
He says the current models generalize dramatically worse than people: https://youtu.be/aR20FWCCjAs?t=1501
Then, he starts to talk about the other ideas but his lawyers / investors prevent him from going into detail: https://youtu.be/aR20FWCCjAs?t=1939
The worrisome thing is that he openly talks about whether to release AGI to the public. So, there could be a world in which some superpower has access to wildly different tech than the public.
To take Hinton's analogy of AGI to extraterrestrial intelligence, this would be akin to a government having made contact but withholding the discovery and the technology from the public: https://youtu.be/e1Hf-o1SzL4?t=30
It's a wild time to be alive.
Then, he starts to talk about the other ideas but his lawyers / investors prevent him from going into detail: https://youtu.be/aR20FWCCjAs?t=1939
The worrisome thing is that he openly talks about whether to release AGI to the public. So, there could be a world in which some superpower has access to wildly different tech than the public.
To take Hinton's analogy of AGI to extraterrestrial intelligence, this would be akin to a government having made contact but withholding the discovery and the technology from the public: https://youtu.be/e1Hf-o1SzL4?t=30
It's a wild time to be alive.
It’s also weird to think that if there is extraterrestrial contact, it will most definitely happen in the specific land mass known as the United States and only the US government will be collecting said technology and hiding it. Out of the entire planet, contact is possible only in the USA.
I'm not sure if you're jabbing at the concept of American supremacy, or Hinton's idea, or my position. I don't live in the USA right now, but I am happy to participate in conversation. That's why I am here.
Can you unpack your ideas a bit more?
Can you unpack your ideas a bit more?
The current idea is keep doing more of the same & expect different results.
Some have been saying this for years now, but the consensus in the AI community and SV has been visibly shifting in the recent months.
Social contagion is astonishingly potent around ideas like this one, and this probably explains why the zeitgeist seems to be immovable for a time and then suddenly change.
Social contagion is astonishingly potent around ideas like this one, and this probably explains why the zeitgeist seems to be immovable for a time and then suddenly change.
People have been saying this before chatgpt and ever since. And they're right.
Its charlatans like sama that muddy the waters by promising the sky to get money for their empire building.
LLMs can make and are great great products. But its sneaky salesmen that are the ones saying scaling is the path to AGI. The reality is that they're just aiming for economies of scale to make their business viable
Its charlatans like sama that muddy the waters by promising the sky to get money for their empire building.
LLMs can make and are great great products. But its sneaky salesmen that are the ones saying scaling is the path to AGI. The reality is that they're just aiming for economies of scale to make their business viable
Ilya has zero damn idea about what he's doing, lol.
This does not mean he's not an accomplished and very talented researcher.
LeCun was sacked from Meta.
Not sure if it's wise to listen to their advice ...
This does not mean he's not an accomplished and very talented researcher.
LeCun was sacked from Meta.
Not sure if it's wise to listen to their advice ...
Not even close, I haven't heard a good argument from either of them. They should read the bitter lesson again.
Maybe you should read the bitter lesson again. Their current rhetoric is a direct extension of that perspective.
How do? Isn’t the bitter lesson about more search and compute? As opposed to clever algorithms
The subtext especially around shoving more effort into foreseeable dead-ends is the apt aspect here.
We can't possibly know they are dead ends
I'm really annoyed by this. Not because I think Ilya is wrong, but because I think he is right. Because for years I think his current statement is right but for the same years I think his previous statement was wrong.
My stance hasn't changed, his has.
There's a big problem in that we reward those who hype, not merit. When the "era of scaling" happened there was a split. Those that claimed "Scale is all you need" and those that claimed "Scale is not enough". The former won, and I even seem to remember a bunch of people with T-shirts at NeurIPS with "scale is all you need" around that time.
So then, why are we again rewarding those same people when they change tunes? Their bet lost, sorry. I'm happy we tried scale and I'm glad we made progress, but at the same time many of us have been working outside the SIAYN paradigm and we struggled to get papers through review[0]. Scaling efforts led to lots of publications and citations, but you got far less by working outside that domain. And FFS, the reason most of you know Gary Marcus is because he was a vocal opposition to SIAYN and had enough initial clout. So as this tune is changing does the money shift towards us? Of course not.
I don't care about being vindicated, I care about trying to do research[1]. I don't care about the money, I care about trying to make AGI. Even Sutton has said that the Bitter Lesson was not about SIAYN!
So why I'm annoyed is that it seems we're going to let those who made big claims and fell short rather than those who correctly predicted the result. Why do we reward those who chase hype more than we reward those who got it right?
[0] a common criticism being "but does it work at scale?" or "needs more experiments". While these critiques/questions are legitimate they are out of place. Let us publish the small scale results first so that we can evidence our requests for more scale. Do you expect us to start at large scale first?
[1] I'm anonymous here, I don't care about the internet points. For the sake of this comment I might as well be any one of those Debbie Downers who pushed back against SIAYN and talked about the limits and how we shouldn't put all our eggs in one basket. There's thousands of us
My stance hasn't changed, his has.
There's a big problem in that we reward those who hype, not merit. When the "era of scaling" happened there was a split. Those that claimed "Scale is all you need" and those that claimed "Scale is not enough". The former won, and I even seem to remember a bunch of people with T-shirts at NeurIPS with "scale is all you need" around that time.
So then, why are we again rewarding those same people when they change tunes? Their bet lost, sorry. I'm happy we tried scale and I'm glad we made progress, but at the same time many of us have been working outside the SIAYN paradigm and we struggled to get papers through review[0]. Scaling efforts led to lots of publications and citations, but you got far less by working outside that domain. And FFS, the reason most of you know Gary Marcus is because he was a vocal opposition to SIAYN and had enough initial clout. So as this tune is changing does the money shift towards us? Of course not.
I don't care about being vindicated, I care about trying to do research[1]. I don't care about the money, I care about trying to make AGI. Even Sutton has said that the Bitter Lesson was not about SIAYN!
So why I'm annoyed is that it seems we're going to let those who made big claims and fell short rather than those who correctly predicted the result. Why do we reward those who chase hype more than we reward those who got it right?
[0] a common criticism being "but does it work at scale?" or "needs more experiments". While these critiques/questions are legitimate they are out of place. Let us publish the small scale results first so that we can evidence our requests for more scale. Do you expect us to start at large scale first?
[1] I'm anonymous here, I don't care about the internet points. For the sake of this comment I might as well be any one of those Debbie Downers who pushed back against SIAYN and talked about the limits and how we shouldn't put all our eggs in one basket. There's thousands of us
I think you are getting caught in marketing semantics. The scale is all you need movement was mostly a way to funnel money to LLMs. Did Ilya ever actually believe transformers with no other improvements would lead to AGI, or did he believe that it would lead to a much more useful AI and wanted to raise money for that but found it hard without claiming it would lead to AGI? At the end of the day it is probably a good thing that so much money went into scaling recently, because it did work, as long as your measure of success is more nuanced than “did it lead to AGI”. And even then it may lead to AGI as the amount of money spent on ai research is much higher now and that new money may be what is actually needed.
My issue isn't that they are wrong, my issue is with rewarding those who are wrong. But your argument is it's fine to reward those who lie? I'm not sure how this isn't worse.
I'm sure that you're right that many people used it as a vehicle rather than being just true believers (I know some people that do), but there were also a lot of true believers.
The movement also stopped a lot of research. It has also resulted in a lot of money being dumped into companies betting on it being true. If we are in fact in a bubble (and it looks this way) then all that damage is on the hands of the SIAYN crowd.
Not being a true believer makes it better, it makes it worse. A lie is far worse than being wrong. Being wrong isn't a big issue, especially in the world of research. But lying is a major issue. It ruins it for everyone. We don't have to do this cycle of boom and bust to get things done. That's literally destructive
I'm sure that you're right that many people used it as a vehicle rather than being just true believers (I know some people that do), but there were also a lot of true believers.
The movement also stopped a lot of research. It has also resulted in a lot of money being dumped into companies betting on it being true. If we are in fact in a bubble (and it looks this way) then all that damage is on the hands of the SIAYN crowd.
Not being a true believer makes it better, it makes it worse. A lie is far worse than being wrong. Being wrong isn't a big issue, especially in the world of research. But lying is a major issue. It ruins it for everyone. We don't have to do this cycle of boom and bust to get things done. That's literally destructive
The issue as I see it is that google invented transformers way before they were released publicly. Clearly there were not enough resources being spent on them which is why the scale movement came about. Would google still be hoarding transformer based LLMs today without Ilya’s hype? Seems like a real possibility to me.
I'm not sure how you get there. The SIAYN movement happened after the AIAYN paper. The latter influenced the name of the former.
So what do you mean by secret? It was published in a paper. That's public
So what do you mean by secret? It was published in a paper. That's public
No one was releasing a product based on the paper though. Ilya had to go and raise a bunch of money for that to happen. Maybe I’m just more cynical and accepting of lying as the way things are done compared to you.
A problem is that the bulk of the people behind these labs are people that were conditioned from an early age to achieve high scores in standardized tests and conflate that with intelligence. Then apply that mentality to their models resulting in these leaderboards that nobody cares about.
Absolutely this, models acheive very highly on kind problems; ones that you can master with sufficient practice. Which is just remarkable, but the world is a wicked learning environment, and repetition is not enough.
The frenzy around AI is to do with growth fueled cocaine capitalism seeking 'more' where rational minds can see that we don't have that much more runway left with our current mode of operation.
While the tech is useful, the mass amounts of money being shoveled into AI has more to do with the ever escaping mirage of a promised land where there will be an infinite amount of 'more'. For some people that means post scarcity, for others it means a world dominating AGI that achieves escape velocity against the current gridlock of geopolitics, for still others it means ejecting the pesky labour class and replacing all labour needs with AI and robots. Varied needs, but all perceived as urgent and inescapable by their vested interests.
I am somewhat relieved that we're not headed into the singularity just yet, I see it as way too risky given the current balance of power and stability across the planet. The outcome of ever accelerating tech progress at the expense of all other dimensions of wellbeing is not good for the majority of life here.
While the tech is useful, the mass amounts of money being shoveled into AI has more to do with the ever escaping mirage of a promised land where there will be an infinite amount of 'more'. For some people that means post scarcity, for others it means a world dominating AGI that achieves escape velocity against the current gridlock of geopolitics, for still others it means ejecting the pesky labour class and replacing all labour needs with AI and robots. Varied needs, but all perceived as urgent and inescapable by their vested interests.
I am somewhat relieved that we're not headed into the singularity just yet, I see it as way too risky given the current balance of power and stability across the planet. The outcome of ever accelerating tech progress at the expense of all other dimensions of wellbeing is not good for the majority of life here.
> The frenzy around AI is to do with growth fueled cocaine capitalism seeking 'more' where rational minds can see that we don't have that much more runway left with our current mode of operation.
When talking with non-tech people around me, it’s really not about “rational minds”, it’s that people really don’t understand how all this works and as such don’t see the limitations of it.
Combine that with a whole lot of FOMO which happens often with investors and you have a whole pile of money being invested.
From what I hear, most companies like Google and Meta have a lot of money to burn, and their official position towards investors is “chances of reaching ASI/AGI are very low, but if we do and we miss out on it, it will mean a huge opportunity loss so it’s worth the investment right now”.
When talking with non-tech people around me, it’s really not about “rational minds”, it’s that people really don’t understand how all this works and as such don’t see the limitations of it.
Combine that with a whole lot of FOMO which happens often with investors and you have a whole pile of money being invested.
From what I hear, most companies like Google and Meta have a lot of money to burn, and their official position towards investors is “chances of reaching ASI/AGI are very low, but if we do and we miss out on it, it will mean a huge opportunity loss so it’s worth the investment right now”.
> When talking with non-tech people around me, it’s really not about “rational minds”, it’s that people really don’t understand how all this works and as such don’t see the limitations of it.
What are the limits? We know the limits for naked LLMs. Less so for LLM + current tools. Even less for LLM + future tools. And can only guess about LLM + other models + future tools. I mean moving forward likely requires complexity, research and engineering. We don't know the limits of this approach even without any major breakthrough. Can't predict, but if breakthrough happens it all will be different, but better than (we can foresee) today.
What are the limits? We know the limits for naked LLMs. Less so for LLM + current tools. Even less for LLM + future tools. And can only guess about LLM + other models + future tools. I mean moving forward likely requires complexity, research and engineering. We don't know the limits of this approach even without any major breakthrough. Can't predict, but if breakthrough happens it all will be different, but better than (we can foresee) today.
We dont have enough gpus.
so the top dogs state the obvious, again?
every LLM easily misaligned, "deceived to deceive" and whatnot and they want to focus on adding MORE ATTACK SURFACE???
and throw more CPU at it?
This is glorious.
time to invest in the pen & paper industry!
every LLM easily misaligned, "deceived to deceive" and whatnot and they want to focus on adding MORE ATTACK SURFACE???
and throw more CPU at it?
This is glorious.
time to invest in the pen & paper industry!
Why is Yann Lecun in same article as Ilya?
I can't tell if you meant this as a slight on Ilya Sutskever or on Yann LeCun. Both are well-known names in AI.
Pretty sure the article is AI slop, so it's kind of connect the dots
Ilya has appeared to shift to closer to Yann's position, though: he's been on the "scaling LLMs will fail to reach AGI" beat for a long time.
Ilya has appeared to shift to closer to Yann's position, though: he's been on the "scaling LLMs will fail to reach AGI" beat for a long time.
,> Pretty sure the article is AI slop
Yeah, the actual video with transcripts (YouTube link in bottom of TFA):
https://www.dwarkesh.com/p/ilya-sutskever-2
Ed: TFA is basically a dupe of
https://news.ycombinator.com/item?id=46048125
Yeah, the actual video with transcripts (YouTube link in bottom of TFA):
https://www.dwarkesh.com/p/ilya-sutskever-2
Ed: TFA is basically a dupe of
https://news.ycombinator.com/item?id=46048125
well, take a stab at it
Watch the original Sutskever interview: https://www.youtube.com/watch?v=aR20FWCCjAs
And LeCun: https://www.youtube.com/watch?v=4__gg83s_Do