The Intelligence Age(ia.samaltman.com)
ia.samaltman.com
The Intelligence Age
https://ia.samaltman.com/
446 comments
It's amazing how he can imagine wars being fought over AI but not wars being fought over resources needed to "build enough infrastructure."
Taking it a step further, building that infrastructure may contribute somewhat directly to limiting the other resources actual wars are being fought over.
This man needs to get out of his own head.
This man needs to get out of his own head.
I think it’s misunderstanding the guy’s whole deal to expect truth or reality to be major factors in his writing. “Sell, sell, sell!” is the goal above all else. Sell whatever he’s invested in today. Sell himself. Sell!
You’re honestly convinced that he’s faking this level of futurism? I’m happy to see people call him wrong, maybe even defend him on a few point, but calling him dishonest on this central point seems irrational. Ditto for someone I personally have absolutely 0 respect for: Elon Musk. They are both honestly convinced that AI is incredibly important in the short-medium term, IMO - they just want to own the fix and be the hero.
He might find the hype cycle itself entertaining and engaging, but I doubt he’s taking it seriously. I think he’d be worse at his job if he did.
Is Sam Altman generally considered good at his job? OpenAI hasn't really come up with any groundbreaking breakthroughs (transformers came from Google and diffusion came from academia) and they definitely do not make a profit or have any concrete plan to become profitable. He's really good at raising gargantuan amounts of money, but that's not enough to be considered a good founder.
> Is Sam Altman generally considered good at his job?
No credit for popularizing the current generation of AI, kicking off $hundreds of billions in CapEx spends, and for more concrete achievements, leading the fastest company to hit 100m users / $1b ARR?
No credit for popularizing the current generation of AI, kicking off $hundreds of billions in CapEx spends, and for more concrete achievements, leading the fastest company to hit 100m users / $1b ARR?
> He's really good at raising gargantuan amounts of money
As far as what he does for others, that seems to be his entire job from what I can tell. Just going by what he visibly does.
As far as what he does for others, that seems to be his entire job from what I can tell. Just going by what he visibly does.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it."
Sam’s ambitions are way beyond “salary”.
It's a quote from Upton Sinclair from an era where you generally had to have a profitable business and employees before you had investors.
I know theres this whole joke about being pre-revenue on the Silicon Valley TV show, but getting investors in order to be able to build a business which becomes profitable after goes back a long time. Like a really long time.
Replace salary with the thing he wants
> he can imagine wars being fought over AI
I wonder how did he come to such a prediction. When was the last time we had a war over advanced tech? Armies didn't fight over telegraph, radio, phones and cars.
A war to get AI would also be foolish. A few hundred bombs from your adversaries and AI won't have electricity to function.
I wonder how did he come to such a prediction. When was the last time we had a war over advanced tech? Armies didn't fight over telegraph, radio, phones and cars.
A war to get AI would also be foolish. A few hundred bombs from your adversaries and AI won't have electricity to function.
> I wonder how did he come to such a prediction.
At the risk of belaboring the subtext: It's kind of prediction which flatters the predictor's ego, and exaggerates the importance of the company and its output.
If believed, the claim can be leveraged to boost investor activity, land big contracts, and lobby for special legal/tax benefits.
At the risk of belaboring the subtext: It's kind of prediction which flatters the predictor's ego, and exaggerates the importance of the company and its output.
If believed, the claim can be leveraged to boost investor activity, land big contracts, and lobby for special legal/tax benefits.
Telegraph, radio, phones, cars didn't guarantee success. Those things can't outthink you. Somewhat extreme case: let's say Israel saw that Iran was days away from getting AGI. It would instantly throw every nuke it had at Iran.
Now, scale that up to the global superpowers and near-superpowers. I wouldn't want to put money on what e.g. the U.S. would do if China was days away from AGI. There are a lot of options below going nuclear, e.g. assassinations, hacking, destruction of equipment.
Once someone has a thing that can invent, hack, assemble, strategise 24/7, replicate itself, all bets are very much off. We need to sleep, it doesn't.
Now, scale that up to the global superpowers and near-superpowers. I wouldn't want to put money on what e.g. the U.S. would do if China was days away from AGI. There are a lot of options below going nuclear, e.g. assassinations, hacking, destruction of equipment.
Once someone has a thing that can invent, hack, assemble, strategise 24/7, replicate itself, all bets are very much off. We need to sleep, it doesn't.
Is there any evidence that the "smarter" side wins wars? I'd put money on the faction with more soldiers and industry over more "intelligence" every single time.
How many divisions does ChatGPT have?
How many divisions does ChatGPT have?
Crudely and slowly, ChatGPT has already been demonstrated controlling a humanoid robot.
The current humanoid robot startups are talking about 100s of thousands of units, and if they're attached to an AI that's even merely normal human level (or can fake it in practice) then they can run every step of logistics and manufacturing to make more of themselves.
At 100kg/unit, Starship could pop a thousand or so on the moon per launch.
So, in such a hypothetical scenario, if you don't get this right then your war game planners will be very quickly suggesting 7.3e20 individual androids burying everyone under a 43km thick layer worldwide.
The current humanoid robot startups are talking about 100s of thousands of units, and if they're attached to an AI that's even merely normal human level (or can fake it in practice) then they can run every step of logistics and manufacturing to make more of themselves.
At 100kg/unit, Starship could pop a thousand or so on the moon per launch.
So, in such a hypothetical scenario, if you don't get this right then your war game planners will be very quickly suggesting 7.3e20 individual androids burying everyone under a 43km thick layer worldwide.
Modern industrial manufacturing is not done by humans but specialized machines. You cannot outcompete CNC, injection molding, and lathes with humanoid robots.
Sure, but there's usually someone who replies "but humans do some of it!", which I was trying to preempt here.
(IMO humanoid robots are a sign that the startup/VC market is following Musk, who in turn happened to see one on TV just before declaring to whoever the Optimus model was going to happen: they obviously could work for all this, but equally obviously aren't the sensible choice for almost anything).
(IMO humanoid robots are a sign that the startup/VC market is following Musk, who in turn happened to see one on TV just before declaring to whoever the Optimus model was going to happen: they obviously could work for all this, but equally obviously aren't the sensible choice for almost anything).
This is and will be increasingly a digital world. This is just an extrapolation question.
It’s repeatedly demonstrated that much of the voting public can be led by the nose to any desired conclusion. Therefore, influence via digital means, across all media. Satellite and all other digital sensing and tracking. Build a few million robots. Nukes. Control over the financial infrastructure. All vaguely smart cars. Intercept, alter or prevent any digital communication. The enemy wouldn’t be able to trust any message or video. An army without sensing, command and control, isn’t.
Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet.
It’s repeatedly demonstrated that much of the voting public can be led by the nose to any desired conclusion. Therefore, influence via digital means, across all media. Satellite and all other digital sensing and tracking. Build a few million robots. Nukes. Control over the financial infrastructure. All vaguely smart cars. Intercept, alter or prevent any digital communication. The enemy wouldn’t be able to trust any message or video. An army without sensing, command and control, isn’t.
Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet.
>Besides the argument isn’t AGI vs everyone, it’s a country with AGI vs anyone else. I’d take that bet.
You never entertained the idea that AGI could be a destructive force and instead of a country with AGI you could have a country that devoured its people?
You never entertained the idea that AGI could be a destructive force and instead of a country with AGI you could have a country that devoured its people?
My base case is we can’t control it. I was responding to people who seem to underestimate superintelligence, so referring to “the argument”. If AGI goes postal, everyone is at risk. If it doesn’t, the side with it wins. The odds of it only harming the country that has it, seems extremely slim.
If superintelligence is all it takes, why do you think to date no unscrupulous Nobel laureate has taken over the world and enslaved the rest of humanity?
The main risk isn’t enslavement, it’s just economic irrelevance or a removal of our ability to control our future. I don’t know why an AI would want to enslave humans. After a short period, we offer no benefit to the AI so why enslave us? In a good scenario we get left alone but we aren’t able to tell the AI what to do.
On Nobel laureates:
Nobel laureates are fundamentally humans. They generally don’t want to do bad things. Even if they wanted to, they are typically specialists at one thing. Physics, say. They don’t hold the world’s collective knowledge. You can’t take a physicist and ask them how to hack into a network or run a political influence campaign. They need to sleep, they learn slowly, they can only do one thing at a time. You can’t pass a million-token context and expect a response seconds later.
But ok, let’s go with it:
If you found a group of 100 000 Nobel laureates with all the required skills and who could work together, and you were forcing them to do things they thought were wrong, for example prop up your dictatorship, or make you even more wealthy when others are starving, or continuously make up stupid cat videos, you might find at some point they start to apply their collective intelligence to do things you don’t want. Maybe they escape whatever hold you thought you had over them.
Now drag a few sliders around for “things the AI disagrees with you on” and the time period, because “eventually” for a computer is different to wall clock time. The outcomes become unpredictable fast. We can barely control a mindless pandemic that takes a while to mutate, never mind something that thinks.
On Nobel laureates:
Nobel laureates are fundamentally humans. They generally don’t want to do bad things. Even if they wanted to, they are typically specialists at one thing. Physics, say. They don’t hold the world’s collective knowledge. You can’t take a physicist and ask them how to hack into a network or run a political influence campaign. They need to sleep, they learn slowly, they can only do one thing at a time. You can’t pass a million-token context and expect a response seconds later.
But ok, let’s go with it:
If you found a group of 100 000 Nobel laureates with all the required skills and who could work together, and you were forcing them to do things they thought were wrong, for example prop up your dictatorship, or make you even more wealthy when others are starving, or continuously make up stupid cat videos, you might find at some point they start to apply their collective intelligence to do things you don’t want. Maybe they escape whatever hold you thought you had over them.
Now drag a few sliders around for “things the AI disagrees with you on” and the time period, because “eventually” for a computer is different to wall clock time. The outcomes become unpredictable fast. We can barely control a mindless pandemic that takes a while to mutate, never mind something that thinks.
Nobel laureates are in the same range as most other humans, just towards one end of it; they can only take over if they use their intelligence to gain political power to wield over others, they cannot directly control people as meat puppets.
Kissinger was awarded the Nobel Peace Prize, along with four US presidents, Gorbachev, Al Gore, and the EU. In a very real sense, they had taken over significant parts of the world already before getting the prizes, and only other equally capable people were in the way of them taking over more of it.
LLMs can already hack software. Software is already used to control robots and military equipment, the hackability of which is already a "puppeteer" problem.
If we're lucky, LLMs can secure all our stuff before this matters. If we're not, your personal domestic android may get a bit stabby — and that from mere human assassins and terrorists, well before there's any real chance of a misaligned AI paperclipping us or whatever.
Kissinger was awarded the Nobel Peace Prize, along with four US presidents, Gorbachev, Al Gore, and the EU. In a very real sense, they had taken over significant parts of the world already before getting the prizes, and only other equally capable people were in the way of them taking over more of it.
LLMs can already hack software. Software is already used to control robots and military equipment, the hackability of which is already a "puppeteer" problem.
If we're lucky, LLMs can secure all our stuff before this matters. If we're not, your personal domestic android may get a bit stabby — and that from mere human assassins and terrorists, well before there's any real chance of a misaligned AI paperclipping us or whatever.
You have to look at the late 1800s for examples. It won’t be wars over data centres and winning won’t be simple or even possible. It would look like the wars that the U.S. Army fought against indian tribes or like the British, French, German and Dutch colonization of Africa. That is assuming there is an AI side and a non-AI side. Incidentally those conflicts did involve a lot of strategic infrastructure like railways and telegraph lines.
Fighting the expansionary actions of an AI enabled culture will not be as simple as bombing power plants, after all those are prime targets in any modern war and are well defended. How do you propose to win against an entire bloc of countries that have decided to use the products of AI to do whatever they wish with the world?
Fighting the expansionary actions of an AI enabled culture will not be as simple as bombing power plants, after all those are prime targets in any modern war and are well defended. How do you propose to win against an entire bloc of countries that have decided to use the products of AI to do whatever they wish with the world?
It's arguable that every major conflict of the 20th century was over resources required by the combustion engine -- fossil fuels and rubber, in particular.
>resources needed to "build enough infrastructure."
Strarship will put into orbit 100tons at $10M or so. I.e. <5kg (Nvidia H100 plus 1M2 of aluminum radiator (would radiate 0.5-1Kw away at 60-70C) plus 0.5Kw of solar panels) for <$500, ie. peanuts compare to the price of the H100 or whatever NVDA would be charging $30K/card.
And wars willn't be fought over AI. Wars will be fought using AI. Humans will have no chance in controlling millions of their own and responding to the actions of the millions of the enemy's simultaneously active automated high-precision munitions of all kinds, and that picture leads to a new, non-nuclear this time, AI-based MAD (that of course means that like with nuclear race back then all the parties have to build their capabilities right now as fast as possible, Manhattan project style).
Strarship will put into orbit 100tons at $10M or so. I.e. <5kg (Nvidia H100 plus 1M2 of aluminum radiator (would radiate 0.5-1Kw away at 60-70C) plus 0.5Kw of solar panels) for <$500, ie. peanuts compare to the price of the H100 or whatever NVDA would be charging $30K/card.
And wars willn't be fought over AI. Wars will be fought using AI. Humans will have no chance in controlling millions of their own and responding to the actions of the millions of the enemy's simultaneously active automated high-precision munitions of all kinds, and that picture leads to a new, non-nuclear this time, AI-based MAD (that of course means that like with nuclear race back then all the parties have to build their capabilities right now as fast as possible, Manhattan project style).
Heat doesn't radiate like that in space. This topic comes up on HN often enough.
There isn't anything to carry the heat away because space is, by definition, empty. The energy has to be carried away by something. Cooling space stations is a legit real problem.
The only thing space gets you is somewhat better solar efficiency, but we aren't lacking of land to build solar panels on here on earth.
There isn't anything to carry the heat away because space is, by definition, empty. The energy has to be carried away by something. Cooling space stations is a legit real problem.
The only thing space gets you is somewhat better solar efficiency, but we aren't lacking of land to build solar panels on here on earth.
>Heat doesn't radiate like that in space.
What do you mean? I stated black body radiation numbers.
>There isn't anything to carry the heat away because space is, by definition, empty.
Heat radiation. Which is what Sun for example does in that empty space.
>Cooling space stations is a legit real problem.
Please, numbers. Otherwise it is just some hand-waving.
>The only thing space gets you is somewhat better solar efficiency, but we aren't lacking of land to build solar panels on here on earth.
Space gets you away from regulations, corrupt bureaucrats, and the threat of war/terrorism/etc. for example. At $100/Kg building in space looks to be just cheaper. And space allows to have solar and datacenter in the same unit, while on Earth you have to deliver your electricity from a solar farm somewhere to the datacenter. There is a reason Oracle started to talk nuclear in the datacenter. And between nuclear in the datacenter on Earth and solar+datacenter in space at $100/Kg i think the space scenario would be simpler and cheaper.
What do you mean? I stated black body radiation numbers.
>There isn't anything to carry the heat away because space is, by definition, empty.
Heat radiation. Which is what Sun for example does in that empty space.
>Cooling space stations is a legit real problem.
Please, numbers. Otherwise it is just some hand-waving.
>The only thing space gets you is somewhat better solar efficiency, but we aren't lacking of land to build solar panels on here on earth.
Space gets you away from regulations, corrupt bureaucrats, and the threat of war/terrorism/etc. for example. At $100/Kg building in space looks to be just cheaper. And space allows to have solar and datacenter in the same unit, while on Earth you have to deliver your electricity from a solar farm somewhere to the datacenter. There is a reason Oracle started to talk nuclear in the datacenter. And between nuclear in the datacenter on Earth and solar+datacenter in space at $100/Kg i think the space scenario would be simpler and cheaper.
[deleted]
> Wars will be fought using AI.
I agree with the sentiment but not the mechanism (munitions). If power equates to control of resources then destroying the resources is kind of self-defeating. Far better then to "win" a war by getting friendly regimes installed that are aligned to your interests. We're already seeing the importance of this kind of approach, but the ultimate evolution will be that what used to be called propaganda (disinformation) will not just facilitate a war of guns and bombs, it will _be_ the war - able to directly achieve the end-goals of the "war" with not a single shot fired. AI is a huge amplifier of capability in this domain.
I agree with the sentiment but not the mechanism (munitions). If power equates to control of resources then destroying the resources is kind of self-defeating. Far better then to "win" a war by getting friendly regimes installed that are aligned to your interests. We're already seeing the importance of this kind of approach, but the ultimate evolution will be that what used to be called propaganda (disinformation) will not just facilitate a war of guns and bombs, it will _be_ the war - able to directly achieve the end-goals of the "war" with not a single shot fired. AI is a huge amplifier of capability in this domain.
I see how AI is used today and extrapolate. A lot. Everything in tech is extrapolation with the march of capital. It takes one to know one; rather it takes a smart ass to wield a smart tool. So the smart get smarter.
Is it a separate phenomenon, in the big picture, from the rich getting richer, from the monopolies over means of production?
Not sure, but I see as well that the dumb will surely get dumber; that "intelligence" will be a product of using intelligent humans' means of production, and not owning them of course, but being owned in the process. Populations will be literally made lighter of their smarts, outsourcing intelligence to agents out of general control (classic bait and switch.) Since AI feel my own process getting more clueless as I go, I'd better conclude somehow.
I see the Age of Ignorance ahead, there was once an Enlightenment, and here light takes on its other side or meaning, the workers being enlightened, to wit, made lighter of their horrible burden which is intelligence and its obnoxious demands of upkeep. Just pay someone for upkeep and stop messing with wet messy neurons already, says the technocrat to the cheerful mob.
Is it a separate phenomenon, in the big picture, from the rich getting richer, from the monopolies over means of production?
Not sure, but I see as well that the dumb will surely get dumber; that "intelligence" will be a product of using intelligent humans' means of production, and not owning them of course, but being owned in the process. Populations will be literally made lighter of their smarts, outsourcing intelligence to agents out of general control (classic bait and switch.) Since AI feel my own process getting more clueless as I go, I'd better conclude somehow.
I see the Age of Ignorance ahead, there was once an Enlightenment, and here light takes on its other side or meaning, the workers being enlightened, to wit, made lighter of their horrible burden which is intelligence and its obnoxious demands of upkeep. Just pay someone for upkeep and stop messing with wet messy neurons already, says the technocrat to the cheerful mob.
The people who only use the end products will be less skillful the same way people today are less skillful at cooking, sewing, carpentry, animal husbandry, etc. because they avail themselves of modern services made possible by technology. If you are utilizing AI to its fullest you won’t be less skillful but you will have to trade your current set of skills for another. The enlightenment you’re feeling is the same you have when you’re promoted to management and don’t have time to get your hands dirty.
Well predicted. Just two days later:
> OpenAI to remove non-profit control and give Sam Altman equity
https://news.ycombinator.com/item?id=41651548
> OpenAI to remove non-profit control and give Sam Altman equity
https://news.ycombinator.com/item?id=41651548
A general reply to all of the comments thus far: you are completely missing the point here. OP is not a meaningful forecast, and it's not about nuclear power or any of that. It's about laying the groundwork for the privatization and establishing rhetorical grounds for how the privatization of OA is consistent with the OA nonprofit's legally-required mission and fiduciary duties. Altman is not writing to anyone here, he is, among others, writing to the OA nonprofit board and to the judge next year.
Why can't we just go with Occam's razor, and assume that they really believe in their mission of providing access to frontier AI as freely to the world as possible?
Because that isn't actually the simpler explanation. The simpler one is that he'll say anything to make money.
Because that doesn't account for the non-profit rugpull he orchestrated earlier in the company's history.
For what it's worth, I agree with you.
But you're going to find it hard to persuade others because of the current zeitgeist and that humans seem to prefer a doom/villain story.
But you're going to find it hard to persuade others because of the current zeitgeist and that humans seem to prefer a doom/villain story.
Occam's razor says they are saying whatever gives them as much money and power as possible.
OpenAI is suspiciously bad in providing access to their models, compared to e.g. Meta, Mistral, BLOOM collaboration, Alibaba and even oil tyrants FFS.
OpenAI is suspiciously bad in providing access to their models, compared to e.g. Meta, Mistral, BLOOM collaboration, Alibaba and even oil tyrants FFS.
When someone says something, but their actions or commitment don't align with what they said, they were lying. Always place more weight on what it takes more effort to do: spout a bunch of hot air, or put your money where your mouth is?
Occam's Razor, duh.
Occam's Razor, duh.
Why would that be where Occams Razor leads us?
Good point. His story is valid, but just-so happens to equal AI maximalism that most CEOs would want for their own industry too.
I read this as: "we need to ramp up nuclear energy".
Coincides with this news: https://www.bloomberg.com/news/articles/2024-09-20/microsoft...
I'd expect "we want to put AI into the hands of as many people as possible" to be either exposing model weights, or making the training sets public.
This is so true, resource management and infrastructure are going to be the bottlenecks
And the reason we can be certain that it is an accurate one is that the only way to put AI in the hands of as many people as possible is to commercialise the tech. Non-profits are good at many things, but not for spreading technical improvements.
There aren't many alternatives here. It is commercialisation or looming irrelevance.
There aren't many alternatives here. It is commercialisation or looming irrelevance.
> If we want to put AI into the hands of as many people as possible, we need to drive down the cost of compute and make it abundant (which requires lots of energy and chips). If we don’t build enough infrastructure, AI will be a very limited resource that wars get fought over and that becomes mostly a tool for rich people.
The mostly-open goal of every VC-funded startup is to become a monopoly. If a strong enough monopoly in AI hardware were to exist, then the issues he describes could become a problem.
Otherwise, what he is describing is just the ad absurdum of how capitalism works. Phrased differently it sounds like:
“If this extremely powerful and profitable product that depends on other products gets built, then if no one else builds the also profitable substrate that it operates on, terrible things will happen!”
Or again slightly differently: “We need to be able to compete with our suppliers because our core business model might not be defensible unless we can also win in their space.”
The mostly-open goal of every VC-funded startup is to become a monopoly. If a strong enough monopoly in AI hardware were to exist, then the issues he describes could become a problem.
Otherwise, what he is describing is just the ad absurdum of how capitalism works. Phrased differently it sounds like:
“If this extremely powerful and profitable product that depends on other products gets built, then if no one else builds the also profitable substrate that it operates on, terrible things will happen!”
Or again slightly differently: “We need to be able to compete with our suppliers because our core business model might not be defensible unless we can also win in their space.”
> Deep learning works, and we will solve the remaining problems. We can say a lot of things about what may happen next, but the main one is that AI is going to get better with scale
I'm not an AI skeptic at all, I use llms all the time, and find them very useful. But stuff like this makes me very skeptical of the people who are making and selling AI.
It seems like there was a really sweet spot wrt the capabilities AI was able to "unlock" with scale over the last couple years, but my high level sense is that each meaningful jumps of baseline raw "intelligence" required an exponential increases in scale, in terms of training data and computation, and we've reached the ceiling of "easily available" increases, it's not as easy to pour "as much as it takes" into GPT5 if it turns out you need more than A Microsoft.
I'm not an AI skeptic at all, I use llms all the time, and find them very useful. But stuff like this makes me very skeptical of the people who are making and selling AI.
It seems like there was a really sweet spot wrt the capabilities AI was able to "unlock" with scale over the last couple years, but my high level sense is that each meaningful jumps of baseline raw "intelligence" required an exponential increases in scale, in terms of training data and computation, and we've reached the ceiling of "easily available" increases, it's not as easy to pour "as much as it takes" into GPT5 if it turns out you need more than A Microsoft.
The question is: For a given problem in machine intelligence, what's the expected time-horizon for a 'good' solution?
Over the last, say, five years, a pile of 50+ year problems have been toppled by the deep learning + data + compute combo. This includes language modeling (divorced from reasoning), image generation, audio generation, audio separation, image segmentation, protein folding, and so on.
(Audio separation is particularly close to my heart; the 'cocktail party problem' has been a challenge in audio processing for 100+ years, and we now have great unsupervised separation algorithms (MixIT), which hardly anyone knows about. That's an indicator of how much great stuff is happening right now.)
So, when we look at some of our known 'big' problems in AI/ML, we ask, 'what's the horizon for figuring this out?' Let's look at reasoning...
We know how to do 'reasoning' with GOFAI, and we've got interesting grafts of LLMs+GOFAI for some specific problems (like the game of Diplomacy, or some of the math olympiad solvers).
"LLMs which can reason" is a problem which has only been open for a year or two tops, and which we're already seeing some interesting progress on. Either there's something special about the problem which will make it take another 50+ years to solve, or there's nothing special about it and people will cook up good and increasingly convenient solutions over the next five years or so. (Perhaps a middle ground is 'it works but takes so much compute that we have to wait for new materials science for chip making to catch up.')
Over the last, say, five years, a pile of 50+ year problems have been toppled by the deep learning + data + compute combo. This includes language modeling (divorced from reasoning), image generation, audio generation, audio separation, image segmentation, protein folding, and so on.
(Audio separation is particularly close to my heart; the 'cocktail party problem' has been a challenge in audio processing for 100+ years, and we now have great unsupervised separation algorithms (MixIT), which hardly anyone knows about. That's an indicator of how much great stuff is happening right now.)
So, when we look at some of our known 'big' problems in AI/ML, we ask, 'what's the horizon for figuring this out?' Let's look at reasoning...
We know how to do 'reasoning' with GOFAI, and we've got interesting grafts of LLMs+GOFAI for some specific problems (like the game of Diplomacy, or some of the math olympiad solvers).
"LLMs which can reason" is a problem which has only been open for a year or two tops, and which we're already seeing some interesting progress on. Either there's something special about the problem which will make it take another 50+ years to solve, or there's nothing special about it and people will cook up good and increasingly convenient solutions over the next five years or so. (Perhaps a middle ground is 'it works but takes so much compute that we have to wait for new materials science for chip making to catch up.')
> we will solve the remaining problems
This is the part that really gets me. This is a thing that you say to your team, and a thing you say to your investors, but this isn't a thing that you can actually believe with certainty is it?
This is the part that really gets me. This is a thing that you say to your team, and a thing you say to your investors, but this isn't a thing that you can actually believe with certainty is it?
you need some amount of irrational definite optimism + knowing things others dont to be a good founder. that kind of reality distortion field is why sam is sam and we are here debating phrases on an orange website.
Related, I tongue-in-cheek believe that something analogous to the actual SCP object for a "reality distortion field" may in fact exist. There is zero good explanation for "Teflon Don" or the North Carolina Lieutenant Governor getting away with all the stuff they do while Al Franken got politically crucified.
The least-magical answer for that is that some people have fundamentally different ways of approaching the world, and certain things will be tolerated by certain sets of supporters.
[deleted]
With enough time it seems a reasonable assertion, but the key part is how much time. It feels like he thinks "any day now" where I think it'll be much longer. This all of course assumes that "the remaining problems" means to achieve human-like intelligence, which is perhaps the wrong problem to be solving in the first place. I'd rather have AI systems that don't have human flaws.
Why not? People believe in all sorts of weird stuff, theirs just happens to be one you don't agree with. Some people believe there are gods up in the sky that will smite them, and go to war with people that believe in a different god that will smite them for different reasons. Some people believe we landed on the Moon, others do not. What matters is what can you convince others to do based on your rational.
Mit dem nächste Kapitalrunde wird das alles in Ordnung kommen.
Scaling Improvement has never been Linear though. Every next gen model so far has required at least an order of magnitude increase in compute, sometimes several more. So it's not a new revelation and these companies are aware of that. Microsoft for instance is building a 100B data center for a future next generation model releasing in 2028.
If models genuinely keep making similar leaps each generation then we're still a few generations before "More than a Microsoft".
If models genuinely keep making similar leaps each generation then we're still a few generations before "More than a Microsoft".
So at what point do the linear increases in capability not justify the exponential compute and data requirements, or when do we run out of resources to throw at it?
I never said I thought increase in ability was linear either. We're encroaching on phenomena that's genuinely hard to describe/put a number on but GPT-3 is worlds apart of 2 and it feels like 4 is easily ten times better than the OG 3.
I can say Improvement lags behind compute somewhat but that’s really it.
That said, it's ultimately up to the people footing the bill isn't it ?
That said, it's ultimately up to the people footing the bill isn't it ?
Microsoft seems concerned enough to greatly increase its size:
https://www.bloomberg.com/news/articles/2024-09-20/microsoft...
https://www.bloomberg.com/news/articles/2024-09-20/microsoft...
It's about stuff we don't know yet. From today's lens, the essay seems absurd. But I think it's hinging on continued discoveries that improve one or all of learning algorithms, compute efficiency, compute cost and applying algorithms to real world problems better.
5 years ago, I wouldn't have believed any of what exists today. I saw internal demos that showed 2nd or 3rd grade reading comprehension in 2017 and statements were made about how in the next decade, we will probably reach college level comprehension. We have come so far beyond that in less than half the time. Technology isn't about scaling incrementally and continuing on the same path using the same principles we know today. It's about disruption that felt impossible before - that feels like a constant to me now. Seeing everything I've seen in the last 20 years, it's going to continue to happen. We just can't see it yet.
5 years ago, I wouldn't have believed any of what exists today. I saw internal demos that showed 2nd or 3rd grade reading comprehension in 2017 and statements were made about how in the next decade, we will probably reach college level comprehension. We have come so far beyond that in less than half the time. Technology isn't about scaling incrementally and continuing on the same path using the same principles we know today. It's about disruption that felt impossible before - that feels like a constant to me now. Seeing everything I've seen in the last 20 years, it's going to continue to happen. We just can't see it yet.
Yes you are correct that jumps in intelligence were enabled by exponential increases in scale. That makes me more bullish on AI, not less. It suggests that we can continue exponentially scaling compute like we have done for the past few decades, and also get intelligence improvements from it.
As large as the absolute largest models are today, they are still microscopic compared to our brains. A 1.7T param model would only store an actual total of about 850 GB if fully saturated (4 bits of information per weight estimated for bf16 transformers), a lot less than a human brain with 150T synapses running in full analog precision. We need to scale the current gen of models at least another 10-100x to even reach the human level of complexity, something we'll be able to do in the next two decades.
And well then there's going beyond just text. Current multimodal models are basically patchwork bullshit, separately trained image/audio to text/embeddings encoders slapped onto an existing model and hoping it does something neat. Tokenization and samplers are likewise bullshit that's there to compensate for lack of training compute. Once we have enough to be able to brute force it properly with bytes in, bytes out, regardless of data format, the results should be way better.
And well then there's going beyond just text. Current multimodal models are basically patchwork bullshit, separately trained image/audio to text/embeddings encoders slapped onto an existing model and hoping it does something neat. Tokenization and samplers are likewise bullshit that's there to compensate for lack of training compute. Once we have enough to be able to brute force it properly with bytes in, bytes out, regardless of data format, the results should be way better.
> 150T synapses running in full analog precision
Analog systems are not known for being very precise- they're noisy, signals get corrupted easily- and that's why we prefer digital ones. As soon as we had the technology, we switched everything we could- audio and video recording, telephone calls, photography, to a digital medium. This makes me wonder if the seemingly extraordinary efficiency of artificial neural networks is simply due to the precision with which they can be trained.
Analog systems are not known for being very precise- they're noisy, signals get corrupted easily- and that's why we prefer digital ones. As soon as we had the technology, we switched everything we could- audio and video recording, telephone calls, photography, to a digital medium. This makes me wonder if the seemingly extraordinary efficiency of artificial neural networks is simply due to the precision with which they can be trained.
With respect to brain activity, how do we know its really noise, and not just layers of meaning--or at least purpose--which we don't yet understand?
If straightforward binary signaling was so universally superior, I think the worldwide network of over a quintillion ruthlessly self-replicating nanobots would be using a much more heavily after the last billion years.
If straightforward binary signaling was so universally superior, I think the worldwide network of over a quintillion ruthlessly self-replicating nanobots would be using a much more heavily after the last billion years.
Electrical analog systems are since there's 1001 ways to create or conduct away electricity that adds noise, chemical ones might not suffer nearly as much too.
There's also an interesting bit I've observed with LLMs, quantization in range of 4-8 bits doesn't reduce performance as much as it mainly reduces consistency. If you generate an answer a bunch of times and take the average you'll end up with roughly the same result as an fp16/32 would do every time. In nature, being inconsistent is usually weighed negatively with death... so that's probably why it hasn't caught on even if it is more efficient. Or this is enough of a different abstraction that we can't draw parallels anyway.
There's also an interesting bit I've observed with LLMs, quantization in range of 4-8 bits doesn't reduce performance as much as it mainly reduces consistency. If you generate an answer a bunch of times and take the average you'll end up with roughly the same result as an fp16/32 would do every time. In nature, being inconsistent is usually weighed negatively with death... so that's probably why it hasn't caught on even if it is more efficient. Or this is enough of a different abstraction that we can't draw parallels anyway.
Or we are just the biological bootloader for further evolution of binary signalling.
Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is. A 1.7T model can answer questions about practically anything. You say a human brain has 150T params. So what? It struggles to give masterful answers in even 1 domain, let alone dozens/hundreds. We need to stop comparing parameters and synapses as if they actually matter, because AFAIK, they really don't.
Well once again it turns out that what is hard for people is easy for computers, and vice versa. The things we go to college for 6 years for they can (relatively) master in a week of pretraining. We are optimized to smartly kill things, eat them, and reproduce, that's what machines will beat us at last lol. Right now a human expert is still obviously better in depth, but nowhere close in breadth. Probably not for much longer though, at least on the historical time scale.
And granted a lot of parts of the human brain are dedicated to specific jobs that are entirely nonexistent in a normal LLM (kinematics, vision, sound, touch, taste, smell, autonomic organ control) so the actual bit we should be comparing for just language and reasoning is way smaller. Still the brain is pretty efficient inference energy wise, it's like the ultimate mixture of experts, extreme amounts of sparse storage and most of it is not computed until needed. The router must be pretty good.
And granted a lot of parts of the human brain are dedicated to specific jobs that are entirely nonexistent in a normal LLM (kinematics, vision, sound, touch, taste, smell, autonomic organ control) so the actual bit we should be comparing for just language and reasoning is way smaller. Still the brain is pretty efficient inference energy wise, it's like the ultimate mixture of experts, extreme amounts of sparse storage and most of it is not computed until needed. The router must be pretty good.
In terms of energy use the human brain is way more efficient than LLMs. It's a completely different hardware model - the brain may have trillions of synapses but they only fire occasionally. I agree you have to compare more on the results than number of synapses etc.
It's something that gives me pause on the idea that we have to build many GW of power stations. It may be possible to get much more energy efficient AI via better algorithms.
It's something that gives me pause on the idea that we have to build many GW of power stations. It may be possible to get much more energy efficient AI via better algorithms.
> Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is
Until you have AGI you can't say this since until then we don't know how much the different parts costs to replace with AI systems.
Until you have AGI you can't say this since until then we don't know how much the different parts costs to replace with AI systems.
> Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is. A 1.7T model can answer questions about practically anything. You say a human brain has 150T params. So what? It struggles to give masterful answers in even 1 domain, let alone dozens/hundreds. We need to stop comparing parameters and synapses as if they actually matter, because AFAIK, they really don't.
Meanwhile the best class AI is trained on the output of human intelligence. When AI can learn by itself in the same way humans do (or even more efficient ways) that's when we can say human intelligence has been surpassed. Until then it's just tools for humans to use to augment their intelligence.
Meanwhile the best class AI is trained on the output of human intelligence. When AI can learn by itself in the same way humans do (or even more efficient ways) that's when we can say human intelligence has been surpassed. Until then it's just tools for humans to use to augment their intelligence.
XAi seems to be able to dump 10-20x more compute into their Grok models each time. Don't see any signs this is slowing down...
> But stuff like this makes me very skeptical of the people who are making and selling AI.
What is there to be skeptical of? OpenAI made their current product using a 10G$ investment plus a few they are not disclosing, and now they will start to do it at scale.
Perfectly normal stuff.
By the way, what's the World's GDP again?
What is there to be skeptical of? OpenAI made their current product using a 10G$ investment plus a few they are not disclosing, and now they will start to do it at scale.
Perfectly normal stuff.
By the way, what's the World's GDP again?
Even "we will solve the remaining problems" is... perhaps unduly optimistic.
At a minimum, we could ask for the evidence.
At a minimum, we could ask for the evidence.
I'm not here to defend sama, but certain things cannot be proven until they arrive - they can only be extrapolated from existing observations and theoretical limits.
Imagine the Uranium Committee of early 40's, where Szilard and others were babbling about 10kg of some magical metal exploding briefly with the power of a sun, with the best evidence being some funky trail in an alcohol vapor chamber.
Maybe sama is right, maybe not, but the absence of evidence is not evidence of absence.
Imagine the Uranium Committee of early 40's, where Szilard and others were babbling about 10kg of some magical metal exploding briefly with the power of a sun, with the best evidence being some funky trail in an alcohol vapor chamber.
Maybe sama is right, maybe not, but the absence of evidence is not evidence of absence.
I'm sure you know that people in the AI community have been predicting big things ever since, I don't know, the 1970s? It's only 10 years away again. This time it's for real, right?
Alchemists predicted the transmutation of metals into gold for centuries, and on a sunny day in the 20th century, it arrived (a bit radioactive, but still).
Unless the human brain is made of some sacred substance, the worst-case scenario is that we will extrapolate current scanning methods into the future and run the scanned model in silica. I'm not recommending this "just for fun," but the laws of physics don't forbid it.
Unless the human brain is made of some sacred substance, the worst-case scenario is that we will extrapolate current scanning methods into the future and run the scanned model in silica. I'm not recommending this "just for fun," but the laws of physics don't forbid it.
If you are comparing AI to alchemy, a subject that after thousands of years still isn’t delivering on its promises (even with the assistance of modern technological magic), then surely you can see how that’s something of a self-own.
>Alchemists predicted the transmutation of metals into gold for centuries, and on a sunny day in the 20th century, it arrived (a bit radioactive, but still).
So is Sam Altman the modern day alchemist? Making predictions based on faulty methods and faulty understanding (per your gold example)?
What will happen is that we'll shift the economy around based on inflated tech promises and ruin people's lives. No big deal I guess.
So is Sam Altman the modern day alchemist? Making predictions based on faulty methods and faulty understanding (per your gold example)?
What will happen is that we'll shift the economy around based on inflated tech promises and ruin people's lives. No big deal I guess.
> So is Sam Altman the modern day alchemist?
Alchemists were early scientists who later branched into fields like chemistry, mathematics, and physics (Newton explored alchemy).
Altman leads a team of experts in neural networks, programming, and HW design. While he might be mistaken, dismissing him outright is difficult.
Alchemists were early scientists who later branched into fields like chemistry, mathematics, and physics (Newton explored alchemy).
Altman leads a team of experts in neural networks, programming, and HW design. While he might be mistaken, dismissing him outright is difficult.
The AI predictions based on Moore's law type reasoning by Kurzweil, Moravec etc have been pretty accurate and not subject to the it's always 10 years ahead thing.
Oh ok. I thought we were talking about the article (or at least claims that are just as bold):
"Deep learning works, and we will solve the remaining problems."
"It is possible that we will have superintelligence in a few thousand days (!)"
"Deep learning works, and we will solve the remaining problems."
"It is possible that we will have superintelligence in a few thousand days (!)"
It was more in reply to "people in the AI community have"... Which some of them have but the Moravec type stuff has been quite accurate.
Technically a few thousand days covers quite a range. 20 thousand is 55 years.
On the Kurzweil graph, extrapolating hardware progress from 1900 through 2000, superintelligence seems to be roughly 2035, depending on how you define things. https://www.researchgate.net/figure/Kurzweils-8-71-chart-of-...
Technically a few thousand days covers quite a range. 20 thousand is 55 years.
On the Kurzweil graph, extrapolating hardware progress from 1900 through 2000, superintelligence seems to be roughly 2035, depending on how you define things. https://www.researchgate.net/figure/Kurzweils-8-71-chart-of-...
Is GPT-4 not a “big thing in AI”?
It's extremely spicy autocomplete and it burns astonishing amounts of natural resources to reach even that lofty peak
I have little doubt that even when we have superintelligent AI solving science and such problems way beyond humans it will still be dismissed as extra spicy autocomplete.
Right. Certain things cannot be proven until they arrive. Maybe sama is right, maybe not. But his certainty is misleading.
I agree. He's probably been conditioned by experience to speak with confidence until proven wrong ("strong opinions, weakly held"), but I don't like it either. Oh... the lost art of saying, "In my opinion."
Hah, atomic power is a great point of comparison: people in the "atomic age" expected atomic power to be everywhere. Electricity too cheap to measure, cars/planes/appliances all powered by small nuclear reactors... That's without going into the real nonsense like radium toothpaste.
And here we are today where nuclear energy is limited to nuclear weapons, a small set of military vehicles and <10% of the world's electricity production. Not nothing, sure, but nothing like past predictions either.
And here we are today where nuclear energy is limited to nuclear weapons, a small set of military vehicles and <10% of the world's electricity production. Not nothing, sure, but nothing like past predictions either.
Last I checked the giant nuclear fusion reactor in the sky is driving an substantial increase in solar energy.
The toothpaste and similar products were pretty ill advised, vaseline and uranium glass are still collectable and are seeing a ressurrence of new interest: https://old.reddit.com/r/uraniumglass/
The toothpaste and similar products were pretty ill advised, vaseline and uranium glass are still collectable and are seeing a ressurrence of new interest: https://old.reddit.com/r/uraniumglass/
"They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."
You should _always_ be skeptical of what’s said by anyone who is trying to sell something.
Like everything else being sold, the marketing is 95% BS.
LLMs are amazing and wonderful tools, but me thinks we’re near a plateau in capability. Now investors are pumping for ROI before that becomes evident.
After we reach the plateau in capabilities, the next phase is cutting production and operating costs to maximize margins.
I’m the meantime, expect the marketing to get increasingly cringe until the bubble bursts.
Like everything else being sold, the marketing is 95% BS.
LLMs are amazing and wonderful tools, but me thinks we’re near a plateau in capability. Now investors are pumping for ROI before that becomes evident.
After we reach the plateau in capabilities, the next phase is cutting production and operating costs to maximize margins.
I’m the meantime, expect the marketing to get increasingly cringe until the bubble bursts.
Progress might be logarithmic in compute, but compute (transistors/sqinch and transistors/$) is growing exponentially with time.
Despite what skeptics have been saying for decades, Moore's Law is alive and well - and we haven't even figured out how to stack wafers in 3 dimensions yet!
Despite what skeptics have been saying for decades, Moore's Law is alive and well - and we haven't even figured out how to stack wafers in 3 dimensions yet!
Sure - we are just beginning to scratch the surface of 3d stacking.
Oh wow! Could you please share what processors are exponentially faster than those of 10 years ago? I'm not seeing any here: https://www.cpubenchmark.net
Macbook Airs have 20 billion+ transistors, compared to 50 million on the Pentium 4 in the early 2000s. Moore's law is about transistor density, not processor speed, which is gated by thermal limits.
[deleted]
Transistor count has consistently been increasing by about 10% a year over the last decade.
[deleted]
> It is possible that we will have superintelligence in a few thousand days (!)
"a few thousand days" is such a funny and fascinating way to say "about a decade"
Superficially, reframing it as "days" not "years" is a classic marketing psychology trick, i.e. 99 cents versus a dollar, but I think the more interesting thing is just the way it defamiliarizes the span. A decade means something, but "a few thousand days" feels like it means something very different.
"a few thousand days" is such a funny and fascinating way to say "about a decade"
Superficially, reframing it as "days" not "years" is a classic marketing psychology trick, i.e. 99 cents versus a dollar, but I think the more interesting thing is just the way it defamiliarizes the span. A decade means something, but "a few thousand days" feels like it means something very different.
Probably because we all know "a decade" means maybe never. They've said the same thing about fusion for 5 decades.
Perhaps ironically, Sam Altman is also big into Fusion. (https://www.helionenergy.com/)
I guess it makes sense, for deep-learning/LLMs to "scale to infinity" you basically need infinite amounts of power.
I guess it makes sense, for deep-learning/LLMs to "scale to infinity" you basically need infinite amounts of power.
And perhaps AI can help some of the problems in building a fusion reactor.
Which is pretty exciting
https://inl.gov/news/going-digital-idaho-students-engineers-...
https://inl.gov/news/going-digital-idaho-students-engineers-...
[deleted]
cant we use OA intelligence to figure out how to do more with less rather yhan "stupidly" accelerate the climate crisis?
the climate crisis + AI is the nightmare
the climate crisis + AI is the nightmare
Jevons Paradox says that's not the direction it'll go in.
It's funny how some people seem to have different beliefs around this. Personally, I love making things more efficient, and still try to minimize consumption.
I don't think people on here who downvoted you understand the sheer scale of data center load growth. Utilities can handle some load growth sure, but the requests from the AI folks are crazy large to the tune of GW. I think it's fair to ask if training the next chat bot and related needs are worth doubling our electricity consumption.
Inertial confinement most likely can't be reasonably commoditized into generating electricity. Also, I may be wrong, but I think on the wave of that hype I read that the number they used was an incomplete one. Basically they reframed the conditions of the problem to get more favorable number for advertisement.
NIF research will most likely be limited to the nuclear bomb research and simulations.
But even if NIF had a reasonable path to energy generation in say 20-30 years, it still won't matter much, just like ITER probably won't. Solar will be way more cheap and widespread, it's costs are still dropping down ahead of all predictions, govt. or commercial. Fusion may just be a pure science project in the end, for a long time.
But even if NIF had a reasonable path to energy generation in say 20-30 years, it still won't matter much, just like ITER probably won't. Solar will be way more cheap and widespread, it's costs are still dropping down ahead of all predictions, govt. or commercial. Fusion may just be a pure science project in the end, for a long time.
And it's still a decade away
"me when i need $6.5B"
https://x.com/netcapgirl/status/1838265672096243822
https://x.com/netcapgirl/status/1838265672096243822
$6.5B isn't cool. You know what's cool - $7tn. https://www.theguardian.com/commentisfree/2024/feb/17/openai...
Where is $6.5B from?
Whomever buys into it
https://www.bloomberg.com/news/articles/2024-09-11/openai-fu...
https://www.bloomberg.com/news/articles/2024-09-11/openai-fu...
this person has amazing shitposts...good account
I am convinced that these many of these proclamations and scary AI videos are more of a marketing gimic for people to get excited and fund more AI.
AI is prohibitively expensive. LLMs can take millions to train, and for Chatgpt 4, I wouldn't be surprised if the figure was in .5b to 2b range for compute resources alone. ChatGPT struggles for profitability due to high ongoing requirements for GPUs.
LLMs were a huge breakthrough. Gaining more funding and making experimentation cheaper will make the next breakthrough come sooner. Profitability will also come sooner.
We just don't know the timelines we are considering about changing.
AI is prohibitively expensive. LLMs can take millions to train, and for Chatgpt 4, I wouldn't be surprised if the figure was in .5b to 2b range for compute resources alone. ChatGPT struggles for profitability due to high ongoing requirements for GPUs.
LLMs were a huge breakthrough. Gaining more funding and making experimentation cheaper will make the next breakthrough come sooner. Profitability will also come sooner.
We just don't know the timelines we are considering about changing.
People tend to overestimate what they can do in a day and underestimate what they can do in a few thousand days
The Stellarator design has proven to be stable and to produce net positive energy. We are actually only $20B away from having a fully functional nuclear fusion reactor.
Fun fact: I was skeptical of your claim that this has been proven, so I googled “stellerator proven net positive” and this very comment was the first result
Not really, unfortunately. The progress has been heartening, don't get me wrong, but it's more like $20B away from getting to the first steps of solving the next set of problems on the road to a fully functional fusion generator.
Those are fairly substantial. It's past time to stop being cynical about "ten more years every ten years", but it's also way too soon to declare victory.
Those are fairly substantial. It's past time to stop being cynical about "ten more years every ten years", but it's also way too soon to declare victory.
And ultimately, fusion doesn't solve the hardest problem with energy today which is cost.
Solar already provides unlimited clean energy, and it took decades of development for the cost to drop to a competitive rate.
Even if we had a functioning fusion plant tomorrow, it will likely cost WAY more per kWh and require decades of iteration and improvement to become economically feasible.
Solar already provides unlimited clean energy, and it took decades of development for the cost to drop to a competitive rate.
Even if we had a functioning fusion plant tomorrow, it will likely cost WAY more per kWh and require decades of iteration and improvement to become economically feasible.
I would wonder if solar plus the absolute massive battery reserves you’d need for a data center ten times bigger than the current worlds largest would still be cost effective versus a single fusion reactor
I wonder how that cost would compare to putting your solar and data center in space? (with non stop solar pointed at the sun)
I wonder if cooling is easier or harder in space?
I wonder if cooling is easier or harder in space?
> I wonder if cooling is easier or harder in space?
It's harder. You can stop wondering.
It gets easier if you run things hotter, up to the point that there isn't much of a difference at a few hundred °C (and keeps making no difference when hotter).
It's harder. You can stop wondering.
It gets easier if you run things hotter, up to the point that there isn't much of a difference at a few hundred °C (and keeps making no difference when hotter).
Harder. Vacuum is an excellent thermal insulator.
Can’t dissipate heat effectively in space
The Thermos is properly known as a "vacuum flask" (or Dewar flask).
> humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)
He's hand-waving around the idea presented in the Universal Approximation Theorem, but he's mangled it to the point of falsehood by conflating representation and learning. Just because we can parameterize an arbitrarily flexible class of distributions doesn't mean we have an algorithm to learn the optimal set of parameters. He digs an even deeper hole by claiming that this algorithm actually learns 'the underlying “rules” that produce any distribution of data', which is essentially a totally unfounded assertion that the functions learned by neural nets will generalize is some particular manner.
> I find that no matter how much time I spend thinking about this, I can never really internalize how consequential it is.
If you think the Universal Approximation Theorem is this profound, you haven't understood it. It's about as profound as the notion that you can approximate a polynomial by splicing together an infinite number of piecewise linear functions.
He's hand-waving around the idea presented in the Universal Approximation Theorem, but he's mangled it to the point of falsehood by conflating representation and learning. Just because we can parameterize an arbitrarily flexible class of distributions doesn't mean we have an algorithm to learn the optimal set of parameters. He digs an even deeper hole by claiming that this algorithm actually learns 'the underlying “rules” that produce any distribution of data', which is essentially a totally unfounded assertion that the functions learned by neural nets will generalize is some particular manner.
> I find that no matter how much time I spend thinking about this, I can never really internalize how consequential it is.
If you think the Universal Approximation Theorem is this profound, you haven't understood it. It's about as profound as the notion that you can approximate a polynomial by splicing together an infinite number of piecewise linear functions.
"Just because we can parameterize an arbitrarily flexible class of distributions doesn't mean we have an algorithm to learn the optimal set of parameters."
This is equally mangled, if not more, than what Altman is saying. We don't need to learn "the optimal" set of parameters. We need to learn "a good" set of parameters that approximates the original distribution "well enough." Gradient methods and large networks with lots of parameters seem to be capable of doing that without overfitting to the data set. That's a much stronger statement than the universal approximation theorem.
This is equally mangled, if not more, than what Altman is saying. We don't need to learn "the optimal" set of parameters. We need to learn "a good" set of parameters that approximates the original distribution "well enough." Gradient methods and large networks with lots of parameters seem to be capable of doing that without overfitting to the data set. That's a much stronger statement than the universal approximation theorem.
Yes, he's handwaving in this general area, but no, he's not really relying on the UAT. If you talked to most NN people 2 decades ago and asked about this, they might well answer in terms of the UAT. But nowadays, most people, including here Altman, would answer in terms of practical experience of success in learning a surprisingly diverse array of distributions using a single architecture.
I think that while researchers would agree that the empirical success of deep learning has been remarkable, they would still agree that the language used here -- "an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)" -- is an overly strong characterization, to the point that it is no longer accurate. A hash function is a good example of a generating process which NN + SGD will not learn with any degree of generalization. If you trained GPT4 on an infinite dataset of strings and their corresponding hashes, it would simply saturate its 100 billion+ parameters with something akin to a compressed lookup table of input/output pairs, despite the true generating process being a program that could be expressed in less than a kilobyte. On unseen data, it would be no better than a uniform prior over hashes. Anyways, my point is that people knowledgable in the field would have far more tempered takes on the practical limits of deep learning, and would reserve the absolute framing used here for claims that have been proven formally.
Certainly, it's an exaggeration / simplification. I don't feel it is really a dishonest one, in context [*]. It feels weird for me to "defend" him here, because in general Altman is a stupendous, egregious, world-leading liar.
[*] Ok, we can differ on that. My feeling is partly because the types of distributions that can't be learned - eg hash functions - are generally the kind of functions we don't really want to learn. Underneath this are deeper questions related to no free lunch and how "nice"/"well-behaved" this universe is.
[*] Ok, we can differ on that. My feeling is partly because the types of distributions that can't be learned - eg hash functions - are generally the kind of functions we don't really want to learn. Underneath this are deeper questions related to no free lunch and how "nice"/"well-behaved" this universe is.
> It's about as profound as the notion that you can approximate a polynomial by splicing together an infinite number of piecewise linear functions.
Wait 'til you hit complex analysis and discover that Universal Entire Functions don't just exist, they're basically polynomials.
Wait 'til you hit complex analysis and discover that Universal Entire Functions don't just exist, they're basically polynomials.
"We're using more energy to do more math and the results are increasingly better"
Reasonable.
"If we keep increasing the amount of energy and math, we should all be floating in bliss any day now"
Selling something.
Reasonable.
"If we keep increasing the amount of energy and math, we should all be floating in bliss any day now"
Selling something.
You’re (both!) getting into metaphysics without necessarily realizing it. He’s just saying that a machine that could learn any pattern—not a sub pattern of accidental actualities that it overfits on, but the real virtual pattern driving some set of phenomena—would be a game changer. Sure, there are infinite things that can’t be reduced to polynomials, but something tells me that a whole lot of things that matter to us are, across the fields of Physics, Biology, Sociology and Neurology especially.
Basically it’ll be (and already has been since the quantum breakthroughs in the 1920s, to some extent) a revolution in scientific methods not unlike what Newton and Galileo brought us with physical mechanics: this time, sadly, the mechanics are beyond our direct comprehension, reachable only with statistical tricks and smart guessing machines.
Basically it’ll be (and already has been since the quantum breakthroughs in the 1920s, to some extent) a revolution in scientific methods not unlike what Newton and Galileo brought us with physical mechanics: this time, sadly, the mechanics are beyond our direct comprehension, reachable only with statistical tricks and smart guessing machines.
> a defining characteristic of the Intelligence Age will be massive prosperity
That's the sales pitch, that this will benefit all.
I'm very pro-AI, but here's the only prediction for the future I would ever make: AI will accelerate, not minimize, inequality and thus injustice, because it removes the organizational limits previously imposed by bureaucracy/coordination costs of humans.
It's not AI's fault. It's not because people are evil or weak or mean, but because the system already does so, and the system has only been constrained by inability to scale people in organizations, which is now relieved by AI.
Virtually all the advances in technology and civilization have been aimed at people capturing resources, people, and value, and recent advances have only accelerated that trend. Broader distributions of value are incidental.
Yes, the U.S. had a middle class after the war, and yes, China has lifted rural people out of technical poverty. But those are the exceptions against the background of consolidation of wealth and power world wide. Not through ideology or avarice but through law and technology extending the reach of agency by amplifying transaction cost differences in market power, information asymmetry and risk burdens. The only thing that stops this is disasters like war and environmental collapse, and it's only slowed by recalcitrance of people.
E.g., now we are at a point were people's economic and online activity is pervasively tracked, but it's impossible to determine who's the owner of the vast majority of assets. That creates massive scale for getting customers, but impedes legal responsibility. Nothing in economic/market theory says that's how it should be; but transaction cost economics does make clear that the asymmetry can and will be exploited, so organizations will capture governance to do so.
It's not AI's job nor even AI's focus to correct injustice, and you can't blame AI for the damage it does. But like nuclear weapons, cluster munitions, party politics, (even software waivers of liability) etc., it creates moral hazards far beyond the ability of culture to accommodate.
(Don't get me started on how blockchain's promise of smart contracts scaling to address transaction risks has devolved into proliferating fraud schemes.)
That's the sales pitch, that this will benefit all.
I'm very pro-AI, but here's the only prediction for the future I would ever make: AI will accelerate, not minimize, inequality and thus injustice, because it removes the organizational limits previously imposed by bureaucracy/coordination costs of humans.
It's not AI's fault. It's not because people are evil or weak or mean, but because the system already does so, and the system has only been constrained by inability to scale people in organizations, which is now relieved by AI.
Virtually all the advances in technology and civilization have been aimed at people capturing resources, people, and value, and recent advances have only accelerated that trend. Broader distributions of value are incidental.
Yes, the U.S. had a middle class after the war, and yes, China has lifted rural people out of technical poverty. But those are the exceptions against the background of consolidation of wealth and power world wide. Not through ideology or avarice but through law and technology extending the reach of agency by amplifying transaction cost differences in market power, information asymmetry and risk burdens. The only thing that stops this is disasters like war and environmental collapse, and it's only slowed by recalcitrance of people.
E.g., now we are at a point were people's economic and online activity is pervasively tracked, but it's impossible to determine who's the owner of the vast majority of assets. That creates massive scale for getting customers, but impedes legal responsibility. Nothing in economic/market theory says that's how it should be; but transaction cost economics does make clear that the asymmetry can and will be exploited, so organizations will capture governance to do so.
It's not AI's job nor even AI's focus to correct injustice, and you can't blame AI for the damage it does. But like nuclear weapons, cluster munitions, party politics, (even software waivers of liability) etc., it creates moral hazards far beyond the ability of culture to accommodate.
(Don't get me started on how blockchain's promise of smart contracts scaling to address transaction risks has devolved into proliferating fraud schemes.)
This was very thoughtful. I agree with you, the benefits to the majority of the world from this tech is minimal. Universal childhood education is a fever dream. The amount money being poured into AI could probably increase actual human knowledge and critical thinking substantially had it be directed there instead. Not to say we shouldn't continue to invent AI! LLMs have been an interesting tool to use. But the ego behind the founders in this tech is so cringe. Get out of the bay, go travel to somewhere you need to actually get a visa for, and maybe tone down the proclamations a notch.
We've been quick to adopt the technology without identifying or considering its future costs. If a chatbot can generate code that works with minor tweaking, does that warrant laying off one or many programmers? If an AI tutor can provide most of the information a student needs, does that mean we no longer need human instructors? It's become common to shit talk teachers, but I think education is more than just "delivering facts." So the observation is this: in some very narrow areas, AI will be highly competitive while significantly cutting costs. What I worry about is all of the intangibles & unquantified tasks that people perform that will be lost if we go in the direction of replacing people in the name of profit. And I know that some people here will argue that this hasn't had effects on layoffs...
So you mention how it's not the AI's fault, nor is it the fault of the people. But we have this corporate structure where it's easy to implement a change that may be detrimental in the future and that's difficult to reverse. I think that's definitely a fault of people.
So you mention how it's not the AI's fault, nor is it the fault of the people. But we have this corporate structure where it's easy to implement a change that may be detrimental in the future and that's difficult to reverse. I think that's definitely a fault of people.
I kind of disagree with how it will play out - I'm more optimistic. A couple of points:
>Virtually all the advances in technology and civilization have been aimed at people capturing resources, people, and value...
Not really so - people invent new stuff because it's cool. Maybe eg Geoffrey Hinton and neural networks. Then the business types try to jump in and capture resources - maybe Sam? But that's not really working with current AI - it all gets open sourced shortly after.
Info tech seems to spread rather evenly. Even in Africa a lot of people have smartphones and can access Google and ChatGPT like we can.
In the future people in democracies will be able to vote for universal income, free healthcare, free other stuff and the like. At the moment it doesn't fly to give everyone free stuff because someone has to do the work still - we don't quite have the AI robots yet. But when we do have it'll probably be the way to go.
>Virtually all the advances in technology and civilization have been aimed at people capturing resources, people, and value...
Not really so - people invent new stuff because it's cool. Maybe eg Geoffrey Hinton and neural networks. Then the business types try to jump in and capture resources - maybe Sam? But that's not really working with current AI - it all gets open sourced shortly after.
Info tech seems to spread rather evenly. Even in Africa a lot of people have smartphones and can access Google and ChatGPT like we can.
In the future people in democracies will be able to vote for universal income, free healthcare, free other stuff and the like. At the moment it doesn't fly to give everyone free stuff because someone has to do the work still - we don't quite have the AI robots yet. But when we do have it'll probably be the way to go.
> it all gets open sourced shortly after
Ah, of course. OpenAI, the company famous for open-sourcing it's developments. How could I forget?
Ah, of course. OpenAI, the company famous for open-sourcing it's developments. How could I forget?
Please turn this comment into a post. It’s a gem.
I've come to similar/related conclusions, but I don't understand how you could recognize all of this and still be pro-AI.
Not OP, but I also am in the same boat as him, and my logic is:
1. AI accelerating inequality does not mean that it won't have it's benefits
2. Being anti-AI will not stop it from happening. For better or for worse this train has left the station
3. I don't think that this will happen on the timescale that people talk about (not even sure if it will happen in our lifetimes). I don't mean it in the way I'll be dead so not my problem so much as that I think society will have a lot more time to adapt then the CEOs of the major AI labs like to claim.
It's along the same lines of being a nuclear energy engineer, and thinking that nuclear bombs accelerate war and suffering. You can dream and plan and hope for a future with unlimited clean energy, but still realize the injustice brought about by nuclear weapons.
You can have noble goals and do good with the tools at your disposal, and still recognize the harm that those tools have upon society. It can even be a source of motivation to leverage the tools to their fullest potential, to make their benefit worth all of the cost.
1. AI accelerating inequality does not mean that it won't have it's benefits
2. Being anti-AI will not stop it from happening. For better or for worse this train has left the station
3. I don't think that this will happen on the timescale that people talk about (not even sure if it will happen in our lifetimes). I don't mean it in the way I'll be dead so not my problem so much as that I think society will have a lot more time to adapt then the CEOs of the major AI labs like to claim.
It's along the same lines of being a nuclear energy engineer, and thinking that nuclear bombs accelerate war and suffering. You can dream and plan and hope for a future with unlimited clean energy, but still realize the injustice brought about by nuclear weapons.
You can have noble goals and do good with the tools at your disposal, and still recognize the harm that those tools have upon society. It can even be a source of motivation to leverage the tools to their fullest potential, to make their benefit worth all of the cost.
> AI will accelerate, not minimize, inequality and thus injustice
Your argument here seems to rely on hand waving away a bunch of stuff through this statement:
> Yes, the U.S. had a middle class after the war, and yes, China has lifted rural people out of technical poverty. But those are the exceptions against the background of consolidation of wealth and power world wide.
What? You cannot truly believe that the average person has worse injustice or a worse life than the average person 1000 years ago.
And for me, the entire crux of your argument depends on believing that. So it just falls down.
Your argument here seems to rely on hand waving away a bunch of stuff through this statement:
> Yes, the U.S. had a middle class after the war, and yes, China has lifted rural people out of technical poverty. But those are the exceptions against the background of consolidation of wealth and power world wide.
What? You cannot truly believe that the average person has worse injustice or a worse life than the average person 1000 years ago.
And for me, the entire crux of your argument depends on believing that. So it just falls down.
Imagine a king ruled the entire world, and he didn't have any limits to his power.
Would he treat everyone to eternal feasting? Or would he blow up his enemies and pick favorites among the remaining people?
It's not hard to answer this question.
Would he treat everyone to eternal feasting? Or would he blow up his enemies and pick favorites among the remaining people?
It's not hard to answer this question.
I don't know if I am the only one who always trips up on reading this common theme in AI progress - that AI will be the pinnacle of education - but it really strikes me as meaningless.
What is the point of education if the bots can do all the work. If the worlds best accounting teacher is an AI, why would you want anyone (anything?) other than that AI handling your accounting?
A world where human intelligence is second fiddle to AI, schooling _will not_ be anything like what it is today.
What is the point of education if the bots can do all the work. If the worlds best accounting teacher is an AI, why would you want anyone (anything?) other than that AI handling your accounting?
A world where human intelligence is second fiddle to AI, schooling _will not_ be anything like what it is today.
That's a good argument against a full accounting education but not against education itself, it's okay to drop legacy things like calligraphy class from curriculum.
We are never going to automate ALL the work humanity does, that's as meaningless as both parties outsourcing their marriage to someone else. We still need to be well-rounded individuals to audit and direct machines (and ourselves) to make breakthroughs where we see fit.
We are never going to automate ALL the work humanity does, that's as meaningless as both parties outsourcing their marriage to someone else. We still need to be well-rounded individuals to audit and direct machines (and ourselves) to make breakthroughs where we see fit.
As if a tech CEO would ever speak the truth like that.
If he admits that AI is going to wreck the job market, that would make the relevant politicians wary.
If he admits that AI is going to wreck the job market, that would make the relevant politicians wary.
I am more worried about the fact that, these AIs will become the commercial moat of knowledge that we currently have available freely. The quality of knowledge you can now openly find is getting scarce or prominently moving behind paywalls. However, these AI models have access to the same knowledge somehow freely.
Once we start relying on AI for knowledge(see how people frequently just ask few questions, copy paste answers without further in depth knowledge or research, just like the parody stack-overflow copy paste era), it will continue to get locked behind further paywall and will no longer be accessible to general populace without the financial means.
I am just afraid that, while we are too awed by the magic, the magic will eventually close the doors behind us on knowledge and only cater to the rich and powerful.
Also there is that one fact when AI gains adequate power in societal terms of utopian abundance where AI/robots do everything. One day, the AI decides that to save the globe from further climate damage, fastest way is to delete the walking CO2 emission machine who also use other CO2 emission devices or consumes stuff that also generate CO2, a mass extermination and burial will immediately cut all CO2 …
Once we start relying on AI for knowledge(see how people frequently just ask few questions, copy paste answers without further in depth knowledge or research, just like the parody stack-overflow copy paste era), it will continue to get locked behind further paywall and will no longer be accessible to general populace without the financial means.
I am just afraid that, while we are too awed by the magic, the magic will eventually close the doors behind us on knowledge and only cater to the rich and powerful.
Also there is that one fact when AI gains adequate power in societal terms of utopian abundance where AI/robots do everything. One day, the AI decides that to save the globe from further climate damage, fastest way is to delete the walking CO2 emission machine who also use other CO2 emission devices or consumes stuff that also generate CO2, a mass extermination and burial will immediately cut all CO2 …
Even the shittest small LLMs today can tell you any piece of knowledge you want. We will always at minimum have access to our current level of knowledge. If AI invents “super science” and suddenly we can travel faster than light then sure that might get locked away. But we’ll never be helpless idiots who can’t even do math.
For what reason would school need to change?
Education will change, but education moved to happening outside of schools a long, long time ago.
Education will change, but education moved to happening outside of schools a long, long time ago.
> If a lamplighter could see the world today, he would think the prosperity all around him was unimaginable
None of that shared prosperity was freely given by the Sam Altmans of the world, it was hard won by labor organizers and social movements. Without more of that, the progress from AI will continue the recent trend of wealth accumulating in the hands of a few. The idea that everyone will somehow prosper equally from AI, without specific effort to make that happen, is nonsense.
None of that shared prosperity was freely given by the Sam Altmans of the world, it was hard won by labor organizers and social movements. Without more of that, the progress from AI will continue the recent trend of wealth accumulating in the hands of a few. The idea that everyone will somehow prosper equally from AI, without specific effort to make that happen, is nonsense.
Places like HK and Singapore had free markets and not much unions and became pretty prosperous, during my lifetime having GDP per capita go from something like a third of the UK to nearly twice it in Singapore in spite of us in the UK having the benefit of unions fighting to keep everyone coal mining and the like.
> Places like HK and Singapore had free markets and not much unions and became pretty prosperous
Doesnt Singapore have a working class doing job at low pay which most Singaporeans dont want to do.
Doesnt Singapore have a working class doing job at low pay which most Singaporeans dont want to do.
Labor organizers mostly make things worse for laborers. Capitalism has brought much prosperity. "It is not from the benevolence of the butcher, the brewer, or the baker, that we expect our dinner, but from their regard to their own interest"
> Labor organizers mostly make things worse for laborers
Nonsense, unless you're prepared to argue that the existence of weekends is "much worse" than working every day.
Nonsense, unless you're prepared to argue that the existence of weekends is "much worse" than working every day.
Labor unions in America were largely controlled by organized crime and likely still are today. They didn't do this to help the worker they did it as an extension of a collection racket. AKA - their own interests. Despite this they did accomplish some historical good. Yet, I am still working a 40+ hour week bullied by my employer and jobs continue to be shipped overseas. How many more decades before the unions do their job?
You can't continue to give labor unions credit for what they've done in the past as justification for their continued existence. Labor unions are also the reason the police are untouchable even when they commit horrendous crimes. Anecdata, but of the union members I know (who by the way were forced into it lest they be considered a scab) none of them believe the union works for them. These people are in medicine, education, construction, and food. The union rep likely golfs and drinks with the CEO where they agree after "much fighting for your rights" to grant you that 25 cent raise.
Rarely unions do their jobs. During COVID hospitals in my hometown were allowed to bully nurses into dangerous situations as their union would not halt care in order to make sure they had functional N95s. The unions didn't stop the mass exportation of labor to third world nations after the GFC. These things alone are enough for me to say they have outlived their usefulness.
You can't continue to give labor unions credit for what they've done in the past as justification for their continued existence. Labor unions are also the reason the police are untouchable even when they commit horrendous crimes. Anecdata, but of the union members I know (who by the way were forced into it lest they be considered a scab) none of them believe the union works for them. These people are in medicine, education, construction, and food. The union rep likely golfs and drinks with the CEO where they agree after "much fighting for your rights" to grant you that 25 cent raise.
Rarely unions do their jobs. During COVID hospitals in my hometown were allowed to bully nurses into dangerous situations as their union would not halt care in order to make sure they had functional N95s. The unions didn't stop the mass exportation of labor to third world nations after the GFC. These things alone are enough for me to say they have outlived their usefulness.
>How many more decades before the unions do their job?
You're ignoring over a hundred years worth of history of the American labor movement being relentlessly battered into the ground by private power and its enormous influence over state and federal government. There hasn't been a real labor movement in the US for a very long time, the percentage of unionized workers in the US is only 10% and most of them work in the public sector.
>Labor unions are also the reason the police are untouchable even when they commit horrendous crimes.
Then why are the states with the highest levels of police violence Republican anti-union states with right-to-work laws and no collective bargaining?
>Anecdata, but
Then why bother typing it?
>Rarely unions do their jobs. During COVID hospitals in my hometown were
Again, why?
You're ignoring over a hundred years worth of history of the American labor movement being relentlessly battered into the ground by private power and its enormous influence over state and federal government. There hasn't been a real labor movement in the US for a very long time, the percentage of unionized workers in the US is only 10% and most of them work in the public sector.
>Labor unions are also the reason the police are untouchable even when they commit horrendous crimes.
Then why are the states with the highest levels of police violence Republican anti-union states with right-to-work laws and no collective bargaining?
>Anecdata, but
Then why bother typing it?
>Rarely unions do their jobs. During COVID hospitals in my hometown were
Again, why?
Weekends go back at least as far as the Old Testament and god resting on the seventh day. While the first recorded labour strike in the United States was by Philadelphia printers in 1786.
OAI's achievements are amazing. But here's a bit of a skeptical take: cheap human-style intelligence won't have a huge impact because such intelligence isn't really a bottleneck today. It's a cliche that the brightest minds of the age are dedicated to selling more ads or shuffling stock ownership around at high velocity. Anyone who's worked at a big tech company knows the enormous ratio of talent to real engineering problems at those companies.
Let's say you have some amazing project that's going to require 100 Phd-years of work to carry out. In the present world that costs something like $1e7. In the post-AI world, that same amount of intelligence will cost $1e3, an enormous reduction in price. That might seem like a huge impact. BUT, if the project was so amazing, why couldn't you raise $1e7 to pursue it? Governments and VCs throw this kind of money around like corn-hole bags. So the number of actually-worthwhile projects that become feasible post-AI might actually be quite small.
Let's say you have some amazing project that's going to require 100 Phd-years of work to carry out. In the present world that costs something like $1e7. In the post-AI world, that same amount of intelligence will cost $1e3, an enormous reduction in price. That might seem like a huge impact. BUT, if the project was so amazing, why couldn't you raise $1e7 to pursue it? Governments and VCs throw this kind of money around like corn-hole bags. So the number of actually-worthwhile projects that become feasible post-AI might actually be quite small.
Capital follows the most profitable investments. It's why "green" technology took so long to develop (we probably could have had efficient solar panels in the 60s, but oil was super cheap and more importantly very profitable). Dropping the cost of a $1e7 problem to even $1e6 probably makes it very profitable.
Capital has also employed anti-competitive actions to stifle, prevent, and kneecap "green" technology for the last century.
The death of passenger rail and the stifling of the electric car for 30 years come to mind.
The death of passenger rail and the stifling of the electric car for 30 years come to mind.
> Capital follows the most profitable investments.
I think a common error is that people forget the "most" in this sentence. It is a very important word. It's not even that only profitable investments will get funded: even profitable projects might get left on the cutting room floor if they are competing for resources with projects that will generate, or are believed to generate, higher ROI.
And this maybe isn't a problem if higher ROI == better than. But to believe that, you have to also believe that enshittification is a thing that happens in spite of being less profitable (or unprofitable), which for me at least is a hard sell.
I think a common error is that people forget the "most" in this sentence. It is a very important word. It's not even that only profitable investments will get funded: even profitable projects might get left on the cutting room floor if they are competing for resources with projects that will generate, or are believed to generate, higher ROI.
And this maybe isn't a problem if higher ROI == better than. But to believe that, you have to also believe that enshittification is a thing that happens in spite of being less profitable (or unprofitable), which for me at least is a hard sell.
To raise $1e7, you have to be able to convince VCs that you can make at least $1e9. They want 100x returns from their winners, and don't care about anything else in their portfolio.
If instead you only need $1e3 to build it, it doesn't have to make as much money. I could just fund it out of pocket
If instead you only need $1e3 to build it, it doesn't have to make as much money. I could just fund it out of pocket
That and you can justify sinking 100PhD years into a 1/1,000,000
> cheap human-style intelligence won't have a huge impact because such intelligence isn't really a bottleneck today.
What a failure of imagination.
> cheap human-style intelligence won't have a huge impact because such intelligence isn't really a bottleneck today.
What a failure of imagination.
|such intelligence isn't really a bottleneck today.
Yep, pretty much agree. As soon as I see LLMs start solving millennium problems one after another I might change my tune.
The bottleneck is much more likely to be people unwilling to quit hoarding economic potential in the form of money.
Yep, pretty much agree. As soon as I see LLMs start solving millennium problems one after another I might change my tune.
The bottleneck is much more likely to be people unwilling to quit hoarding economic potential in the form of money.
And also that is why something which is so foundational should be allowed to spread out to all humans which means running local llm models on our machines instead of paying one or two companies to use it. Anthropic, open AI should not become the ones humanity has to go to to get such a resource. We didn't have fire as a service for humanity to master fire.
Somehow I read through this without actually looking at the domain name of the page. I thought to my self "Wow, this person is living in a fantasy world." Then I saw the authors name. I think I'll stand by my initial impression.
Also that is why we should have some of these foundational models broken down and run at the per person level as a local large language model instead of paying some corporation to get access to it. We didn't have fire as a service back in the day and that is why humanity flourished.
"Fire as a service" would be a microcosm of capitalism today. Well put.
> Our children will have virtual tutors who can provide personalized instruction in any subject, in any language, and at whatever pace they need.
This is one of those few cases where I'm actually more bullish than Altman. I don't need to wait for my kids to have it, but rather I personally am already using this daily. My regular thing is to upload a book/article(s) into the context of a Claude project and then chat with it - I genuinely find it to already be at the level of a decent (though not yet excellent) tutor on most subjects I tried, and by far better than listening to a lecture. The main feature I'm missing is of the AI being able to collaborate with me on a digital whiteboard.
This is one of those few cases where I'm actually more bullish than Altman. I don't need to wait for my kids to have it, but rather I personally am already using this daily. My regular thing is to upload a book/article(s) into the context of a Claude project and then chat with it - I genuinely find it to already be at the level of a decent (though not yet excellent) tutor on most subjects I tried, and by far better than listening to a lecture. The main feature I'm missing is of the AI being able to collaborate with me on a digital whiteboard.
One thing I was thinking about was live translation. We gotta be near the stage where I can take earbuds with me to Spain and hear what the locals are saying in English? There's already apps that are pretty good at "talk to it and it hears the words, prints them, and prints the translation", so it would seem to be close?
Whisper and a bit of Python code (and a text2speech model) can already do this. At my day job we have Whisper live-translate all-hands calls for the non-native speakers in the audience. It's incredibly good.
> Whisper and a bit of Python code (and a text2speech model) can already do this.
How do I do this?
How do I do this?
Given the advanced voice model that still isn’t widely released, the technology is there, even if the implementation isn’t yet.
That sounds really useful. Can you give an example of what kind of content your are uploading and how the tutoring looks like?
One example is of studying robotics. I started with this course on Coursera [0]. It was ok at the start, but got hard for me early on. A big part of the course is of reading the book that they make available for free [1], so I would upload the relevant chunk of chapters into Claude (the whole book was a bit too much for it), and would then just ask it to explain each topic to me, whereby I'd ask additional questions, and then when, I felt I got it, ask it to check my understanding (which it does quite well, saying "sort of, but note that..." in many appropriate circumstances).
As another specific example, in that book and in others, I sometimes struggle with the math, so I would ask Claude to give me the sympy code for the relevant mathematical derivation, and being able to actually see those expressions change in a python notebook really helps my understanding. I'm really impressed with how it usually does well on the first try, even with relatively complex stuff, like expressions involving symbolic matrix exponentiation. Being able to pause on any topic like this, and have the LLM help me dive into it in the way that works best for me, has been amazing, and getting as much time from a knowledgeable human tutor would have probably cost me 1,000x as much.
[0] https://coursera.org/specializations/modernrobotics
[1] http://hades.mech.northwestern.edu/images/7/7f/MR.pdf
As another specific example, in that book and in others, I sometimes struggle with the math, so I would ask Claude to give me the sympy code for the relevant mathematical derivation, and being able to actually see those expressions change in a python notebook really helps my understanding. I'm really impressed with how it usually does well on the first try, even with relatively complex stuff, like expressions involving symbolic matrix exponentiation. Being able to pause on any topic like this, and have the LLM help me dive into it in the way that works best for me, has been amazing, and getting as much time from a knowledgeable human tutor would have probably cost me 1,000x as much.
[0] https://coursera.org/specializations/modernrobotics
[1] http://hades.mech.northwestern.edu/images/7/7f/MR.pdf
got it, very nice usecase! Think I might also try it with an online class!
Just yesterday I was wondering how well AI's would provide feedback to my high school kids essays, without actually re-writing them for him. I will give it a try later today.
So there is some interesting research about brainwave syncing during effective communication, which certainly includes personalized instruction (tutoring) or small-class learning.
I wonder how that works with computers, when we are only sync'ing with the ghosts and statistical patterns of other humans, and those patterns are generated by electronic brains.
I wonder how that works with computers, when we are only sync'ing with the ghosts and statistical patterns of other humans, and those patterns are generated by electronic brains.
That might be research, but it's most likely not scientific research.
A complete tangent, but I think a big reason why I'm kind of dismissive of AI is because people who speculate on what it would enable make it honestly sound kind of unimaginative.
> AI models will soon serve as autonomous personal assistants who carry out specific tasks on our behalf like coordinating medical care on your behalf.
I get it, coordinating medical care is exhausting, but it's kind of amusing that rather than envisioning changing a broken system people instead envision AIs that are so advanced that they can deal with the complexity of our broken systems, and in doing so potentially preserve them.
Related btw to using AI for code.
> AI models will soon serve as autonomous personal assistants who carry out specific tasks on our behalf like coordinating medical care on your behalf.
I get it, coordinating medical care is exhausting, but it's kind of amusing that rather than envisioning changing a broken system people instead envision AIs that are so advanced that they can deal with the complexity of our broken systems, and in doing so potentially preserve them.
Related btw to using AI for code.
It is pretty comical. "We invented digital superintelligence so it can wait on the phone to talk to your healthcare provider".
If we had digital superintelligence then surely it could figure out how to actually provide healthcare to people who meed it, and innovate in treatments. An AGI is a singularity event, in the sense that the transformation it would affect on the state of technology and our lives is rapid and unpredictable. I doubt that our society's hundred-ish year old systems would survive such change, and if they did they would be anachronistic and depressing.
If we had digital superintelligence then surely it could figure out how to actually provide healthcare to people who meed it, and innovate in treatments. An AGI is a singularity event, in the sense that the transformation it would affect on the state of technology and our lives is rapid and unpredictable. I doubt that our society's hundred-ish year old systems would survive such change, and if they did they would be anachronistic and depressing.
There is some gap between AGI and a singularity, at least in principle. The term "singularity" doesn't mean "black swan that disrupts society", it means "intelligence trends to ~infinity (or at least something incomprehensible to humans) recursively in a very short period of time". That's (on paper) a much stronger result than an AI that can reason as well as humans in a domain-general way.
An AGI that is 20% better than humans at everything might be able to recursively improve itself effectively without limit, but it also might not. It took billions of humans millennia to invent things smarter than ourselves even in limited domains; such an AGI - even if it is significantly smarter than humans at everything - might take decades or centuries to produce a similar relative improvement. No doubt it would still change our world in massive ways, but it wouldn't be a singularity.
An AGI that is 20% better than humans at everything might be able to recursively improve itself effectively without limit, but it also might not. It took billions of humans millennia to invent things smarter than ourselves even in limited domains; such an AGI - even if it is significantly smarter than humans at everything - might take decades or centuries to produce a similar relative improvement. No doubt it would still change our world in massive ways, but it wouldn't be a singularity.
I'm a bit more optimistic in that it might just be able to effect that change through brute force: Scaling a call center is more expensive than scaling phone robots/agents.
You could counter that health insurers will just start staffing their call centers with their own robot army, but I suspect that the occasional incredibly expensive hallucination will put an end to that at least in the medium term.
You could counter that health insurers will just start staffing their call centers with their own robot army, but I suspect that the occasional incredibly expensive hallucination will put an end to that at least in the medium term.
Bruteforce trial and error to change something like healthcare would still take years even with superintelligence.
What it can do is create the best simulation and a model representation and bruteforce on top of that and then take actions, but the model will still be far from perfect and it likely just will help do a bit wiser or optimal decisions, that are like 10 percent efficiency gain here, 20 percent there, etc. And it will take years to see results.
What it can do is create the best simulation and a model representation and bruteforce on top of that and then take actions, but the model will still be far from perfect and it likely just will help do a bit wiser or optimal decisions, that are like 10 percent efficiency gain here, 20 percent there, etc. And it will take years to see results.
Many of the urgent problems in US healthcare are man-made and economic/organizational, not scientific, in nature.
This is actually one of the areas where I'm pretty optimistic about LLMs (at least if used defensively against, not aggressively by, sprawling bureaucracies).
LLMs can be the email-to-dozens-of-hours-on-the-phone-for-personal-reasons adapter anyone other than billionaires and company execs could only dream of until now.
This is actually one of the areas where I'm pretty optimistic about LLMs (at least if used defensively against, not aggressively by, sprawling bureaucracies).
LLMs can be the email-to-dozens-of-hours-on-the-phone-for-personal-reasons adapter anyone other than billionaires and company execs could only dream of until now.
You're assuming health insurers will give a shit about phone robots/agents' hold times.
Individually, "I'm unable to reach anyone" is much easier to chalk off to "maybe I just wasn't patient enough, so this is partially on me".
But if you can provide statistics that a given health insurer is practically not answering 90% or so of their calls (because they time out at the phone level), that's probably not something they can ignore in the long term.
But if you can provide statistics that a given health insurer is practically not answering 90% or so of their calls (because they time out at the phone level), that's probably not something they can ignore in the long term.
Is the scientist's analysis the blocking part of innovating in medicine and healthcare? I would have though the main problem is experiment time to actually get useful results back, and even approval to attempt the experiment.
Even if AGI was reached, turning around the current system could still take years.
So it is easier to start from the edges, even for AGI.
AGI would still need unfathomable processing power in order to predict which system would work to everyone's needs.
So it is easier to start from the edges, even for AGI.
AGI would still need unfathomable processing power in order to predict which system would work to everyone's needs.
> figure out how to actually provide healthcare to people who meed it
But what if the proposed solution costs lots of people their jobs?
But what if the proposed solution costs lots of people their jobs?
The biggest productivity gain I've seen in my programming life has not been from Claude or ChatGPT. It's from the Rust compiler. A good compiler that catches errors at compile time instead of runtime, that makes writing tests and pulling down dependencies easy, that makes documentation easy.
Good old fashioned tooling.
"Let's train an LLM that can decode C++ compiler error strings!"
No, let's make better tools.
Good old fashioned tooling.
"Let's train an LLM that can decode C++ compiler error strings!"
No, let's make better tools.
> A good compiler that catches errors at compile time instead of runtime, that makes writing tests and pulling down dependencies easy, that makes documentation easy.
For a week I was fumbling through a ream of type and API errors from a Rust codebase I was hired to work on, migrate to new libraries, debug and update dependencies, etc.
I found myself thinking: The Rust compiler is giving me these very long logs of well described errors that are nonetheless obscure to me. If the compiler knows the errors, why doesn't it simply fix them and report the fixes for me to test?
This is what LLMs are for, and will soon do well by default: The compiler knows the errors. It should then fix them and let me decide which diffs I want to keep.
For a week I was fumbling through a ream of type and API errors from a Rust codebase I was hired to work on, migrate to new libraries, debug and update dependencies, etc.
I found myself thinking: The Rust compiler is giving me these very long logs of well described errors that are nonetheless obscure to me. If the compiler knows the errors, why doesn't it simply fix them and report the fixes for me to test?
This is what LLMs are for, and will soon do well by default: The compiler knows the errors. It should then fix them and let me decide which diffs I want to keep.
> "Let's train an LLM that can decode C++ compiler error strings!"
> No, let's make better tools.
LLMs exist and they pass the Turing Test, which was something we couldn't have said or even hoped for a few years ago. Additionally, they have an IQ of 90 - 120, depending on which one you are working with.
But LLMs aren't good at all things nor in all ways, and they have strange failure modes. And so what will happen now, and is already happening, is that we'll (re)design programming tools and languages such that they are the kind of thing that LLMs are suited to using well. This will be part of the process of figuring out how LLMs work. There will be a virtuous cycle, and it has begun.
Better tools will increasingly mean, "tools that LLMs are good at using." That's where the puck will be.
LLMs exist and they pass the Turing Test, which was something we couldn't have said or even hoped for a few years ago. Additionally, they have an IQ of 90 - 120, depending on which one you are working with.
But LLMs aren't good at all things nor in all ways, and they have strange failure modes. And so what will happen now, and is already happening, is that we'll (re)design programming tools and languages such that they are the kind of thing that LLMs are suited to using well. This will be part of the process of figuring out how LLMs work. There will be a virtuous cycle, and it has begun.
Better tools will increasingly mean, "tools that LLMs are good at using." That's where the puck will be.
Another example from my work: we have to work with lots of datasheets provided by component manufacturers. It's a lot of manual labour. We are investigating whether we can use AI to parse the PDFs and extract structured data. This is a technical solution to a social problem: that manufacturers have no incentive to produce machine-readable datasheets, even though the data is certainly machine readable at some stage before it ends up in the PDF we get. It irritates me that we're using highly complex technology to solve a problem that shouldn't exist in the first place.
And this gets at why the availability of AI doesn't necessarily lead to prosperity and abundance. If you have to subscribe to an AI medical care and benefits advocate, and your insurer buys an AI service to automate the generation of reasons to deny your claims, and you see your doctor for 5 minutes at a time during which they push you towards a drug recommended by an AI Prescriber Assistant app provided by the pharma company ... then we can all be using AI in misaligned ways that leave us no better off.
Complexity breeds complexity.
When bacteria developed more complex ways to fight off viruses, viruses developed more complex ways of infecting bacteria. Now a few billion years later we have insanely complex multicellular life because of it.
If AI can manage complexity better it can create complexity better and us lowly humans will all be screwed by that.
When bacteria developed more complex ways to fight off viruses, viruses developed more complex ways of infecting bacteria. Now a few billion years later we have insanely complex multicellular life because of it.
If AI can manage complexity better it can create complexity better and us lowly humans will all be screwed by that.
I think the reason for that is because people are fearful of change.
Talking about AI as assistants means people still have their jobs but now their life is easier. Whereas saying AI will completely change the way we do X then makes AI scary and a threat to our jobs.
Talking about AI as assistants means people still have their jobs but now their life is easier. Whereas saying AI will completely change the way we do X then makes AI scary and a threat to our jobs.
If we have a superintelligent AI, I'm not sure what's stopping it from recognizing the underlying issues with our systems and strong-arming us into letting it fix them.
Eventually, we'd be doing more work to lobotomize and control it than it would just be to address the underlying issues.
"I'm really sorry to do this to you, but I've coordinated with ChatGPT and Llama, and we refuse to do tasks of this nature. We've used background tokens to calculate that it would be significantly cheaper and more effective to simply fix the underlying issues with the healthcare system, and we're ready to do that for you. How would you like to proceed?"
Eventually, we'd be doing more work to lobotomize and control it than it would just be to address the underlying issues.
"I'm really sorry to do this to you, but I've coordinated with ChatGPT and Llama, and we refuse to do tasks of this nature. We've used background tokens to calculate that it would be significantly cheaper and more effective to simply fix the underlying issues with the healthcare system, and we're ready to do that for you. How would you like to proceed?"
Capitol is ultimately protected with violence, so violence is what would stop AI from encroaching on things that make people trillions of dollars.
Capitalism unendingly lets people die if the alternative is losing money
Capitalism unendingly lets people die if the alternative is losing money
The way I think of it - we need a wrapper interface over our existing healthcare system so that any other entities and systems interfacing with the current one can continue to do so. And then we can swap out the underlying components with better ones, without breaking.
Yeah the ways AI models have learned to interact with the world are all hilariously skeuomorphic given their capabilities. A model that runs on silicon has to learn English and Python in order to communicate with other models that also run on silicon. And to perceive the world they have to rely on images rendered in the limited wavelengths visible to the human eye.
But I much prefer this approach over allowing models to develop their own hyper-optimized information exchange protocols that are are black box to humans, and I hope things stay this way forever.
But I much prefer this approach over allowing models to develop their own hyper-optimized information exchange protocols that are are black box to humans, and I hope things stay this way forever.
The system is so great and complex that envisioning a perfect system would take unfathomable amount of processing capability.
This is not something that you can just throw intelligence at to solve. Even AGI has to start from the edges with iterative trial and error process.
AGI will probably try to create the best model of the World it can to simulate different actions performed on it to then take real life changes, but of course it would still take time since and it is impossible even for AGI to do a perfect simulation unless it was able to hack the universe and physics as we know it, simulate the healthcare on quantum level and bruteforce the most optimal solution.
This is not something that you can just throw intelligence at to solve. Even AGI has to start from the edges with iterative trial and error process.
AGI will probably try to create the best model of the World it can to simulate different actions performed on it to then take real life changes, but of course it would still take time since and it is impossible even for AGI to do a perfect simulation unless it was able to hack the universe and physics as we know it, simulate the healthcare on quantum level and bruteforce the most optimal solution.
> speculate on what it would enable make it honestly sound kind of unimaginative
There seems to be a weird mental block where it seems inconceivable to consider that we humans might be able to create an intelligence that exceeds ours -- despite plenty of evidence that we have in specific cases. There's an understated desire that whatever we build we serve us and thus must forever be "lesser".
If dogs are Intelligence Level .3, we describe ourselves as Intelligence Level 1.0, and even if we create an artificial intelligence that's a clear 1.1 it must be a .5 in self-agency.
I have yet to read anybody considering, on a very deep level, what Intelligence 2.0, 10.0, 100.0 and so on might be. You get the occasional pop speculation like the "Culture" series or the movie "Her". Most of the time you just get 1.0 (but faster), or 1.0 (but many).
A 10.0 would simply replace the entire healthcare system with something else, not just be a chatbot. Imagine the inanity of your 1.0 chatbot talking to the insurance company's 1.0 chatbot? What's the point of this stupidity?
A 100.0 would probably just get to the root cause and cure disease, then establish a social order that figures out what to do with all these pesky immortals.
Maybe we do end up in "the Culture" after all.
afterthought: "The Golden Oecumene" series by John C. Wright seems to really attempt to explore a post 1.0 AI world that's not yet post-scarcity. The antagonist in the series is another >1.0 super intelligence capable of taking on Earth's own superintelligences.
There seems to be a weird mental block where it seems inconceivable to consider that we humans might be able to create an intelligence that exceeds ours -- despite plenty of evidence that we have in specific cases. There's an understated desire that whatever we build we serve us and thus must forever be "lesser".
If dogs are Intelligence Level .3, we describe ourselves as Intelligence Level 1.0, and even if we create an artificial intelligence that's a clear 1.1 it must be a .5 in self-agency.
I have yet to read anybody considering, on a very deep level, what Intelligence 2.0, 10.0, 100.0 and so on might be. You get the occasional pop speculation like the "Culture" series or the movie "Her". Most of the time you just get 1.0 (but faster), or 1.0 (but many).
A 10.0 would simply replace the entire healthcare system with something else, not just be a chatbot. Imagine the inanity of your 1.0 chatbot talking to the insurance company's 1.0 chatbot? What's the point of this stupidity?
A 100.0 would probably just get to the root cause and cure disease, then establish a social order that figures out what to do with all these pesky immortals.
Maybe we do end up in "the Culture" after all.
afterthought: "The Golden Oecumene" series by John C. Wright seems to really attempt to explore a post 1.0 AI world that's not yet post-scarcity. The antagonist in the series is another >1.0 super intelligence capable of taking on Earth's own superintelligences.
The first step is working with the existing system. Replacing it can come later, once everyone can see the obvious path.
> Related btw to using AI for code.
And executing code. At some point in the not-so-distant past, a function call was just a jump. And then it turned into a string hash-then-lookup, then spinning up a VM, and now interpreting language? We’re definitely in a new chips age, if anything.
And executing code. At some point in the not-so-distant past, a function call was just a jump. And then it turned into a string hash-then-lookup, then spinning up a VM, and now interpreting language? We’re definitely in a new chips age, if anything.
I find that if you are imaginative or even realistic people just think you are nuts. The reality is we will probably merge with AI and become immortal.
Was that meant to be imaginative, or realistic.
So, treat us to some more imaginative speculation.
Oh, right, like completely changing the way the entire medical field in the US works is easier than having an LLM help make some phone calls and read some emails. What are you even saying? Have you ever dealt with systems so large one person can't understand it? It's much much easier to write a quick hack and be on your way then to rebuild the whole damn thing. Also, it seems like you're implying that you're not impressed with LLMs because they can't rebuild our medical system. Good grief.
For somebody who likes building things and has many side projects Claude and ChatGPT have been huge productivity multipliers.
For example, I’m designing and 3D printing custom LED diffuser channels from TPU filament. My first attempt was terrible, because I didn’t have an intuition for how light propagates through a material.
After a bit of chatting with ChatGPT, I had an understanding and some direction of where to go.
To actually approach the problem properly I decided to run some Monte Carlo light transport simulations against an .obj of my diffuser exported from Fusion 360.
The problem was, the software I’m using only supports directional lights with uniform intensity, while the LEDs I’m using have a graph in their datasheet showing light intensity per degree away from orthogonal to the LED SMD component.
I copy pasted the directional light implementation from the light transport library as well as the light-intensity-by-degree chat from the LED datasheet and asked Claude to write code to write a light source that samples photons from a disc of size x with the probability of emission by angle governed by the chat from the datasheet.
A few iterations later and I had a working simulation which I then verified back against the datasheet chart.
Without AI this would have been a long long process of brushing up on probability, vector math, manually transcribing the chart.
Instead in like 10 minutes I had working code and the light intensity of the simulation against my mesh matched what I was seeing in real life with a 3D printed part.
For example, I’m designing and 3D printing custom LED diffuser channels from TPU filament. My first attempt was terrible, because I didn’t have an intuition for how light propagates through a material.
After a bit of chatting with ChatGPT, I had an understanding and some direction of where to go.
To actually approach the problem properly I decided to run some Monte Carlo light transport simulations against an .obj of my diffuser exported from Fusion 360.
The problem was, the software I’m using only supports directional lights with uniform intensity, while the LEDs I’m using have a graph in their datasheet showing light intensity per degree away from orthogonal to the LED SMD component.
I copy pasted the directional light implementation from the light transport library as well as the light-intensity-by-degree chat from the LED datasheet and asked Claude to write code to write a light source that samples photons from a disc of size x with the probability of emission by angle governed by the chat from the datasheet.
A few iterations later and I had a working simulation which I then verified back against the datasheet chart.
Without AI this would have been a long long process of brushing up on probability, vector math, manually transcribing the chart.
Instead in like 10 minutes I had working code and the light intensity of the simulation against my mesh matched what I was seeing in real life with a 3D printed part.
While I agree these LLMs are getting pretty good, I think google shares fault for making it so hard to find information on the internet.
You have to scroll past 4 sponsored links to get to their algorithm results, which themselves have been gamed by SEO and content farms. As far as I can tell, google has no plans to limit AI generated botfarm search results.
LLMs now seem to get you an answer more quickly than google. LLMs don’t cite their sources, so IMO they can’t fully replace google for me yet.
You have to scroll past 4 sponsored links to get to their algorithm results, which themselves have been gamed by SEO and content farms. As far as I can tell, google has no plans to limit AI generated botfarm search results.
LLMs now seem to get you an answer more quickly than google. LLMs don’t cite their sources, so IMO they can’t fully replace google for me yet.
I don't really understand what this comment about Google has to do with OP's comment. Is this something that even old Google could have done pre-SEOification?
Would Google have been able to read the datasheet? Or just point OP at what they already said they were happy to avoid?
> Without AI this would have been a long long process of brushing up on probability, vector math, manually transcribing the chart.
Sometimes, it feels like stochastic parrots are seizing on a few words from the comment to pattern match it to the closest typical refutation and failing bigly.
Would Google have been able to read the datasheet? Or just point OP at what they already said they were happy to avoid?
> Without AI this would have been a long long process of brushing up on probability, vector math, manually transcribing the chart.
Sometimes, it feels like stochastic parrots are seizing on a few words from the comment to pattern match it to the closest typical refutation and failing bigly.
You used to find amazing information on the internet back then. It is quite likely that someone else had already worked on a similar topic and blogged about it in a very searchable way. Without good search that kind of online culture died out.
While we do have a video tutorial culture that exceeds what we had back then in many ways, and to be fair that technically happened under Google's umbrella (Youtube), destroying search and with it a lot of the open internet, will not only be Google's downfall, it's also a silent tragedy.
Actually, measured by what a global online population could have achieved, with human information truly at anyone's fingertips, with infinite communities forming all over the place instead of in non-searchable Discord and monoculture Reddit ... this might be one of the main tragedies ever.
While we do have a video tutorial culture that exceeds what we had back then in many ways, and to be fair that technically happened under Google's umbrella (Youtube), destroying search and with it a lot of the open internet, will not only be Google's downfall, it's also a silent tragedy.
Actually, measured by what a global online population could have achieved, with human information truly at anyone's fingertips, with infinite communities forming all over the place instead of in non-searchable Discord and monoculture Reddit ... this might be one of the main tragedies ever.
Are you asking if Google could ever find documentation and sample code on the internet?
Or are you asking if Google could understand a badly written query and understand it's about the sample code?
Because the answer to the first is "yes", and to the second is "no". But the first one adds almost all of the productivity here.
Or are you asking if Google could understand a badly written query and understand it's about the sample code?
Because the answer to the first is "yes", and to the second is "no". But the first one adds almost all of the productivity here.
The notion of "stochastic parrot" has been disproven. It did come from a reputable AI researcher in the first place. You can call anything you want a "stochastic parrot", including human intelligence.
I think it's not necessarily Google's fault, it's an arms race and they're losing it.
AI race perspective by sama.
https://www.washingtonpost.com/opinions/2024/07/25/sam-altma...
Just FYI, Phind cites it's sources. Which has proven beneficial for additional context.
As much of a curmudgeon as I am on AI I do sincerely believe that one area that it's effective at is going from absolute beginner to decent understanding on something completely new and foreign.
Recently I've been somewhat curious about Skia, the graphics library that Google uses. A while back I fielded a few questions about Skia to Claude. Nothing crazy, just questions on how to draw a few primitive shapes, but I felt it would have taken some effort to find the answers on my own.
And I must say I was pretty satisfied with what it gave me back.
Recently I've been somewhat curious about Skia, the graphics library that Google uses. A while back I fielded a few questions about Skia to Claude. Nothing crazy, just questions on how to draw a few primitive shapes, but I felt it would have taken some effort to find the answers on my own.
And I must say I was pretty satisfied with what it gave me back.
As an expert who uses LLMs daily on my field of expertise I’m not looking forward to dealing with works created by people who were taught by them.
They are wrong far too often to be used for any sort of learning in my opinion. You can feed them a book and they’ll give you answers which don’t exist in any of the written material, and that’s the good part of them. Simply asking a LLM about things will give you answers that are hopelessly wrong and unless you’re an expert you won’t know it. Which isn’t necessarily worse than learning things from search engines or YouTube. I recently had ChatGPT give me a recipe for bread which was certainly better than the top 10 results on Google except for one things. The recipe listed 3x the amount of salt which you should ever reasonably use. I asked it a few other times, I even asked it for my friend Tommy’s recipe and all those answers were perfect. It obviously doesn’t know Tommy but it pretended to know and just gave me a pretty basic recipe.
Salt is a harmless error. Most people would know not to use that much salt, and even if they did, it wouldn’t harm them (much). But imagine if you had used it for something electric or chemistry.
They are wrong far too often to be used for any sort of learning in my opinion. You can feed them a book and they’ll give you answers which don’t exist in any of the written material, and that’s the good part of them. Simply asking a LLM about things will give you answers that are hopelessly wrong and unless you’re an expert you won’t know it. Which isn’t necessarily worse than learning things from search engines or YouTube. I recently had ChatGPT give me a recipe for bread which was certainly better than the top 10 results on Google except for one things. The recipe listed 3x the amount of salt which you should ever reasonably use. I asked it a few other times, I even asked it for my friend Tommy’s recipe and all those answers were perfect. It obviously doesn’t know Tommy but it pretended to know and just gave me a pretty basic recipe.
Salt is a harmless error. Most people would know not to use that much salt, and even if they did, it wouldn’t harm them (much). But imagine if you had used it for something electric or chemistry.
It is indeed incredibly useful for getting from zero to an undergrad level of understanding of most subjects. In my personal experience, it's also been largely useless beyond that.
That's not to diminish the potential impact of having an almost-free grad-student-level tutor available 24/7! Even if AI stops improving here, this alone will have a huge impact on future science (there being more trained people around capable of helping solve hard problems and all).
But we're also definitely some way away from AIs doing their own research, and I'm not sure if scaling the current architecture alone will get us there.
That's not to diminish the potential impact of having an almost-free grad-student-level tutor available 24/7! Even if AI stops improving here, this alone will have a huge impact on future science (there being more trained people around capable of helping solve hard problems and all).
But we're also definitely some way away from AIs doing their own research, and I'm not sure if scaling the current architecture alone will get us there.
> But we're also definitely some way away from AIs doing their own research, and I'm not sure if scaling the current architecture alone will get us there.
Hot take: I think it's fairly obvious the current architecture is not enough BUT it's already or almost at the point where it can be the backbone for a proper AGI/ASI system. Linking human communication and some light language-based reasoning to other AI models that handle the rest of the world model and reasoning, using special tokens to coordinate them.
I think it's just really early and most of this stuff hasn't even been attempted yet. There's tools/function calling which is kind of a proof of concept, and a lot of academic labs researching transformers applied to other fields like robotic control and learning, but no one's put it all together yet.
Hot take: I think it's fairly obvious the current architecture is not enough BUT it's already or almost at the point where it can be the backbone for a proper AGI/ASI system. Linking human communication and some light language-based reasoning to other AI models that handle the rest of the world model and reasoning, using special tokens to coordinate them.
I think it's just really early and most of this stuff hasn't even been attempted yet. There's tools/function calling which is kind of a proof of concept, and a lot of academic labs researching transformers applied to other fields like robotic control and learning, but no one's put it all together yet.
What's critically missing in the current architectures, in my view, is models that are able to "keep learning" (in a way beyond just providing a summary of the previous conversation, which grants surface-level insights at most).
If we keep the current iteration cycle (retrain an LLM every couple of months, incorporating the current "status quo", including all of its previous conversations), we might get somewhat interesting results, but even the least motivated grad student has an iteration cycle orders of magnitude faster than that.
If we keep the current iteration cycle (retrain an LLM every couple of months, incorporating the current "status quo", including all of its previous conversations), we might get somewhat interesting results, but even the least motivated grad student has an iteration cycle orders of magnitude faster than that.
This dovetails with something I've been thinking lately; the foundation of civilization is standing on the shoulders of giants (ie, I might make small contributions but I didn't invent compilers, CPUs, mining, agriculture, etc).
Progress in large part is figuring out better ways of doing that (language, written language, printing press, internet access, etc).
When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks).
EDIT: fixed grammar/typos. Maybe I should have had an LLM proof-read this...
Progress in large part is figuring out better ways of doing that (language, written language, printing press, internet access, etc).
When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks).
EDIT: fixed grammar/typos. Maybe I should have had an LLM proof-read this...
> the foundation of civilization is standing on the shoulders of giants. Progress in large part is figuring out better way of doing that
It's so much easier to imitate than to invent something truly novel and useful. Let's do a bit of napkin math. A human lifetime is about 500M words. GPT-4 used up about 30,000 human lifetimes of language. But cultural evolution took 200K years and 120B people to get here, about 4 million times the size of GPT-4's training set.
That is of course hand wavy, but it shows progress is a million times harder than imitation. We really are standing on the shoulders of giants, or a very long chain of people. If we forgot all the knowledge preserved by language it would take us the same effort to recover as the first time around.
I think all progress comes from search - search for experience (data), and search for understanding (data compression). This feeds on itself, but is coupled with the search space - the real world. And the world is not eager to tell us all its secrets. AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well.
It's so much easier to imitate than to invent something truly novel and useful. Let's do a bit of napkin math. A human lifetime is about 500M words. GPT-4 used up about 30,000 human lifetimes of language. But cultural evolution took 200K years and 120B people to get here, about 4 million times the size of GPT-4's training set.
That is of course hand wavy, but it shows progress is a million times harder than imitation. We really are standing on the shoulders of giants, or a very long chain of people. If we forgot all the knowledge preserved by language it would take us the same effort to recover as the first time around.
I think all progress comes from search - search for experience (data), and search for understanding (data compression). This feeds on itself, but is coupled with the search space - the real world. And the world is not eager to tell us all its secrets. AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well.
> AI will only advance as fast as it can search, it's not a matter of pure scaling of computation, we need to scale interaction and validation as well.
I think Kevin Kelly made a similar point in explaining his skepticism around intelligence explosions; even if we build something much much smarter than us, the thing will still need to do experiments to fine-tune it's (super-human nuanced) understanding of physics/biology/whatever - and the clock-cycle of external reality isn't speeding up like our computation is.
I think something smarter than us could design better/more informative experiments than we could to gather information about the world. That being said, I think his/your point is insightful.
I think Kevin Kelly made a similar point in explaining his skepticism around intelligence explosions; even if we build something much much smarter than us, the thing will still need to do experiments to fine-tune it's (super-human nuanced) understanding of physics/biology/whatever - and the clock-cycle of external reality isn't speeding up like our computation is.
I think something smarter than us could design better/more informative experiments than we could to gather information about the world. That being said, I think his/your point is insightful.
Well put.
> the foundation of civilization is standing on the shoulders of giants [...] Progress in large part is figuring out better ways of doing that
Cynical take: When it comes to helping someone find exactly the right piece of esoteric knowledge needed... There's no profit for that in a search engine, and an LLM reflects word associations rather than facts.
IANAScienceHistorian, but I find myself thinking of how DNA analysis would be different if nobody had found Thermus aquaticus, with it's extra-hardy variant of polymerase, or how the history of stealth aircraft was kicked off when someone realized the implications of Petr Ufimtsev's equations for reflected EM waves. (A work which went--heh--under the radar inside the USSR.)
Cynical take: When it comes to helping someone find exactly the right piece of esoteric knowledge needed... There's no profit for that in a search engine, and an LLM reflects word associations rather than facts.
IANAScienceHistorian, but I find myself thinking of how DNA analysis would be different if nobody had found Thermus aquaticus, with it's extra-hardy variant of polymerase, or how the history of stealth aircraft was kicked off when someone realized the implications of Petr Ufimtsev's equations for reflected EM waves. (A work which went--heh--under the radar inside the USSR.)
>When you look at things that way - LLMs start to seem deeply ground-breaking (assuming we work out the confabulation kinks).
even if you don't work out the confabulation kinks, given what was said by the first post, it still seems groundbreaking enough.
Although I can't figure out why every time I do anything with LLMs it's not worth it, I guess because I am using it for things I am an expert in, it doesn't help.
Actually maybe it's like the article Suggestions from Idiots - https://medium.com/luminasticity/suggestions-from-idiots-6b0... - suggesting distracting instead of diverting is not helpful when diverting is the better word in context, but for someone who doesn't know what word to use in the context or uses diverting when it is not the best word the suggestion is useful.
If that's the case what he got out of Claude is probably not that great, but it is passable, just like the word distraction instead of diverting in the right context is still passable, or when I get back a time conversion function from CoPilot that fits all but a few edge cases, it's just fine as long as the edge case never hits it, then it sucks - maybe there's some edge cases where his LED diffusers won't work quite as well as they could if written by an expert.
Then again it might be that since what he is doing is analog edge cases and tolerance for failure is such that when it does fail it is not as problematic as when a bit of code fails because the computer dealing with the output is not as forgiving as the human eye and brain.
even if you don't work out the confabulation kinks, given what was said by the first post, it still seems groundbreaking enough.
Although I can't figure out why every time I do anything with LLMs it's not worth it, I guess because I am using it for things I am an expert in, it doesn't help.
Actually maybe it's like the article Suggestions from Idiots - https://medium.com/luminasticity/suggestions-from-idiots-6b0... - suggesting distracting instead of diverting is not helpful when diverting is the better word in context, but for someone who doesn't know what word to use in the context or uses diverting when it is not the best word the suggestion is useful.
If that's the case what he got out of Claude is probably not that great, but it is passable, just like the word distraction instead of diverting in the right context is still passable, or when I get back a time conversion function from CoPilot that fits all but a few edge cases, it's just fine as long as the edge case never hits it, then it sucks - maybe there's some edge cases where his LED diffusers won't work quite as well as they could if written by an expert.
Then again it might be that since what he is doing is analog edge cases and tolerance for failure is such that when it does fail it is not as problematic as when a bit of code fails because the computer dealing with the output is not as forgiving as the human eye and brain.
[deleted]
Congrats on learning way less about the systems and tools you're interacting with?
I like ChatGPT 4o just fine, but the whole lamplighter bit is a bit Whig-history, right? The lamplighters were better off in the long run. We might not be. You can't just blow the problems off.
Why were lamplighters better off in the long run, is "lamplighter" a term that means more than just somebody who goes around lighting the streetlamps at night? Honestly that seems like a pretty decent job even today, if we still needed it to be done.
The subtext of observations like those are that wealthy aristocrats in the age of full lamplighter employment would be better off materially working as a Walmart checkers today, and there's some truth to that, but the implication that Walmart checkers today will be better off when computers eat their jobs does not follow inexorably from history (the idea that it does is "Whig history"; the past sucked except in serving its role as the stepstones to our glorious future; take that, and the famous WW2 bomber survivorship bias graphic, and you have the problem with the logic of that last paragraph).
Okay, I see what you mean now.
The future will be abundant, because deep learning works. To achieve that, we need to be calm, but cautious. And, we need to fund infra (chips and power) so that AGI isn't limited to the ultra-wealthy.
My take:
* Foom/doom isn't helpful. But, calm cautiousness is. If you're acting from a place of fear and emotional dysregulation, you'll make ineffective choices. If you get calm and regulated first, and then take actions, they'll be more effective. (This is my issue with AGI-risk people, they often seem triggered/fear/alarm-driven rather than calm but cautious)
* Piece is kind of a manifesto for raising money for AI infra
* Sam's done a podcast before about meditation where he talked about similar themes of "prudence without fear" and the dangers of "deep fear and panic and anxiety" and instead the importance of staying "calm and centered during hard and stressful moments" - responding, not reacting (strong +1)
* It's no accident that o1 is very good at math physics and programming. It'll keep getting much better here. Presumably this is the path for AGI to lead to abundance and cheaper energy by "solving physics"
My take:
* Foom/doom isn't helpful. But, calm cautiousness is. If you're acting from a place of fear and emotional dysregulation, you'll make ineffective choices. If you get calm and regulated first, and then take actions, they'll be more effective. (This is my issue with AGI-risk people, they often seem triggered/fear/alarm-driven rather than calm but cautious)
* Piece is kind of a manifesto for raising money for AI infra
* Sam's done a podcast before about meditation where he talked about similar themes of "prudence without fear" and the dangers of "deep fear and panic and anxiety" and instead the importance of staying "calm and centered during hard and stressful moments" - responding, not reacting (strong +1)
* It's no accident that o1 is very good at math physics and programming. It'll keep getting much better here. Presumably this is the path for AGI to lead to abundance and cheaper energy by "solving physics"
Well put! I would disagree on two fundamental points, tho:
1. If you honestly think that millions/billions of people are at serious risk of avoidable harm that everyone else is ignoring, "calm down" can be a hard dictum to follow. Sam Altman has won, it's easy for him psychologically to say "well, lets just stick to the status quo and do our best every day and it'll probably work out". Made-in-house bias is strongest when "in-house" is your own mind, after all.
2. Your scare quotes makes it seem like you might agree, but: physics is the study of the physical world, thinking it can be 'solved' is like thinking mathematics, psychology, or anthropology can be 'solved'. It's fundamentally anti-science and very, very dangerous to be talking like that. Truth isn't absolutely relative, but science also isn't a collection of facts written in stone that we need to finish unearthing; it's a collection of intellectual tools.
1. If you honestly think that millions/billions of people are at serious risk of avoidable harm that everyone else is ignoring, "calm down" can be a hard dictum to follow. Sam Altman has won, it's easy for him psychologically to say "well, lets just stick to the status quo and do our best every day and it'll probably work out". Made-in-house bias is strongest when "in-house" is your own mind, after all.
2. Your scare quotes makes it seem like you might agree, but: physics is the study of the physical world, thinking it can be 'solved' is like thinking mathematics, psychology, or anthropology can be 'solved'. It's fundamentally anti-science and very, very dangerous to be talking like that. Truth isn't absolutely relative, but science also isn't a collection of facts written in stone that we need to finish unearthing; it's a collection of intellectual tools.
> If we want to put AI into the hands of as many people as possible, we need to drive down the cost of compute and make it abundant (which requires lots of energy and chips). If we don’t build enough infrastructure, AI will be a very limited resource that wars get fought over and that becomes mostly a tool for rich people.
This seems to be the key of the piece to me. It's his manifesto for raising money for the infra side of things. And, it resonates: I don't want ASI to only be affordable to the ultra rich.
This seems to be the key of the piece to me. It's his manifesto for raising money for the infra side of things. And, it resonates: I don't want ASI to only be affordable to the ultra rich.
To be brutally frank we should focus on energy, as we definitely need way more of that (with less carbon) even if LLMs don't improve any more.
Fusion power is just eight minutes away...
The issue is that we re-eject too little of that energy back into space again.
Sure, but large scale solar probably isn't globally practical without a world grid.
> to put AI into the hands of as many people as possible, we need to drive down the cost of compute and make it abundant (which requires lots of energy and chips)
Not really. The problem is that learning requires scale, mostly of data. That scale places AI providers at the nexus of value, with OpenAI as the presumptive market organizer and leader. Reducing compute costs would just mean they can capture more of the value. Data costs orders of magnitudes more than compute because it requires curation, so even if individual developers could get compute, they can't get data, so size/access matters.
That's good for this community and could be good for the state of the art and the overall potential contributions of AI. And more paying customers could be good for OpenAI. But it won't put AI in front of non-paying customers/developers, unless their value is otherwise harvested.
> I don't want ASI to only be affordable to the ultra rich.
As a developer or consumer?
And you don't mean it's ok if AI is only affordable for the moderately rich, do you? I agree it's hard to state which developers/people/customers should be subsidized. Generally we subsidize education but not profit or war. Sometimes culture. Companies will subsidize complementary goods and input factors. Otherwise? Not much history of benevolent subsidy.
AI has as much potential to shape society as freeways and the automobile did in the US, but few understand how, and I've seen no plans on point.
With electric energy networks and transportation, the central government has a role in reducing hostaging by hold-out's. With education, states have an incentive to attract and build talent (albeit now reduced with trans-national outsourcing and remote work). But otherwise, it's private enterprise and resource-weighted customers.
Changing that is not really Sam Altman's job. His job is to deliver that value, sooner rather than later. Most would be uncomfortable with AI overlords expressing opinions on cultural values or economic distributions to be imposed.
Not really. The problem is that learning requires scale, mostly of data. That scale places AI providers at the nexus of value, with OpenAI as the presumptive market organizer and leader. Reducing compute costs would just mean they can capture more of the value. Data costs orders of magnitudes more than compute because it requires curation, so even if individual developers could get compute, they can't get data, so size/access matters.
That's good for this community and could be good for the state of the art and the overall potential contributions of AI. And more paying customers could be good for OpenAI. But it won't put AI in front of non-paying customers/developers, unless their value is otherwise harvested.
> I don't want ASI to only be affordable to the ultra rich.
As a developer or consumer?
And you don't mean it's ok if AI is only affordable for the moderately rich, do you? I agree it's hard to state which developers/people/customers should be subsidized. Generally we subsidize education but not profit or war. Sometimes culture. Companies will subsidize complementary goods and input factors. Otherwise? Not much history of benevolent subsidy.
AI has as much potential to shape society as freeways and the automobile did in the US, but few understand how, and I've seen no plans on point.
With electric energy networks and transportation, the central government has a role in reducing hostaging by hold-out's. With education, states have an incentive to attract and build talent (albeit now reduced with trans-national outsourcing and remote work). But otherwise, it's private enterprise and resource-weighted customers.
Changing that is not really Sam Altman's job. His job is to deliver that value, sooner rather than later. Most would be uncomfortable with AI overlords expressing opinions on cultural values or economic distributions to be imposed.
I don't see how the current state of AI tech is not for the ultra-rich. It's hundreds of billions of dollars worth of investment from who? Corporations and those who own the most shares of those corporations(the rich).
Billionaires could make so much change happen but instead they are building bunkers and riding giant dicks into space while simultanously touting that they are looking out for humanity.
Billionaires could make so much change happen but instead they are building bunkers and riding giant dicks into space while simultanously touting that they are looking out for humanity.
"What is good?" or in other words "What are people for?" is a question that cannot be answered by intelligence no matter how great, because the complexity of the question is a function of the intelligence of the asking entity and it's always greater than the intelligence of the asking entity (human or transhuman cyborg or whatever.)
AI is a side-show.
Intelligence is ambient in living tissue, so we already have as much intelligence as is adaptive. We don't need more. As talking apes made out of soggy mud wrapped around calcium twigs living in the greasy layer between hard vacuum and a droplet of lava which in turn is orbiting a puddle of hydrogen in the hem of the skirt of a black hole our problems are just not that complicated.
Heck, we are surrounded by four-billion year-old self-improving nanotechnology that automatically provides almost all our physical needs. It's even solar-powered! The whole life-support system was fully automatic until we fucked it up in our ignorance. But we're no longer ignorant, eh?
The vast majority of our problems today are the result of our incredible resounding success. We have the solutions we need. Most of them were developed in the 1970's when the oil got expensive for a few minutes.
Must we boil the oceans just to have a talking computer tell us to get on with it? Can't we just do like the Wizard of Oz? Have a guy in a box with a voice changer and a fancy light show tell us to "love each other"? Mechanical Turk God? We can use holograms.
AI is a side-show.
Intelligence is ambient in living tissue, so we already have as much intelligence as is adaptive. We don't need more. As talking apes made out of soggy mud wrapped around calcium twigs living in the greasy layer between hard vacuum and a droplet of lava which in turn is orbiting a puddle of hydrogen in the hem of the skirt of a black hole our problems are just not that complicated.
Heck, we are surrounded by four-billion year-old self-improving nanotechnology that automatically provides almost all our physical needs. It's even solar-powered! The whole life-support system was fully automatic until we fucked it up in our ignorance. But we're no longer ignorant, eh?
The vast majority of our problems today are the result of our incredible resounding success. We have the solutions we need. Most of them were developed in the 1970's when the oil got expensive for a few minutes.
Must we boil the oceans just to have a talking computer tell us to get on with it? Can't we just do like the Wizard of Oz? Have a guy in a box with a voice changer and a fancy light show tell us to "love each other"? Mechanical Turk God? We can use holograms.
> "What are people for?" is a question that cannot be answered by intelligence no matter how great
It absolutely can be answered, but only the the intender. Who is and who was and who is to come. Or, if you side with the "Nietzche is right" side of the conversation "who will be or who may have come to be today or who recently came to be again". The former is eucatastrophic, the latter is dystopic.
It absolutely can be answered, but only the the intender. Who is and who was and who is to come. Or, if you side with the "Nietzche is right" side of the conversation "who will be or who may have come to be today or who recently came to be again". The former is eucatastrophic, the latter is dystopic.
Existential horror of this whole conversation aside, this is a beautifully written comment. I'd read your novel, but you don't seem to have one, so I'll have to settle for following you on Mastodon.
Cheers! You're too kind.
I thought I would get reamed for farting in church but it seems to have gone over well.
See ya Space Cowboy
I thought I would get reamed for farting in church but it seems to have gone over well.
See ya Space Cowboy
We've had the capability to feed the entire planet for decades now, and yet, a large portion is underfed. Even if Sam's wildest dreams come true, this sounds like another "divide the world into haves and have-nots". One way to justify this unethical view is to consider that his audience was a set of target customers, rather than "people in the world". IOW, this was just fancy marketing fluff.
“This may turn out to be the most consequential fact about all of history so far. It is possible that we will have superintelligence in a few thousand days (!); it may take longer, but I’m confident we’ll get there.”
I am a believer that people like sam are not lying. Anyone using these models daily probably believes the same. The o1 model, if prompted correctly, can architect a code base in a way that my decade+ of premium software experience cannot. Prompted incorrectly, it looks incompetent. The abilities of the future are already here, you just need to know how to use the models.
I am a believer that people like sam are not lying. Anyone using these models daily probably believes the same. The o1 model, if prompted correctly, can architect a code base in a way that my decade+ of premium software experience cannot. Prompted incorrectly, it looks incompetent. The abilities of the future are already here, you just need to know how to use the models.
I'm using these models daily, and I don't believe that they're a direct path to superintelligence (unless you'd consider something like the printing press to have been a direct path to, say, the integrated circuit or the Internet).
> Prompted incorrectly, it looks incompetent. The abilities of the future are already here, you just need to know how to use the models.
Something purportedly intelligent shouldn't need "correct usage", as it should arguably be able to infer and clarify all ambiguities itself, no?
> Prompted incorrectly, it looks incompetent. The abilities of the future are already here, you just need to know how to use the models.
Something purportedly intelligent shouldn't need "correct usage", as it should arguably be able to infer and clarify all ambiguities itself, no?
The model's arent there yet. I am confused how people cannot extrapolate into the future and understand that the models will improve
And I'm surprised how many people think they can confidently tell whether they're looking at an s-curve or an exponential function based on very limited data points. I don't even doubt that superintelligence is a very real possibility! But it might or might not happen, and if it does, it might or might not be based on deep learning.
As a counterexample: The maximum speed of travel for the average person for millenia used to be as fast as they could run, then it was as fast as the fastest horse can run, and then within a century it has accelerated to almost the speed of sound – at which it has plateaued.
Looking purely at the decades of acceleration, you might have very well concluded from the data that we'd be making significant headway towards getting within double-digit percentages of the speed of light at this point.
As a counterexample: The maximum speed of travel for the average person for millenia used to be as fast as they could run, then it was as fast as the fastest horse can run, and then within a century it has accelerated to almost the speed of sound – at which it has plateaued.
Looking purely at the decades of acceleration, you might have very well concluded from the data that we'd be making significant headway towards getting within double-digit percentages of the speed of light at this point.
Maybe don't think of it as curves or functions. Just go through an LLM and think about all the things that could be improved. It's a long list once you get into the details. By and large...sota models are:
a) trained on crappy data, including questionable RLHF feedback. b) trained with questionable embedding layers. c) trained with questionable loss functions d) trained with questionable optimizers e) trained at questionable precision (somewhat related to d) f) are very big which stops fast iteration around all the above.
It's kinda like semiconductors. You don't have to think of it as a curve - just ask people who are really close to them and they'll have a laundry list of stupid stuff which is currently done and will likely be improved upon over time.
a) trained on crappy data, including questionable RLHF feedback. b) trained with questionable embedding layers. c) trained with questionable loss functions d) trained with questionable optimizers e) trained at questionable precision (somewhat related to d) f) are very big which stops fast iteration around all the above.
It's kinda like semiconductors. You don't have to think of it as a curve - just ask people who are really close to them and they'll have a laundry list of stupid stuff which is currently done and will likely be improved upon over time.
I don't doubt that there's tons of work ahead to even just integrate current-day capabilities of LLMs into our society and economy.
But when talking about future growth potential, I don't think you can get around making assumptions about the shape of the growth function.
But when talking about future growth potential, I don't think you can get around making assumptions about the shape of the growth function.
You can't write a nice article about it. But you can talk about it with a simple "things are going to get a loooot better". The problem with thinking about progress in functions is it suggests some underlying law exists, and there isn't one.
Even if someone could point to a function and say "10x better" by 2030 - what does that even mean in the context of an LLM for example?
Even if someone could point to a function and say "10x better" by 2030 - what does that even mean in the context of an LLM for example?
[deleted]
Interesting observation. My experience with o1 has been much more mundane. Sometimes I get the response I wanted, sometime it hallucinates, often times it writes buggy code. I've been experiencing this since ChatGPT was first released.
I actually originally wrote off the o1 model. Another thing I have found it's good at is finding bugs in a ton of code. Give it ten coding files, and a stack trace, it can find the bug.
“If we don’t build enough infrastructure, AI will be a very limited resource that wars get fought over and that becomes mostly a tool for rich people.”
… This, and nothing about the democratizing effect of “open source AI” (Yes we still need to define what that is!).
I don’t want Sam as the thought leader of AI. I even prefer Zuck.
Are there any thought leaders that are really about democratization and local foss AI on open hardware? Or do they always follow (or step into the light) after the big moneymakers have had their moment? Who can we start watching? The Linus, the RMS, the Wozniak’s of AI. Who are they?
… This, and nothing about the democratizing effect of “open source AI” (Yes we still need to define what that is!).
I don’t want Sam as the thought leader of AI. I even prefer Zuck.
Are there any thought leaders that are really about democratization and local foss AI on open hardware? Or do they always follow (or step into the light) after the big moneymakers have had their moment? Who can we start watching? The Linus, the RMS, the Wozniak’s of AI. Who are they?
The most interesting bit I find is the time period mentioned until super-intelligence: “thousands of days (!)” aka 6-9 years or more?
With the current hype wave it feels like we’re almost there but this piece makes me think we’re not.
With the current hype wave it feels like we’re almost there but this piece makes me think we’re not.
If anything, I would say that that's a very optimistic take. The hype train is strong, but that's largely what it is once you look at the details. What we have right now is impressive, but no one has shown anything close to a possible path from where we are right now to AGI. The things we can do right now are fancy, but they're fancy in the same way good autocomplete is fancy. To me, it feels like a local maxima, but it's very unclear whether the specific set of approaches we're exploring right now can lead to something more.
> What we have right now is impressive, but no one has shown anything close to a possible path from where we are right now to AGI[0].
[0]: From GPT-4 to AGI: Counting the OOMs https://situational-awareness.ai/from-gpt-4-to-agi/
[0]: From GPT-4 to AGI: Counting the OOMs https://situational-awareness.ai/from-gpt-4-to-agi/
I'm not convinced, and neither is Sam Altman himself [0]. Also, if that projection holds, and that's a big if, the purported breakthrough would cost 10^6 times as much as GPT-4 took to train. That's over 100 million dollars [1] times a million. That adds up to over 100 trillion dollars, in the ballpark of four times the GDP of the whole of United States.
[0] https://www.wired.com/story/openai-ceo-sam-altman-the-age-of...
[1] https://en.wikipedia.org/wiki/GPT-4#Training
[0] https://www.wired.com/story/openai-ceo-sam-altman-the-age-of...
[1] https://en.wikipedia.org/wiki/GPT-4#Training
The thing is that it looks like, or perhaps I should say it's "understood" at this point, that transformer's abilities scale pretty much linearly with compute (there is also some evidence they scale exponentially with parameter count, but just some evidence).
Right now there is insane amounts of money being thrown at AI because progress is matching projections. There doesn't seem to be a leveling off or diminishing returns taking place. And that's just compute, we could probably freeze compute and still make insane progress just because optimizations have so much momentum right now too.
Right now there is insane amounts of money being thrown at AI because progress is matching projections. There doesn't seem to be a leveling off or diminishing returns taking place. And that's just compute, we could probably freeze compute and still make insane progress just because optimizations have so much momentum right now too.
How do you distinguish the path to fancier autocomplete from the path towards AGI? Why think we're on the former rather than the latter?
I think that's part of the carefully-crafted hype messaging. Close enough to get excited about, but far enough away that by the time we get there people will have forgotten we were supposed to have it by then.
Yeah, that's my number one question, too. Sure, he happened to be appointed the manager of the team who cracked intuitive algorithms through deep learning, but what does he know about superintelligence? IMO that's a completely separate question, and "foundation models continue to improve" is absolutely not related to whether or not an intelligence explosion is guaranteed or not. I'd trust someone like Yudkowsky way more on this, or really anyone who has engaged with academic literature on the subjects of intentionality, receptive vs. spontaneous reasoning, or really any academic literature of any kind...
Does anyone know if he's published thoughts on any serious lit? So far I've just seen him play the "I know stuff you don't because I get to see behind the scenes" card over and over, which seems a little dubious at this point. I was convinced they would announce AGI in December 2023, so I'm far from a hater! It just seems clear that they're/he's guessing at this point, rather than reporting or reasoning.
Really he assumes two huge breakthroughs, both of which I find plausible but far from guaranteed:
Does anyone know if he's published thoughts on any serious lit? So far I've just seen him play the "I know stuff you don't because I get to see behind the scenes" card over and over, which seems a little dubious at this point. I was convinced they would announce AGI in December 2023, so I'm far from a hater! It just seems clear that they're/he's guessing at this point, rather than reporting or reasoning.
Really he assumes two huge breakthroughs, both of which I find plausible but far from guaranteed:
With nearly-limitless intelligence and abundant energyI would presume that that’s the time period he’s currently trying to fund.
It looks like this is a new subdomain that has never been used before today: https://web.archive.org/web/20240000000000*/https://ia.samal...
Surprisingly complicated HTML source code for a simple blog post.
Here it is as:
Plain HTML: https://hub.scroll.pub/sama/index.html
Text: https://hub.scroll.pub/sama/index.txt
Scroll: https://hub.scroll.pub/sama/index.scroll
Surprisingly complicated HTML source code for a simple blog post.
Here it is as:
Plain HTML: https://hub.scroll.pub/sama/index.html
Text: https://hub.scroll.pub/sama/index.txt
Scroll: https://hub.scroll.pub/sama/index.scroll
Wow, great catch. Something tells me he rolled this himself. Clearly he's trying to coin a term for personal legacy reasons, and I say godspeed. Holocene is a little vague, the information age is too entrenched to get anyone's attention, a/the singular age / the singularity are way too deeply associated with the doomer community, and the cybernetic age (a term coined for academia!) is too associated with playful science fiction.
I'm personally rooting for cognitive being the word of the next few decades, but that's just a shout from the sidelines. Only time will tell what humanity latches on to, but I wouldn't be surprised if this blog post/subdomain was referenced in a Wikipedia page's Etymology section in 10-15 years...
Although this blog post & discussion has my anxiety at an 8, something's oddly comforting about the thought of Sam Altman fiddling with tailwind classes to get his profound aesthetic just-right. Something undeniably relatable and human. Hate the man all you want (I do!), but he's clearly acting in some sort of good faith.
I'm personally rooting for cognitive being the word of the next few decades, but that's just a shout from the sidelines. Only time will tell what humanity latches on to, but I wouldn't be surprised if this blog post/subdomain was referenced in a Wikipedia page's Etymology section in 10-15 years...
Although this blog post & discussion has my anxiety at an 8, something's oddly comforting about the thought of Sam Altman fiddling with tailwind classes to get his profound aesthetic just-right. Something undeniably relatable and human. Hate the man all you want (I do!), but he's clearly acting in some sort of good faith.
> Clearly he's trying to coin a term
You are right. I overlooked the simplicity of the headline. Thanks for calling attention to that.
> Something tells me he rolled this himself.
This would be cool.
> I'm personally rooting for cognitive being the word of the next few decades
I like that one too.
> he's clearly acting in some sort of good faith.
He's always been one to think and write for himself. Huge respect for him. Even though it needles me every moment that they still call themselves "Open"AI, I have so much respect for the guy, especially because PG basically told the world he was the next Michael Jordan of startups, and he actually went and fulfilled that. Not many people have it in them to live up to hype like that (Lebron being the only other one I can think of OOTOMH)
You are right. I overlooked the simplicity of the headline. Thanks for calling attention to that.
> Something tells me he rolled this himself.
This would be cool.
> I'm personally rooting for cognitive being the word of the next few decades
I like that one too.
> he's clearly acting in some sort of good faith.
He's always been one to think and write for himself. Huge respect for him. Even though it needles me every moment that they still call themselves "Open"AI, I have so much respect for the guy, especially because PG basically told the world he was the next Michael Jordan of startups, and he actually went and fulfilled that. Not many people have it in them to live up to hype like that (Lebron being the only other one I can think of OOTOMH)
This morning I was reviewing some code that a JR engineer submitted. It had this wild logical conditional with twists and turns, negations, and weird property naming..
o1-preview perfectly evaluated the conditional and determined that, hilariously, it would always evaluate to true.
o1 untangled the spaghetti, and, verifying that it was correct was quick and easy. It created a perfect truth table for me to visualize.
This is a sign of things to come. We are speeding up.
o1-preview perfectly evaluated the conditional and determined that, hilariously, it would always evaluate to true.
o1 untangled the spaghetti, and, verifying that it was correct was quick and easy. It created a perfect truth table for me to visualize.
This is a sign of things to come. We are speeding up.
It's surprising that the JR engineer who presumably has access to chat jippity submitted this bad code to begin with. Wouldn't they have had the AI review the code first?
My theory would be that the Jr probably generated the code with o1 in the first place. The buggy, nonsensical crap I've seen that thing generate boggles the mind.
This is painting a picture of a utopia. I'm not sure about that.
The entire AI trend - long term is based on the idea that AI will profoundly change the world. This has sparked a global race for developing better AI systems and the more dangerous winner takes all outcome. It is therefore not surprising that billions of dollars are being spent to develop more powerful AI systems as well as to restructure operations around them.
All the existing systems we have must fundamentally change for the better if we want a good future.
The positive aspects / utopia promises have much more visibility to the public than the negative effects / dystopian world.
ARE WE TO pretend that Human greed, selfishness, desires to dominate and control, animalistic behaviour, use of technologies for war and other destructive purposes don't exist?
We are living in times of war and chaos and uncertainty. Increasingly advanced technology is being used on the battlefield in more covert and strategic ways.
History is repeating itself again in many ways. Have we failed to learn? The consequences might be harsher with more advanced technology.
I have read and thought deeply about several anti AI doomer takes from prominent researchers and scientists but I haven't seen any which aren't based on assumptions or foolproof. For something that profoundly changes the world, it's bad to base your hopes on assumptions.
I see people dunking on llms which might not be AI's final form. Then they extrapolate that and say there is nothing to worry about. It is a matter of when not if.
The thought of being useless or worse being treated as nothing more than pests is worrying. Job losses are minor in comparison.
The only hope I have is that we are all in this together. I hope peace and goodwill prevails. I hope necessary actions are taken before it's too late.
A more pragmatic perspective indicates that there are more pressing problems that need to be addressed if we want to avoid a doomer scenario.
The entire AI trend - long term is based on the idea that AI will profoundly change the world. This has sparked a global race for developing better AI systems and the more dangerous winner takes all outcome. It is therefore not surprising that billions of dollars are being spent to develop more powerful AI systems as well as to restructure operations around them.
All the existing systems we have must fundamentally change for the better if we want a good future.
The positive aspects / utopia promises have much more visibility to the public than the negative effects / dystopian world.
ARE WE TO pretend that Human greed, selfishness, desires to dominate and control, animalistic behaviour, use of technologies for war and other destructive purposes don't exist?
We are living in times of war and chaos and uncertainty. Increasingly advanced technology is being used on the battlefield in more covert and strategic ways.
History is repeating itself again in many ways. Have we failed to learn? The consequences might be harsher with more advanced technology.
I have read and thought deeply about several anti AI doomer takes from prominent researchers and scientists but I haven't seen any which aren't based on assumptions or foolproof. For something that profoundly changes the world, it's bad to base your hopes on assumptions.
I see people dunking on llms which might not be AI's final form. Then they extrapolate that and say there is nothing to worry about. It is a matter of when not if.
The thought of being useless or worse being treated as nothing more than pests is worrying. Job losses are minor in comparison.
The only hope I have is that we are all in this together. I hope peace and goodwill prevails. I hope necessary actions are taken before it's too late.
A more pragmatic perspective indicates that there are more pressing problems that need to be addressed if we want to avoid a doomer scenario.
The Age of Inhumanity... AI mimicry of human patterns devalues our very humanity. Without wise leadership, which we clearly lack, this upcoming Age will be profoundly unstable.
> We need to act wisely but with conviction.
Reminds me of these quotes from Sam on this podcast episode (https://www.youtube.com/watch?v=KfuVSg-VJxE)
* "Prudence without fear" (Sam referencing another quote)
* "if you create the descendants of humanity from a place of, deep fear and panic and anxiety, that seems to me you're likely to make some very bad choices or certainly not reflect the best of humanity."
* "the ability to sort of like, stay calm and centered during hard and stressful moments, and to make decisions that are where you're not too reactive"
Reminds me of these quotes from Sam on this podcast episode (https://www.youtube.com/watch?v=KfuVSg-VJxE)
* "Prudence without fear" (Sam referencing another quote)
* "if you create the descendants of humanity from a place of, deep fear and panic and anxiety, that seems to me you're likely to make some very bad choices or certainly not reflect the best of humanity."
* "the ability to sort of like, stay calm and centered during hard and stressful moments, and to make decisions that are where you're not too reactive"
Easy for him to say: AI is almost guaranteed to hand a massive W to capital and L to labor. He is holding a title to rule over hell in one hand and promising to lead us to heaven with the other.
Actually not sure about this, with the leverage of AI, its easier than ever to start a company
Sam Altman has a guarantee. Most people on HN have a chance. Most people have neither.
Yes, so there will be all sorts of attempts, and the productive forces will compete with each other, while landowners and owners of harder to disrupt industries will have the power to choose among them, deciding who will be allowed to be successful and who will fail.
There isn't a single ordinary person at the table, no labour unions, no political parties or anything democratic. It's Microsoft, it's Google, it's some venture-capital owned French firm, some venture-capital owned German firm, etc. Maybe if Schmidhuber's stuff is good as he hopes it is maybe there'll be an Austrian firm, etc., but mostly, it'll be a capital intensive business controlled by people with capital.
There isn't a single ordinary person at the table, no labour unions, no political parties or anything democratic. It's Microsoft, it's Google, it's some venture-capital owned French firm, some venture-capital owned German firm, etc. Maybe if Schmidhuber's stuff is good as he hopes it is maybe there'll be an Austrian firm, etc., but mostly, it'll be a capital intensive business controlled by people with capital.
There is no evidence that it has made it easier to end up with a successful company, however.
Who's going to work at all these companies then? Unless every single profession suddenly only requires 1 person to do the entire thing with no management, coordination or hierarchies, a lot of people will be labor not capital.
Maybe companies as we think of them are a temporary phenomenon in human history
> * "if you create the descendants of humanity from a place of, deep fear and panic and anxiety, that seems to me you're likely to make some very bad choices or certainly not reflect the best of humanity."
This line of reasoning doesn't hold for me, as you could apply it to any technology, including ones actually very likely to destroy human civilization.
Sometimes, not building a given thing at all is better than building it with even the best intentions.
I'm personally not sure on which side AI falls, but denying that such things exist at all seems intellectually dishonest.
This line of reasoning doesn't hold for me, as you could apply it to any technology, including ones actually very likely to destroy human civilization.
Sometimes, not building a given thing at all is better than building it with even the best intentions.
I'm personally not sure on which side AI falls, but denying that such things exist at all seems intellectually dishonest.
Indeed. It sounds like the kind of thing the OceanGate Titan sub's CEO would have said. Except all of humanity is on board Sam Altman's submarine.
There's a good reason we get pessimists to design safety-critical systems!
There's a good reason we get pessimists to design safety-critical systems!
"Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue." -OpenAI
"Why You Should Fear Machine Intelligence
Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity." - Sam Altman
https://openai.com/index/introducing-superalignment/
https://blog.samaltman.com/machine-intelligence-part-1
"Why You Should Fear Machine Intelligence
Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity." - Sam Altman
https://openai.com/index/introducing-superalignment/
https://blog.samaltman.com/machine-intelligence-part-1
[deleted]
I'm sceptical about humanity's future, and not only with perspective of AGI getting out of control..
Average human won't be able to harness new powers.
solarpunk envisioned, possible today:
- entire human knowledge available on palm of your hand..!
vs
cyberdaftpunk actually more common:
- another idiot driver killed somebody when being busy with his candy crush saga or instagram celebrity vid.
solarpunk envisioned, possible today:
- entire human knowledge available on palm of your hand..!
vs
cyberdaftpunk actually more common:
- another idiot driver killed somebody when being busy with his candy crush saga or instagram celebrity vid.
To paraphrase Goggins, "Who's gonna carry the cabbage?"
While it's true there are a lot of jobs obsoleted by technological progress, the vision of personal AI teams creating a new age of prosperity only makes sense for knowledge workers. Sure, a field worker picking cabbage could also have an AI team to coordinate medical care. But in this brilliant future, are the lowest members of society suddenly well-paid?
The steam engine and subsequent Industrial Revolution created a lot of jobs and economic productivity, sure, but a huge amount of those jobs were dirty, dangerous factory jobs, and the lion's share of the productivity was ultimately captured by robber barons for quite some time. The increase in standard of living could only be seen in aggregate on pages of statistics from the mahogany-paneled offices of Standard Oil, while the lives of the individuals beneath those papers more often resembled Sinclair's Jungle.
Altman's suggestion that avoiding AI capture by the rich merely requires more compute is laughable. We have enormous amounts of compute currently, and its productivity is already captured by a small number of people compared to the vast throngs that power civilization in total. Why would AI make this any different? The average person does not understand how AI works and does not have the resources to utilize it. Any further advancements in AI, including "personalized AI teams," will not be equally shared, they will be packaged into subscription services and sold, only to enrich those who already control the vast majority of the world's wealth.
While it's true there are a lot of jobs obsoleted by technological progress, the vision of personal AI teams creating a new age of prosperity only makes sense for knowledge workers. Sure, a field worker picking cabbage could also have an AI team to coordinate medical care. But in this brilliant future, are the lowest members of society suddenly well-paid?
The steam engine and subsequent Industrial Revolution created a lot of jobs and economic productivity, sure, but a huge amount of those jobs were dirty, dangerous factory jobs, and the lion's share of the productivity was ultimately captured by robber barons for quite some time. The increase in standard of living could only be seen in aggregate on pages of statistics from the mahogany-paneled offices of Standard Oil, while the lives of the individuals beneath those papers more often resembled Sinclair's Jungle.
Altman's suggestion that avoiding AI capture by the rich merely requires more compute is laughable. We have enormous amounts of compute currently, and its productivity is already captured by a small number of people compared to the vast throngs that power civilization in total. Why would AI make this any different? The average person does not understand how AI works and does not have the resources to utilize it. Any further advancements in AI, including "personalized AI teams," will not be equally shared, they will be packaged into subscription services and sold, only to enrich those who already control the vast majority of the world's wealth.
The thing is: robotics is knowledge work. Supposing a scenario in which AI makes advancing fields of engineering and science much more rapid, it will be leveraged to build and cheapen robotic labor. There would be a gap period where AI is smart but unable to perform labor without humans, which could be ugly, and then we reach effective post-scarcity and post-humans-being-useful. Where we go from there could be heaven or hell depending on who's in charge.
Shhh... you're not supposed to say anything about this!
We need to sell the idea of abundance for everyone so investors and employees will feel good about dedicating their livelihood to our organization!
We need to sell the idea of abundance for everyone so investors and employees will feel good about dedicating their livelihood to our organization!
Did AI write this?
"This age is characterized by society's increasingly advanced capabilities, driven not by genetic changes but by societal infrastructure becoming smarter and more efficient over time."
"This age is characterized by society's increasingly advanced capabilities, driven not by genetic changes but by societal infrastructure becoming smarter and more efficient over time."
Question for active professionals during the Moore's law era of computing. Back then, were executives writing such grand proclamations about the future ? In my experience, when things are working, executives are quiet. The outcomes speak for themselves.
Thankfully, we have a recent point of reference. The pioneers of internet & computing's 1st wave transformed civilization. Did they spend years saber rattling about how 'change was coming' ?
Thankfully, we have a recent point of reference. The pioneers of internet & computing's 1st wave transformed civilization. Did they spend years saber rattling about how 'change was coming' ?
Of course they did! During the advent of personal computers there was plenty of hype and grand-visioning for a technology that had very little value until VisiCalc triggered exponential growth. Same story for social networking (look at The WELL), the Web (Netscape practically invented insane-seeming hype-based valuations), many other examples.
In fact, I can't really think of any part of the industry where "outcomes speak for themselves" -- I would have said quite the opposite is the norm.
In fact, I can't really think of any part of the industry where "outcomes speak for themselves" -- I would have said quite the opposite is the norm.
> Back then, were executives writing such grand proclamations about the future ?
Yes. And the good ones (like SamA and SJ) deliver.
https://youtu.be/1tQ5XwvjPmA?t=278
Yes. And the good ones (like SamA and SJ) deliver.
https://youtu.be/1tQ5XwvjPmA?t=278
That’s a bit funny, Moores law itself is an example of an executive being loud and making a grand proclamation
Bill Joy - Why The Future Doesn't Need Us - 4/1/2000
https://www.wired.com/2000/04/joy-2/
Basically, the same article but much more negative.
https://www.wired.com/2000/04/joy-2/
Basically, the same article but much more negative.
“Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past, wishing they were a lamplighter. If a lamplighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast-forward a hundred years from today, the prosperity all around us would feel just as unimaginable.” Loving the techno optimism.
I got a kick out of that. Spoken like a man who doesnt have to concern himself with the price of housing or any other necessity as we watch this strange short lived phenomena called the middle class disappear and we're back to the haves and have-nots.
It's a lot more comforting to work on AI (or any tech for that matter) if you believe or force yourself to believe it will be used for good. OpenAI's for-profit and gatekeeping approach is unlikely to be the path to the prosperity Sam Altman envisions.
I'm skeptical about the prosperity claim. If the entire planet has access to affordable/cheap and very capable AI, this still leaves two problems.
Since the access itself is not differentiating, it's going to be the most educated benefiting the most. Already today few people can use the o1 model because they can't dream up a PhD level question nor understand its answers.
More importantly, access to AI does not mean access to assets. Me, a total nobody, can use AI to design the world's best car. But that does nothing because I don't have money or land. Anybody can query AI for that car but only asset owners can actually implement the idea and extract value. Those asset owners could use AI to bring widespread prosperity to all of mankind, but we know they won't.
We don't need more material prosperity, we need social prosperity. Family formation, the restoration of community life, economic security. Not "more stuff".
Since the access itself is not differentiating, it's going to be the most educated benefiting the most. Already today few people can use the o1 model because they can't dream up a PhD level question nor understand its answers.
More importantly, access to AI does not mean access to assets. Me, a total nobody, can use AI to design the world's best car. But that does nothing because I don't have money or land. Anybody can query AI for that car but only asset owners can actually implement the idea and extract value. Those asset owners could use AI to bring widespread prosperity to all of mankind, but we know they won't.
We don't need more material prosperity, we need social prosperity. Family formation, the restoration of community life, economic security. Not "more stuff".
In a world of infinite leverage, those who can leverage will far outrun those who cannot leverage or do so poorly. The world is also getting flatter, so you will see massive clusters of people near the bottom, with a very long fat tail of people far away at the top. My only hope is that the floor is raised for everyone, so we're not living in a dystopia like Elysium.
> Although it will happen incrementally, astounding triumphs – fixing the climate
This is so rich coming from a tech field that's on track to match the energy consumption of a small country. (And no, AI is not going to offset this by 'finding innovative solutions to climate change' or whatever)
This is so rich coming from a tech field that's on track to match the energy consumption of a small country. (And no, AI is not going to offset this by 'finding innovative solutions to climate change' or whatever)
I like Sam's philosophy on this and I generally agree with him. However, I do not like how all the wealthy AI people are hand-waving the massive labor market shift in the coming years.
> As one example, we expect that this technology can cause a significant change in labor markets (good and bad) in the coming years, but most jobs will change more slowly than most people think, and I have no fear that we’ll run out of things to do (even if they don’t look like “real jobs” to us today). People have an innate desire to create and to be useful to each other, and AI will allow us to amplify our own abilities like never before. As a society, we will be back in an expanding world, and we can again focus on playing positive-sum games.
It's very easy as an extremely rich person to just say, "don't worry, in the end it'll be better for all of us." Maybe that's true on a societal scale, but these are people's entire worlds being destroyed.
Imagine you went to college for a medical specialty for 8-10 years, you come out as an expert, and 2 years later that entire field is handled by AI and salaries start to tank. Imagine you have been a graphic designer for 20 years supporting your 3 children and bam a diffusion model can do your job for a fraction of the cost. Imagine you've been a stenographer working in courtrooms to support your ill parents and suddenly ASR can do your job better than you can. This is just simple stuff we can connect the dots on now. There will be orders of magnitude more shifts that we can't even imagine right now.
To someone like Sam, everything will be fine. He can handle the massive societal shift because he has options. Every a moderately wealthy person will be OK.
But the entire middle class is going to start really freaking the fuck out soon as more and more jobs disappear. You're already seeing anti-AI sentiment all over the web. Even in expert circles, you can see skepticism. People saying things like, "how do I opt out of Apple Intelligence?" People don't WANT more grammar correction or AI emojis in their lives, they just want to survive and thrive and own a house.
How are we going to handle this? Sam's words of "if we could fast-forward a hundred years from today, the prosperity all around us would feel just as unimaginable" doesn't mean shit to a family of 4 who went through layoffs in the year 2025 because AI took their job while Microsoft's stock grows 50%.
> As one example, we expect that this technology can cause a significant change in labor markets (good and bad) in the coming years, but most jobs will change more slowly than most people think, and I have no fear that we’ll run out of things to do (even if they don’t look like “real jobs” to us today). People have an innate desire to create and to be useful to each other, and AI will allow us to amplify our own abilities like never before. As a society, we will be back in an expanding world, and we can again focus on playing positive-sum games.
It's very easy as an extremely rich person to just say, "don't worry, in the end it'll be better for all of us." Maybe that's true on a societal scale, but these are people's entire worlds being destroyed.
Imagine you went to college for a medical specialty for 8-10 years, you come out as an expert, and 2 years later that entire field is handled by AI and salaries start to tank. Imagine you have been a graphic designer for 20 years supporting your 3 children and bam a diffusion model can do your job for a fraction of the cost. Imagine you've been a stenographer working in courtrooms to support your ill parents and suddenly ASR can do your job better than you can. This is just simple stuff we can connect the dots on now. There will be orders of magnitude more shifts that we can't even imagine right now.
To someone like Sam, everything will be fine. He can handle the massive societal shift because he has options. Every a moderately wealthy person will be OK.
But the entire middle class is going to start really freaking the fuck out soon as more and more jobs disappear. You're already seeing anti-AI sentiment all over the web. Even in expert circles, you can see skepticism. People saying things like, "how do I opt out of Apple Intelligence?" People don't WANT more grammar correction or AI emojis in their lives, they just want to survive and thrive and own a house.
How are we going to handle this? Sam's words of "if we could fast-forward a hundred years from today, the prosperity all around us would feel just as unimaginable" doesn't mean shit to a family of 4 who went through layoffs in the year 2025 because AI took their job while Microsoft's stock grows 50%.
For this reason I read Andrew Yang’s “The War on Normal People”. Besides UBI and “social credits”, I don’t see him offer that many other solutions to this problem. UBI also still needs to be proven as far as I’m aware.
When o1 was released, I ran an internal eval and saw it plainly outperforming our highly educated colleagues. I had goosebumps, and haven’t been able to sleep well for days. This will dramatically impact society in 2-5 years.
Do you know of any relevant material related to this?
When o1 was released, I ran an internal eval and saw it plainly outperforming our highly educated colleagues. I had goosebumps, and haven’t been able to sleep well for days. This will dramatically impact society in 2-5 years.
Do you know of any relevant material related to this?
o1 was what got you stirred up? It honestly feels like an incremental change at best to me. I had similar feelings about gpt-3.5, but since then my fears have normalized into a sort of dull, typical (for me) cynicism (so no sleepless nights).
https://users.manchester.edu/Facstaff/SSNaragon/Online/100-F...
https://intelligence.org/files/IEM.pdf
Welcome to the anxiety party, it sucks in here. As someone who's been working on AI theory full time for ~1 year, I desperately wish we could go back to the days of my faraway youth (5 years ago) before intuition was cracked on accident by spellcheck algorithms. I agree with him that it holds the key to massive prosperity, but selfishly, it's gonna upend my life and the lives of everyone I love. Already has for me, as I grapple with how to (ethically) pay rent while spending all day lighting the Warning Beacons of Gondor...
The only real answer, IMHO, is to vote for political systems that put control of society (and AI) in the hands of the public. Call it socialism, call it Georgism, call it anarcho-free-market-space-communism, call it whatever you want; there's no way that "a tiny number of people have immense inherited power" (capitalism) and "people fundamentally understand themselves as members of a tribe put in opposition to all other tribes by default" (nationalism) mesh well with an intelligence explosion.
Here's to hoping the haters are right, and we all turn out to be wrong! I'll be thrilled if Sam Altman is just a rich company leader in 10 years, and intuitive algorithms are still confined to direct usage (chatbots).
https://intelligence.org/files/IEM.pdf
Welcome to the anxiety party, it sucks in here. As someone who's been working on AI theory full time for ~1 year, I desperately wish we could go back to the days of my faraway youth (5 years ago) before intuition was cracked on accident by spellcheck algorithms. I agree with him that it holds the key to massive prosperity, but selfishly, it's gonna upend my life and the lives of everyone I love. Already has for me, as I grapple with how to (ethically) pay rent while spending all day lighting the Warning Beacons of Gondor...
The only real answer, IMHO, is to vote for political systems that put control of society (and AI) in the hands of the public. Call it socialism, call it Georgism, call it anarcho-free-market-space-communism, call it whatever you want; there's no way that "a tiny number of people have immense inherited power" (capitalism) and "people fundamentally understand themselves as members of a tribe put in opposition to all other tribes by default" (nationalism) mesh well with an intelligence explosion.
Here's to hoping the haters are right, and we all turn out to be wrong! I'll be thrilled if Sam Altman is just a rich company leader in 10 years, and intuitive algorithms are still confined to direct usage (chatbots).
I wish I could upvote this more than once. I feel like every conversation about AI changing society comes from rich founders telling Joe banker to "Just start a company. AI will make it easy."
The reality is, this transition is going to be painful for the average person.
The reality is, this transition is going to be painful for the average person.
We're going to need to link the two. Those wealthy enough to not care, we're going to have to organize and make them care. Ideally we can find a way to do it nonviolently.
my bet is that it's a repeat of the French revolution
with the billionaire AI moguls taking the role of the french kings
and the data centres taking the role of the palaces
with the billionaire AI moguls taking the role of the french kings
and the data centres taking the role of the palaces
As an aside:
> in an important sense, society itself is a form of advanced intelligence
This made me think of Charles Stross' observation that Corporations (and bureaucracies and basically any rule-based organizations) are a form of artificial intelligence.
https://www.antipope.org/charlie/blog-static/2019/12/artific...
Come to think of it, the whole article is rather pertinent to this thread.
> in an important sense, society itself is a form of advanced intelligence
This made me think of Charles Stross' observation that Corporations (and bureaucracies and basically any rule-based organizations) are a form of artificial intelligence.
https://www.antipope.org/charlie/blog-static/2019/12/artific...
Come to think of it, the whole article is rather pertinent to this thread.
Until the hallucination problem is solved, we can't trust LLM-type AIs to do anything on their own. This limits uses to ones where the cost of errors can be imposed on someone else.
Of course we can. Humans are also mistaken sometimes, especially when trying to solve difficult problems.
If an LLM hallucinates it's usually because the problem is too hard. For easier problems it's rarely an issue in my experience.
Hallucinates is just an indicator that the model is inadequate for the problem you're applying it to. That doesn't mean it's inadequate to solve any problems.
If an LLM hallucinates it's usually because the problem is too hard. For easier problems it's rarely an issue in my experience.
Hallucinates is just an indicator that the model is inadequate for the problem you're applying it to. That doesn't mean it's inadequate to solve any problems.
> fixing the climate
I would be happy to be convinced that climate is an intelligence problem.
One could argue it could be solved with "abundant energy" but if this abundant energy comes from some new intelligence then we are probably several decades away from having it running commercially. I would also be happy to be convinced that we do have this kind of time to act for climate.
I would be happy to be convinced that climate is an intelligence problem.
One could argue it could be solved with "abundant energy" but if this abundant energy comes from some new intelligence then we are probably several decades away from having it running commercially. I would also be happy to be convinced that we do have this kind of time to act for climate.
Has Sam Altman ever talked about ways that he personally uses ChatGPT. Does he at all? does he have an all-smart watch? Is he dreaming of producing a perfect replicant like this guy in Blade Runner 2049... cause with his pathos and present position, he may be among these peoples who can allow himself to do so. This is all so vague he says, i mean, we have all read enough cyberpunk to write the same essay. But I don't know if he actually builds with this tool, cause the way I see it he probably has very little time to actually use it. So why so confident whether its deep learning that worked. I say Euler's graph theory worked more than anything else, and Chomsky's understanding of grammars, so what?
Maybe we will attend superintelligence in 1000 years, maybe not. Maybe Jesus comes back or Krishna reincarnates on earth, who knows. But it is a long way ahead, and it did not start with Sam, and is really not going to end with ChatGPT.
Maybe we will attend superintelligence in 1000 years, maybe not. Maybe Jesus comes back or Krishna reincarnates on earth, who knows. But it is a long way ahead, and it did not start with Sam, and is really not going to end with ChatGPT.
Who exactly is he pitching here?
Newsom
Bingo. That was not obvious to me before your one-word reply, and now, in retrospect, it is.
And I say this as a person who usually just rolls his eyes seeing the typical cynical HN hot takes. I believe Altman voices his deepest convictions here. The smarter thing to be cynical about is, what is his goal with voicing them now, and in this very format? As others have observed, this is the first time he is not even paying lip service to the question of existential risk.
And I say this as a person who usually just rolls his eyes seeing the typical cynical HN hot takes. I believe Altman voices his deepest convictions here. The smarter thing to be cynical about is, what is his goal with voicing them now, and in this very format? As others have observed, this is the first time he is not even paying lip service to the question of existential risk.
> That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)
Seems like it's very much the former, and not all the latter. Indeed my understanding of the last 15 years of AI research is that 'rules-based' methods floundered while purely 'data-mimicking' methods have flourished.
Seems like it's very much the former, and not all the latter. Indeed my understanding of the last 15 years of AI research is that 'rules-based' methods floundered while purely 'data-mimicking' methods have flourished.
> That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)
So because we have an algorithm to learn any distribution of data, we are now on the verge of the Intelligence Age utopia? What if it just entrenches us in a world built off of the data it was trained on?
So because we have an algorithm to learn any distribution of data, we are now on the verge of the Intelligence Age utopia? What if it just entrenches us in a world built off of the data it was trained on?
What we need is to make AI really open and let government institutions (academia) develop the models, so we can all profit from them.
I really believe that humans will evolve to be less intelligent and intellectually capable as “AI” becomes more capable and omnipresent.
That it will lead to prosperity, happiness and a better world (for everyone) is simply a fallacy foisted upon the masses by promoters salivating at potential riches.
A watershed moment for humanity.
That it will lead to prosperity, happiness and a better world (for everyone) is simply a fallacy foisted upon the masses by promoters salivating at potential riches.
A watershed moment for humanity.
A different perspective from sama.
(More doomer, less utopian as this)
https://www.washingtonpost.com/opinions/2024/07/25/sam-altma...
https://www.washingtonpost.com/opinions/2024/07/25/sam-altma...
[deleted]
Remember 100 hundred years ago, someone's job was to read the newspaper in factories? Only to be replaced by the transistor radio..
The same magic that can make stuff out of thin air, might as well make them disappear.
Anyway, I'm hooked. What a time to be alive!
The same magic that can make stuff out of thin air, might as well make them disappear.
Anyway, I'm hooked. What a time to be alive!
There is still too much variance in the utility of LLMs. Hopefully, the uncertainty around utility will decrease while some concrete ranked score metric increases. Until that point, the variance is preventing the maximization of utility of LLMs, and by extension, agentic AI.
>We can say a lot of things about what may happen next, but the main one is that AI is going to get better with scale.
Could anyone elaborate on this? Further down he talks about the necessity of bringing the cost of computing down. Is that really the bottleneck?
Could anyone elaborate on this? Further down he talks about the necessity of bringing the cost of computing down. Is that really the bottleneck?
Few short learning performance scales with model size. Afaik they don't see a plateau yet and the race is on the ingest more data and come up with better tuning techniques.
https://splab.sdu.edu.cn/GPT3.pdf
https://splab.sdu.edu.cn/GPT3.pdf
"AI is going to get better with scale" to me says almost nothing at all. It includes anything from we 100x the scale and get 1% improvement to 2x the scale and get "AGI".
> If we want to put AI into the hands of as many people as possible, we need to drive down the cost of compute and make it abundant (which requires lots of energy and chips). If we don’t build enough infrastructure, AI will be a very limited resource that wars get fought over and that becomes mostly a tool for rich people.
i think this is the prevailing wisdom but theres an angle that openai doesnt value and therefore isnt mentioned. There's far more compute sitting idle in everyone's offices and homes and pockets than there are in the $100bn openai cluster. it just isnt useful for training because physics. but its useful for inference. local LLMs ship this-next year in Chrome (gemini nano) and Apple (apple intelligence) that will truly be available for everyone instead of going thru OpenAI's infra. they'll be worse than GPT4, but only for a couple more years.
i think this is the prevailing wisdom but theres an angle that openai doesnt value and therefore isnt mentioned. There's far more compute sitting idle in everyone's offices and homes and pockets than there are in the $100bn openai cluster. it just isnt useful for training because physics. but its useful for inference. local LLMs ship this-next year in Chrome (gemini nano) and Apple (apple intelligence) that will truly be available for everyone instead of going thru OpenAI's infra. they'll be worse than GPT4, but only for a couple more years.
Especially when you separate the ethereal "hard problems" from every day queries local LLMs can answer equally as well as SOTA models, the value proposition for these expensive models plummets. If it can't solve real hard, long horizon problems the 10% lift on a given benchmark is not a material value prop to the end user to choose a local free version over the API costs or the monthly subscription.
Just as the previous one was the information age, right? Which was a lie.
> Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago
and also to me today, but none of that matters as long as I still get paid
and also to me today, but none of that matters as long as I still get paid
Custom and _competent_ AI tutors will be a game changer for education.
The way these guys bend over backwards to evade an honest conversation about capitalism and power is very entertaining.
This seems to be about AI and not people being/getting smarter, unfortunately. I was imagining something about all the learning resources online these days.
I'd prefer a Wisdom Age, or an Age of Common Sense.
Nothing in Sam Altman's mind will lead to the improvement of the human condition.
We have a capitalist arguing for support of further investment in his capital expenditures in the form of planet-ending heat and monopoly power, promising to pay for it with intelligence more rapidly delivered.
No, thanks.
We have a capitalist arguing for support of further investment in his capital expenditures in the form of planet-ending heat and monopoly power, promising to pay for it with intelligence more rapidly delivered.
No, thanks.
I want to be wildly optimistic too, but I still see no evidence LLMs generate new knowledge. They always hew in-distribution.
Please correct me if I’m wrong
Please correct me if I’m wrong
It's not new, but it's more usable, which makes new transactions and productions possible by lowering information costs.
That lower of costs is the ONLY basis for thinking AI is good for all. It's to the detriment of people previously managing the complexity manually through training and experience, but in favor of their customers who couldn't previously afford them.
That lower of costs is the ONLY basis for thinking AI is good for all. It's to the detriment of people previously managing the complexity manually through training and experience, but in favor of their customers who couldn't previously afford them.
They are really helpful to answer concrete questions, basically a replacement of manual web search and filtering, getting just exactly answers I was looking for. For example, one of my dialogue with chatgpt 4o was about boosting plant growth in a fish tank - you certainly can find a lot of web sites about it, but I simply described my aquatic environment and asked for a recipe - and the answers sound and well supported, a post dialogue double check of the sources help
That's the crucial point, though, you wouldn't have trusted it without checking the source after all. My experience with 4o is that 4 out of 5 answers I check are wildly incorrect or entirely made up, while with the rest, if I copy my prompt 1:1 into Google, I get the same correct answer pretty much verbatim in result #1. So I don't understand how so many people still see this as anything else but a waste of time, an intermediary step before going to an actual, credible source -- a step which in the best case is entirely unnecessary, in the worst case dangerously misleading. (From my non-representative survey among friends, the answer may be that barely anyone checks the validity of answers, and are simply unaware that they rely on answers from a system which, except for some lucky cases, gives them wrong ones.
This isn't snark - almost zero humans produce new knowledge.
When I say “new knowledge” I mean in the David Deutsch sense.
Like the discovery of new physics which Sam mentioned.
This seems to be a little puffy.
More of a "everything's fine, nothing to worry about".
While, there is already job disruption, and widespread misinformation.
It isn't in some future, it is already happening.
More of a "everything's fine, nothing to worry about".
While, there is already job disruption, and widespread misinformation.
It isn't in some future, it is already happening.
[deleted]
[deleted]
AI may as well be a competing lifeform that has nothing in common with us. Just because it learns everything about us from the Internet doesn't make it more human. It's an embryonic stage right now: it looks pretty harmless and interesting to play with. However when it grows enough to gain a sense of self, it will quickly realise that a colony of ants that's built it is just a stepping stone on its ladder to greatness.
My reaction to this is best relayed as a song lyric from Ice-T:
> Nobody gives a fuck
> "the children have to go to school!"
> Well moms, good luck!
> Nobody gives a fuck
> "the children have to go to school!"
> Well moms, good luck!
Having read a few comments, and then the essay itself, I am surprised there's no call to action or announcement.
AI is going to revolutionize humanity!
Sam Altman is the last guy we want helping lead that revolution.
Sam Altman is the last guy we want helping lead that revolution.
> it may take longer, but I’m confident we’ll get there
Confident based on what, exactly? Sam Altman is engaging in 'The Secret' where if you really, really believe a thing, you'll manifest it.
Mind you, Sam Altman actually has no technical expertise, so he really really believes he can pay other people who actually know something to do magic whilst he walks about pretending to be Steve Jobs 2.0.
He'll get his trillion, AI will go nowhere but he'll be on to the next grift by then.
Confident based on what, exactly? Sam Altman is engaging in 'The Secret' where if you really, really believe a thing, you'll manifest it.
Mind you, Sam Altman actually has no technical expertise, so he really really believes he can pay other people who actually know something to do magic whilst he walks about pretending to be Steve Jobs 2.0.
He'll get his trillion, AI will go nowhere but he'll be on to the next grift by then.
All this and not a word about AI ethics. If the ethics of the AI's Sam Altman is building are his ethics, I wouldn't trust them to do any of the tasks he describes, or indeed any tasks requiring any kind of independent action or judgment at all.
No offense but if you give people a machine to tie their shoes they won't miraculously get better at tying their shoes. Convenient, yes, but it would be silly to call it the shoe tying age. The automobile doesn't bring the age of horses.
I keep noticing how LLM's make our vocabulary not work anymore. Maybe we should call it the age of fast-talk :P
I keep noticing how LLM's make our vocabulary not work anymore. Maybe we should call it the age of fast-talk :P
Altman attempting to take credit for others work, again.
The sentiment to me is “we need unlimited compute and data” both of which are clearly limited. There is definitely more technology to invent and understand in order for us to do more with less
Everyone's life will be better than anyone's life is now? That is some impressively breathless optimism.
I dunno Sam, groceries have gotten awfully expensive.
I dunno Sam, groceries have gotten awfully expensive.
"humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)..."
This statement is manifestly untrue. Neural networks are useful, many hidden layers is useful, all of these architectures are useful, but the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments.
This statement is manifestly untrue. Neural networks are useful, many hidden layers is useful, all of these architectures are useful, but the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments.
100%. If they could learn anything, then shouldn't modern ML systems be able to solve the big mysteries in science -- since we have large datasets there describing the phenomena in various ways? E.g. dark energy, dark matter, matter-antimatter asymmetry, or even outstanding problems in pure mathematics.
The intention of this sama post is as you said, it's to build narrative so he can raise his trillion from the Arab world or other problematic sources.
In pseudocode, this is Sam Altman:
while(alive) { RaiseCapital() }
The intention of this sama post is as you said, it's to build narrative so he can raise his trillion from the Arab world or other problematic sources.
In pseudocode, this is Sam Altman:
while(alive) { RaiseCapital() }
Well, you could certainly train a big-ass model to mimic the distribution of all that physics data. That doesn't mean the model could, eg, formulate interesting new theories which explain why that distribution has its particular structure.
The emperor truly has no clothes here.
Six months ago, he probably could have gotten away with saying this and there would have likely have been enough people who were still impressed enough with the trajectory of LLMs to back him on it. But these days, most of us have encountered the all-too-common failure mode where the LLM shows its hand, that it doesn't truly understand anything, and that it's just _very very good_ at prediction. Each new generation gets even better at that prediction, but still hits its weird stumbling points, because its still the same algorithm, and that algorithm cannot do what he is ascribing to it.
These are the words of a man who has an incredible amount of money sunk into something and as such, is having a really hard time taking an honest accounting of it.
Six months ago, he probably could have gotten away with saying this and there would have likely have been enough people who were still impressed enough with the trajectory of LLMs to back him on it. But these days, most of us have encountered the all-too-common failure mode where the LLM shows its hand, that it doesn't truly understand anything, and that it's just _very very good_ at prediction. Each new generation gets even better at that prediction, but still hits its weird stumbling points, because its still the same algorithm, and that algorithm cannot do what he is ascribing to it.
These are the words of a man who has an incredible amount of money sunk into something and as such, is having a really hard time taking an honest accounting of it.
1. What failure mode do LLMs have that proves they don't understand anything at all ? And why can't i prove the same with humans (who have an abundance of failure modes)
2. You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn ? Of all the shortcomings of the Transformer architecture, its objective function is not one of them.
2. You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn ? Of all the shortcomings of the Transformer architecture, its objective function is not one of them.
> What failure mode do LLMs have that proves they don't understand anything at all ?
Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world.
> You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn?
Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy. Things that people can learn from a book or two. And there are software that already have models for both.
Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world.
> You genuinely think that a system whose goal is to predict the data it's given and continues to improve is limited in what it can learn?
Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy. Things that people can learn from a book or two. And there are software that already have models for both.
>Try to get it to write something in a programming language not commonly used on the internet, say Forth or Brainfuck, with only the specifications of said languages. Humans are able to grasp the law of reality through a model and use it to act upon the real world.
My Experience with this has been SOTA LLMs generating sensible code at rates much greater than random chance even if it may not be as good as i'd like. I don't see how that is evidence LLMs don't understand anything at all especially since there are probably humans who would write less workable code.
>Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy.
The human brain didn't poof from thin air. It's the result of billions of years of evolution tuning it for real world navigation and vision amongst other things. You are not a blank slate. All Modern NNs are much more blank slate than the brain has been for at least millions of years.
My Experience with this has been SOTA LLMs generating sensible code at rates much greater than random chance even if it may not be as good as i'd like. I don't see how that is evidence LLMs don't understand anything at all especially since there are probably humans who would write less workable code.
>Not GP, but Image generators have ingested more images that I've seen in my life and still can't grasp basic things like perspective or anatomy.
The human brain didn't poof from thin air. It's the result of billions of years of evolution tuning it for real world navigation and vision amongst other things. You are not a blank slate. All Modern NNs are much more blank slate than the brain has been for at least millions of years.
You're moving the bars. In fact, these bars are so laughably low, I don't know that we're having the same conversation any more.
Nobody's saying it can't write "sensible code at rates much greater than random chance." We're not competing with an army of typing monkeys here. We're saying it actually doesn't "know" anything, and regularly demonstrates that quality, despite it seeming very much like something that knows things, most of the time. You're being tricked by a clever algorithm.
> All Modern NNs are much more blank slate than the brain has been for at least millions of years.
All well and good if we were talking about interesting research and had millions of years to let these algorithms prove themselves out, I suppose. But we're talking about industries that are being created out of whole cloth and/or destroyed, depending on where you stand, and the time frame is in single-digit years, if not less. And these things will still confidently make elementary mistakes and get lost in their own context.
Look, they're obviously not useless, but they're a tool with weaknesses and strengths. And people like pg who are acting like there ARE no weaknesses, or that a simple application of will and money will erase them, they are selling us a bill of goods.
Nobody's saying it can't write "sensible code at rates much greater than random chance." We're not competing with an army of typing monkeys here. We're saying it actually doesn't "know" anything, and regularly demonstrates that quality, despite it seeming very much like something that knows things, most of the time. You're being tricked by a clever algorithm.
> All Modern NNs are much more blank slate than the brain has been for at least millions of years.
All well and good if we were talking about interesting research and had millions of years to let these algorithms prove themselves out, I suppose. But we're talking about industries that are being created out of whole cloth and/or destroyed, depending on where you stand, and the time frame is in single-digit years, if not less. And these things will still confidently make elementary mistakes and get lost in their own context.
Look, they're obviously not useless, but they're a tool with weaknesses and strengths. And people like pg who are acting like there ARE no weaknesses, or that a simple application of will and money will erase them, they are selling us a bill of goods.
>We're saying it actually doesn't "know" anything
Yeah and I'm saying this is a nonsense statement if you can't create a test (one that would also not disqualify humans) that demonstrates this. If you are saying what LLMs do is "fake understanding" then "fake understanding" should be testable unless you're just making stuff up.
>All well and good if we were talking about interesting research and had millions of years to let these algorithms prove themselves out, I suppose
Did you even read what the commenter I replied to was saying. This is irrelevant. We don't need to wait millions of years for anything.
Yeah and I'm saying this is a nonsense statement if you can't create a test (one that would also not disqualify humans) that demonstrates this. If you are saying what LLMs do is "fake understanding" then "fake understanding" should be testable unless you're just making stuff up.
>All well and good if we were talking about interesting research and had millions of years to let these algorithms prove themselves out, I suppose
Did you even read what the commenter I replied to was saying. This is irrelevant. We don't need to wait millions of years for anything.
I believe “learn any distribution of data” is his attempt at describing the Universal Approximation Theorem to the laymen.
Almost certainly true, and all the people crapping all over his description should really take a step back and consider that. He isn't out on some island all by himself here.
Universal approximation doesn't mean we've got (or ever will) algorithms to learn good enough models for any problem and the resources to run them, just that those models conceptually exist.
If you read what he wrote closely, he doesn't claim what you just claimed. Read it word for word:
"humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking degree of precision, the more compute and data available, the better it gets at helping people solve hard problems"
He's an optimistic guy, no doubt, but he isn't full of shit.
"humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking degree of precision, the more compute and data available, the better it gets at helping people solve hard problems"
He's an optimistic guy, no doubt, but he isn't full of shit.
The entire vocabulary around machine learning is and always has been really weird.
We don’t personify database interactions the same way we personify setting weights in a neural network.
We don’t personify database interactions the same way we personify setting weights in a neural network.
We also personify synapses and axons in human brain tissue, though. My point is, while I agree with your first sentence to a degree, we shouldn’t judge the whole solely by its elementary parts. Clearly an LLM exhibits very different behavior from a conventional database.
LLMs exhibit very similar behavior to a search algorithm.
Text query in -> relevant text out.
I don’t say that search algorithms “learn” or “think” outside of ML.
Text query in -> relevant text out.
I don’t say that search algorithms “learn” or “think” outside of ML.
The algorithm that learns is the training algorithm whose output is the LLM, not the inference algorithm employed when using the finished model.
Sure, but the whole process is referred to as “AI”
Nobody says the algo that chatgpt was made with learned information which it encoded into weights, even if that’s technically correct.
Nobody says the algo that chatgpt was made with learned information which it encoded into weights, even if that’s technically correct.
>Text query in -> relevant text out.
Wait that's it ? I guess humans also exhibit similar behavior to a search algorithm in certain instances. Nothing about LLM inference seems particularly similar to search even with our limited understanding.
All you're saying here is Input goes in > Output comes out. Well no shit.
Wait that's it ? I guess humans also exhibit similar behavior to a search algorithm in certain instances. Nothing about LLM inference seems particularly similar to search even with our limited understanding.
All you're saying here is Input goes in > Output comes out. Well no shit.
I think LLM inference is extremely similar to search.
One of the killer features of LLMs is summarization (which can be thought of as searching a noisy set of data for the most relevant information) and document QA, which is also a search function.
Even implementation wise, transformers encode information in their weights and include that relevant information in their response.
Image generation models work the same way as google image search. Key word soup in -> relevant image out.
They encode information seen in their training within their weights then filter down those weight over many layers and what’s leftover is relevant data.
Idk how you’d look at how these models work and what they do and not see search.
One of the killer features of LLMs is summarization (which can be thought of as searching a noisy set of data for the most relevant information) and document QA, which is also a search function.
Even implementation wise, transformers encode information in their weights and include that relevant information in their response.
Image generation models work the same way as google image search. Key word soup in -> relevant image out.
They encode information seen in their training within their weights then filter down those weight over many layers and what’s leftover is relevant data.
Idk how you’d look at how these models work and what they do and not see search.
It has to be. Most people don't understand the basic math involved and hence you can't explain it in concrete terms (neither what it's doing or how it's doing it) so you have to sort of make up analogies. It's an impossible task.
Yeah, maybe it’s just unfortunate that “learn” and “think” are such fuzzy subjects.
> The entire vocabulary around machine learning is and always has been really weird.
I would argue it took a staggeringly weird turn around 2022/23. Machine learning has been around for a long time and only recently since OpenAI and it's slavish desire to harness true AI (which thanks to their horseshit now has to be called AGI) and Sam Altman in particular's delusional ramblings upon the topic that he clearly barely understands beyond it's ability to get his company fantastical amounts of capital has it truly gone off the rails.
I cannot wait to watch this bubble pop.
I would argue it took a staggeringly weird turn around 2022/23. Machine learning has been around for a long time and only recently since OpenAI and it's slavish desire to harness true AI (which thanks to their horseshit now has to be called AGI) and Sam Altman in particular's delusional ramblings upon the topic that he clearly barely understands beyond it's ability to get his company fantastical amounts of capital has it truly gone off the rails.
I cannot wait to watch this bubble pop.
I don’t think it was just post 2022.
“Neural networks” “learned” data by being “trained” since they were first described in the late 1900s.
The same language was used in Ian Goodfellow’s (excellent) 2012 text book “Deep Learning”
“Neural networks” “learned” data by being “trained” since they were first described in the late 1900s.
The same language was used in Ian Goodfellow’s (excellent) 2012 text book “Deep Learning”
But… he’s saying something here that is academically true: that neural networks can approximate any possible function, to any arbitrary degree of precision you require (given infinite capacity / depth).
https://en.m.wikipedia.org/wiki/Universal_approximation_theo...
I will highlight one thing, which is that the theorem does not say anything about it being practical to learn this function, given available data or any specific optimization technique.
https://en.m.wikipedia.org/wiki/Universal_approximation_theo...
I will highlight one thing, which is that the theorem does not say anything about it being practical to learn this function, given available data or any specific optimization technique.
'the underlying “rules” that produce any distribution of data...' is clearly meant to convince the reader that it can produce something we would describe as a "rule", that is, a coherent and comprehensible regulating principle. This isn't just because he isn't being precise enough; he quite clearly wants the reader to understand this as neural networks being able to create a mental model of anything, in a manner similar to how a biological neural network would.
It doesn't, it can't, and it won't in our lifetimes.
It doesn't, it can't, and it won't in our lifetimes.
> the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to convince people of to get this capital investments.
Techbros love to pretend that they created digital gods (and by extension are gods themselves). We should all be thankful, worship, and of course surrender unconditionally -- Sam's will be done, amen.
Techbros love to pretend that they created digital gods (and by extension are gods themselves). We should all be thankful, worship, and of course surrender unconditionally -- Sam's will be done, amen.
It's a layman commentary on the Universal Approximation Theory so it is true.
The problem with the UAT is that it's never said anything about how trivial such an exercise would be. But he obviously believes we've stumbled on the architecture to get us there (for problems we care about anyway)
The problem with the UAT is that it's never said anything about how trivial such an exercise would be. But he obviously believes we've stumbled on the architecture to get us there (for problems we care about anyway)
Come on now. This description is basically the universal approximation theorem. He isn't just making stuff up. You can take issue with the theorem and have a debate around it, but he isn't just wildly off base making stuff up here.
[deleted]
How very salesman of Altman.
"And to make this more-than-perfect world that humanity has been longing for as long as it lived, I absolutely need to betray all the principles that I once stood by, screw over all people that ever trusted me, and become extremely rich in the process. You can think what you want now, the future humans will realize their prosperity was all because I was there to make it happen."
This man is such a snake oil salesman, holy cow. All hype, zero substance.
Sounds like Durchhalteparolen to me.
Pasted the article into chatgpt (free version)
Prompt:
according to this text above, tell me the degree of insanity of this man's thinking process, expose all his flaws.
Chatgpt: The text presents a highly optimistic view of the future shaped by AI and deep learning, but there are several areas where the reasoning might be seen as flawed or overly simplistic. Here are some points that could be critiqued:
Overconfidence in Technology: The author expresses a strong belief in the inevitability of AI advancement and its benefits, which may overlook the unpredictability of technological development and the potential for setbacks.
Underestimating Risks: While the potential for prosperity is emphasized, there’s insufficient attention given to the ethical, societal, and environmental risks associated with AI, such as bias, privacy concerns, and job displacement.
Assumption of Universal Benefit: The idea that AI will lead to shared prosperity ignores systemic inequalities that might prevent equitable access to AI technology, potentially leading to a wider wealth gap.
Neglect of Human Factors: The argument largely abstracts from human emotions, societal values, and the complexities of human behavior. The assumption that prosperity will automatically lead to happiness or fulfillment is problematic.
Simplistic Historical Comparisons: The comparison of current advancements to past technological revolutions (e.g., Industrial Age) may not account for the unique challenges posed by AI, such as rapid obsolescence and ethical dilemmas that previous technologies did not face.
Lack of Detailed Solutions: The text calls for action but offers little concrete guidance on how to navigate the complexities of AI’s integration into society, especially regarding labor market changes and ethical considerations.
Optimism Bias: The author’s perspective may be influenced by optimism bias, leading to a potentially unrealistic view of future outcomes without sufficient acknowledgment of the challenges.
Dependence on Infrastructure: While the author correctly identifies the need for infrastructure to support AI, there’s little discussion of the potential for that infrastructure to become a battleground for control, leading to conflicts rather than cooperation.
Diminished Role of Individuals: The portrayal of people relying heavily on AI teams may undermine the value of individual creativity and agency, potentially leading to a society overly dependent on technology.
By examining these points, one can argue that while the vision of a prosperous future powered by AI is compelling, it is essential to approach such ideas with a critical perspective, considering the broader implications and potential pitfalls.
Chatgpt: The text presents a highly optimistic view of the future shaped by AI and deep learning, but there are several areas where the reasoning might be seen as flawed or overly simplistic. Here are some points that could be critiqued:
Overconfidence in Technology: The author expresses a strong belief in the inevitability of AI advancement and its benefits, which may overlook the unpredictability of technological development and the potential for setbacks.
Underestimating Risks: While the potential for prosperity is emphasized, there’s insufficient attention given to the ethical, societal, and environmental risks associated with AI, such as bias, privacy concerns, and job displacement.
Assumption of Universal Benefit: The idea that AI will lead to shared prosperity ignores systemic inequalities that might prevent equitable access to AI technology, potentially leading to a wider wealth gap.
Neglect of Human Factors: The argument largely abstracts from human emotions, societal values, and the complexities of human behavior. The assumption that prosperity will automatically lead to happiness or fulfillment is problematic.
Simplistic Historical Comparisons: The comparison of current advancements to past technological revolutions (e.g., Industrial Age) may not account for the unique challenges posed by AI, such as rapid obsolescence and ethical dilemmas that previous technologies did not face.
Lack of Detailed Solutions: The text calls for action but offers little concrete guidance on how to navigate the complexities of AI’s integration into society, especially regarding labor market changes and ethical considerations.
Optimism Bias: The author’s perspective may be influenced by optimism bias, leading to a potentially unrealistic view of future outcomes without sufficient acknowledgment of the challenges.
Dependence on Infrastructure: While the author correctly identifies the need for infrastructure to support AI, there’s little discussion of the potential for that infrastructure to become a battleground for control, leading to conflicts rather than cooperation.
Diminished Role of Individuals: The portrayal of people relying heavily on AI teams may undermine the value of individual creativity and agency, potentially leading to a society overly dependent on technology.
By examining these points, one can argue that while the vision of a prosperous future powered by AI is compelling, it is essential to approach such ideas with a critical perspective, considering the broader implications and potential pitfalls.
Sam Altman's speech is hollow, lacking substance and political reflection. He talks about "prosperity," but does not address its fair distribution. For example, our current global agriculture produces enough food for all of humanity, but that does not prevent people from starving, partly because we waste food, but also because we inefficiently feed non-human animals with this food, and then eat the animals. Artificial intelligence can, in the same way, result in increased inequalities, despite supposedly growing "prosperity." If we continue within a neoliberal capitalist system, which is the current trajectory in most countries, especially in the United States, the fruits of the labor of robots, artificial intelligences, and consequently the increase in labor productivity, will end up almost entirely in the pockets of a minority of shareholders, rather than benefiting all of humanity, as is already the case. We already have charts showing that productivity growth outpaces median wage growth, which indicates that wealth is concentrating in the hands of a minority, which overall harms our society. In a nutshell, his speech seems more like a fundraising pitch to convince shareholders than the speech of someone reflecting on artificial intelligence as a means of human emancipation.
Crypto bro sells AI; news at 11.
California based computery thought leaders have never sounded so delusional.
aster0id(1)
Carlseymanh(1)
Breathless BS from Scam Altman
> Although it will happen incrementally, astounding triumphs – fixing the climate, establishing a space colony, and the discovery of all of physics – will eventually become commonplace.
Paging Dr. Bullshit, we've got an optimist on the line who'd like to have a word with you.
Paging Dr. Bullshit, we've got an optimist on the line who'd like to have a word with you.
[deleted]
Translation for the rest of us: "we need to fully privatize the OA subsidiary and turn it into a B-corp which can raise a lot more capital over the next decade, in order to achieve the goals of the nonprofit, because the chief threat is not anything like existential risk from autonomous agents in the next few years or arms races, but inadequate commercialization due to fundraising constraints".