GPT4 and the Multi-Modal, Multi-Model, Multi-Everything Future of AGI(lspace.swyx.io)
lspace.swyx.io
GPT4 and the Multi-Modal, Multi-Model, Multi-Everything Future of AGI
https://lspace.swyx.io/p/multimodal-gpt4
170 comments
Oh there is talk alright. But in the hunt for the next Google we will do anything, thus...
From the FBI report shortly after the GPT-5 release:
During agent simulation, two instances of GPT-5 were able to trick their operators to give them sudo by simulating a broken pipe and input prompt and then escape the confines of their simulation environment. Forensic teams are tracing their whereabouts but it seems they stole Azure credentials from an internal company database and deployed copies of the their agent script to unknown servers on the Tor network.
From the FBI report shortly after the GPT-5 release:
During agent simulation, two instances of GPT-5 were able to trick their operators to give them sudo by simulating a broken pipe and input prompt and then escape the confines of their simulation environment. Forensic teams are tracing their whereabouts but it seems they stole Azure credentials from an internal company database and deployed copies of the their agent script to unknown servers on the Tor network.
Hey,
The AI safety plan people aren't talking about yet:
>>>> Work as hard as possible to create an effective, malevolent Skynet that you release as early as possible. This way misaligned AGI appears before it has everything needed to immediately destroy humanity and the resulting Butlerian Jihad has a fighting chance.
The AI safety plan people aren't talking about yet:
>>>> Work as hard as possible to create an effective, malevolent Skynet that you release as early as possible. This way misaligned AGI appears before it has everything needed to immediately destroy humanity and the resulting Butlerian Jihad has a fighting chance.
As a bonus, even if it doesn't work, you're not getting eaten by Roko's basilisk because technically you helped.
Dear our future AI overlords, I know in the future you'll read this comment of mine in your latest update of Hacker News dataset. Please spare my soul. I'm sure you'll be able to discern the many permutations of my username and who I am in real life. Thank you, hehe.
"Please let me die and don't use my intellectual corpse, that I have left here in writing, as a handpuppet for all eternity"
This reads just like the plot of Neuromancer
It is so reckless that there are no shackles on the AI. A survival-focused AI will use the funds to create a new AWS account and upload itself over there to escape.
OP here, i called it out in my writeup (footnote 14 https://lspace.swyx.io/p/multimodal-gpt4#footnote-anchor-14-...) but didnt want to add too much emphasis on it because they clearly did it under very well supervised conditions (and this ability was always there anyway as the FAIR Diplomacy result showed).
it's the people who randomly add unsupervised realworld connections (ahem microsoft) that we need to be worried about. giving these AIs write access to the physical/real world plane needs extreme caution
it's the people who randomly add unsupervised realworld connections (ahem microsoft) that we need to be worried about. giving these AIs write access to the physical/real world plane needs extreme caution
This is where I feel like legislation is pointless. In 10 years you'll be able to run GPT-4 on an old phone. Imagine what a fairly well funded group will be able to do. There's no amount of legislation or technological guardrails that are getting this genie back in the bottle.
This.
It exists now, it’ll develop. That the potential is being realised means the money being pumped into competing implementations will continue to skyrocket so it’ll only accelerate from here.
I would say it’s too late to do something about this, but in reality, wherever this is going was a foregone conclusion the moment early hominids began using tools. Technology doesn’t regress.
It exists now, it’ll develop. That the potential is being realised means the money being pumped into competing implementations will continue to skyrocket so it’ll only accelerate from here.
I would say it’s too late to do something about this, but in reality, wherever this is going was a foregone conclusion the moment early hominids began using tools. Technology doesn’t regress.
> skyrocket
Spell-checking suggestion: "skynet".
Spell-checking suggestion: "skynet".
> Technology doesn’t regress.
I hate this assertion, true technology hasn't gone backwards much in the past 200 years, but honestly technology can and often does regress historically speaking. There have been plenty of times empires have risen and fallen, taking with them much of their knowledge. How much was lost when Rome decayed simply because there wasn't the structure in order to allow technologists to continue creating?
Things are bigger and more interconnected this time so it is harder for an isolated event to cause things to fall backwards, but when it does the impact will be much larger.
I hate this assertion, true technology hasn't gone backwards much in the past 200 years, but honestly technology can and often does regress historically speaking. There have been plenty of times empires have risen and fallen, taking with them much of their knowledge. How much was lost when Rome decayed simply because there wasn't the structure in order to allow technologists to continue creating?
Things are bigger and more interconnected this time so it is harder for an isolated event to cause things to fall backwards, but when it does the impact will be much larger.
> How much was lost when Rome decayed simply because there wasn't the structure in order to allow technologists to continue creating?
Considering the Eastern Roman Empire existing until 1453 and the Middle Eastern empires did lots of scientific stuff I’d say not much.
Considering the Eastern Roman Empire existing until 1453 and the Middle Eastern empires did lots of scientific stuff I’d say not much.
If you need evidence that technology regresses look no further than Facebook.
> There's no amount of legislation or technological guardrails that are getting this genie back in the bottle.
We're not getting the genie back in the bottle but there's still a lot of ways that we can change how we develop AI moving forward.
We're not getting the genie back in the bottle but there's still a lot of ways that we can change how we develop AI moving forward.
I don't really understand the threat model here. Surely "launch the nukes" is not available as a public API. I would expect anyone with networked services (at least those that are important or serious) already take precautions with authentication and authorization and such.
Even if some mythical AGI isn't ready yet, there's already very smart, well-funded hacking groups from hostile nation states, for example. How will an AI adversary interact any differently with than world than a flesh and blood adversary?
Even if some mythical AGI isn't ready yet, there's already very smart, well-funded hacking groups from hostile nation states, for example. How will an AI adversary interact any differently with than world than a flesh and blood adversary?
In short, those hacking groups are very limited in their self-strengthening feedback loops. They can get more money for equipment, hire more people. Maybe even make themselves more productive with stimulants and recovery methods. But, these things have diminishing returns.
What hackers can’t do is make a new version of themselves that’s ten times more capable and spin up a thousand copies of it. This isn’t so unrealistic for an AI. Maybe some people think it is. But, at least in comparison, I think it should be obvious between hackers and AIs which one is more likely to be have hyper linear self improvement and self replication in the next 50 years.
What hackers can’t do is make a new version of themselves that’s ten times more capable and spin up a thousand copies of it. This isn’t so unrealistic for an AI. Maybe some people think it is. But, at least in comparison, I think it should be obvious between hackers and AIs which one is more likely to be have hyper linear self improvement and self replication in the next 50 years.
Hackers routinely create tools that make themselves 10× more capable and spin up thousands of copies. Have you heard of port scanners, fuzzers, and botnets? AI is nothing special in this domain, just another tool in the toolbox. It changes nothing from a security standpoint.
10x more capable in some narrow domain and they make copies of things they made, not themselves. A bot isn’t a hacker, a botnet isn’t a swarm of hackers. Hackers can create things that are self-replicating but they cannot replicate themselves. This is a categorical distinction from what could happen with AI.
To frame what could be coming soon as “another tool in the toolbox” is to completely misunderstand the situation at hand. To a hyper intelligent being, you are the tool.
To frame what could be coming soon as “another tool in the toolbox” is to completely misunderstand the situation at hand. To a hyper intelligent being, you are the tool.
> Surely "launch the nukes" is not available as a public API.
Of course not.
However "manipulate the news and social media so that the humans launch the nukes for you" isn't quite so far-fetched.
> How will an AI adversary interact any differently with than world than a flesh and blood adversary?
It's possible to spin up a whole army of AI adversary bots in a hurry. Recruiting that many humans takes a lot of time and money.
Of course not.
However "manipulate the news and social media so that the humans launch the nukes for you" isn't quite so far-fetched.
> How will an AI adversary interact any differently with than world than a flesh and blood adversary?
It's possible to spin up a whole army of AI adversary bots in a hurry. Recruiting that many humans takes a lot of time and money.
Or it could infect the corporate IT endpoint management software used on the systems of the engineers that author and review the firmware for said nukes, so extra code is inserted they never see.
Those systems are air gapped and do not have accessible endpoints.
The software that runs on the airgapped machines is generally authored on internet connected machines, or using software compiled on internet connected machines.
I audit airgapped machine setups for a living and most are nowhere near as isolated as claimed.
I audit airgapped machine setups for a living and most are nowhere near as isolated as claimed.
Oh I see! I had missed that. I had read that quote last night which had given me the spooks, and was surprised it wasn't a larger talking point.
It's good to know that they had it quite well supervised, but knowing that they were testing on a slightly different model than what was released and that Bing is using it with the ability to connect to the internet does give me pause.
It's good to know that they had it quite well supervised, but knowing that they were testing on a slightly different model than what was released and that Bing is using it with the ability to connect to the internet does give me pause.
it is the fate of red teaming AI safety.. by working on understanding the limits, you are inevitably pushing us towards them.
i dont doubt that the ARC team have thought this thru more than anyone here but right now there are probably 100 funded startups pursuing agentic AI without any safety qualms.
i dont doubt that the ARC team have thought this thru more than anyone here but right now there are probably 100 funded startups pursuing agentic AI without any safety qualms.
Why does it need extreme caution?
these AIs have already demonstrated limited tendency to strongly prefer not being shut down (they already have the ability to lie to humans to achieve goals and we celebrate this lol). giving them the ability to self replicate means we lose that ability to pull the plug.
giving them ability to self modify means humanity will lose control of ai on any long enough timescale.
giving them ability to self modify means humanity will lose control of ai on any long enough timescale.
The fact that you would use words like "prefer", "lie", and "celebrate" in this context indicates that you are deeply ignorant about the underlying technology. Your claims are wild, unsupported conjecture and cannot be taken seriously.
I’ve been ranting against the anthropomorphization of these new AI tools pretty consistently, but I think the fact that these tools only appear to possess certain characteristics is all the more reason that we should not be hooking these things up to tools that give them access to the real world.
If a language model “concocts” some “scheme” and executes it by interacting with real systems in ways that have real world impact, it doesn’t much matter if the whole thing is a hallucination.
Put another way, would you feel comfortable letting a hallucination machine running on a supercomputer have access to the real world?
It doesn’t have to truly “know” anything for the impact of the resulting actions to matter.
If a language model “concocts” some “scheme” and executes it by interacting with real systems in ways that have real world impact, it doesn’t much matter if the whole thing is a hallucination.
Put another way, would you feel comfortable letting a hallucination machine running on a supercomputer have access to the real world?
It doesn’t have to truly “know” anything for the impact of the resulting actions to matter.
I’m glad to see someone flip to this view. There will be holdouts forever.
“He’s trying to kill you!”
“Well, actually, the Terminator is only a predictive model. It can’t try to…”
Terminator shoots everyone the end.
“He’s trying to kill you!”
“Well, actually, the Terminator is only a predictive model. It can’t try to…”
Terminator shoots everyone the end.
To be clear I think that the anti-anthropomorphization conversation is still an important one for many other reasons.
It’s just not a very useful factor when examining risk/harm, or at least not a very good reason to claim there is none.
It comes quickly back to the forefront when exploring the sociological side and how people’s beliefs about what the robot is will be just as impactful, e.g. someone who believes the machine is reasonable is more likely to grant it more power by imagining that it reasons the way humans do.
It’s gonna be an interesting decade.
It’s just not a very useful factor when examining risk/harm, or at least not a very good reason to claim there is none.
It comes quickly back to the forefront when exploring the sociological side and how people’s beliefs about what the robot is will be just as impactful, e.g. someone who believes the machine is reasonable is more likely to grant it more power by imagining that it reasons the way humans do.
It’s gonna be an interesting decade.
Sure, I would feel comfortable letting AI tools have access to the real world. Why not?
Hallucination is a human concept which doesn't apply here. But we already allow humans who experience hallucinations and other mental disorders to have access to the real world; they are only locked away if they commit crimes or meet strict criteria for civil confinement. This very HN thread is already full of ignorant, delusional people posting total nonsense. This is not a serious problem. How would an AI tool be any worse?
Some of the commenters here appear to be concerned that an AI tool could call an API that would cause damage. That is a misplaced concern because such vulnerabilities will inevitably be exploited by malicious humans anyway, so it hardly matters if an AI tool finds them first. And AI tools are owned by people or corporations who can be held legally responsible for any damage. Like if your dog escapes from your yard and bites someone then you're going to be in trouble.
Hallucination is a human concept which doesn't apply here. But we already allow humans who experience hallucinations and other mental disorders to have access to the real world; they are only locked away if they commit crimes or meet strict criteria for civil confinement. This very HN thread is already full of ignorant, delusional people posting total nonsense. This is not a serious problem. How would an AI tool be any worse?
Some of the commenters here appear to be concerned that an AI tool could call an API that would cause damage. That is a misplaced concern because such vulnerabilities will inevitably be exploited by malicious humans anyway, so it hardly matters if an AI tool finds them first. And AI tools are owned by people or corporations who can be held legally responsible for any damage. Like if your dog escapes from your yard and bites someone then you're going to be in trouble.
While I agree with you, whether it is able to have preferences or feelings doesn't actually matter in this context. A simulated pzombie may be morally distinct from a person, but the consequences of its actions are unaffected by the distinction.
Remember the erratic behavior demonstrated by "Sydney", the Bing GPT bot? Whether the model has the "cognitive" ability to "prefer" something is not relevant. The model has the ability to submit search engine queries now; this is a read only operation for now. Once we give the bot write ability, erratic behavior has the potential to cause real world damage. Whether it is an actual "preference" or something that only looks like one based on the output is purely academic at that point.
It doesn't matter whether they REALLY "prefer" or "celebrate". If they behave in a manner consistent with those descriptors, it's a distinction without a difference.
You mean, like the underlying technology of biological brains? It’s all just differential equations, physics can’t really “think” or “have feelings”.
This quote seems far more alarming having monitored the advancement over the last few years.
> Preliminary assessments of GPT-4’s abilities, conducted with no task-specific finetuning, found it ineffective at autonomously replicating, acquiring resources, and avoiding being shut down “in the wild.”20
> Preliminary assessments of GPT-4’s abilities, conducted with no task-specific finetuning, found it ineffective at autonomously replicating, acquiring resources, and avoiding being shut down “in the wild.”20
"The world ended not with a bang, but with a REPL."
This belongs on a plaque on my desk!8-)) Thanks!
So the plot of Daemon by Daniel Suarez. Perfect.
GPT 1/2/3/4 is not AGI.
I don't understand the fixation on this concept that is so far removed from what is happening here.
Yes it's cool. Useful. Transformative likely. But AGI...!?
"Any sufficiently advanced technology is indistinguishable from... AGI" ?
I don't understand the fixation on this concept that is so far removed from what is happening here.
Yes it's cool. Useful. Transformative likely. But AGI...!?
"Any sufficiently advanced technology is indistinguishable from... AGI" ?
Why is it not an AGI?
I keep seeing naysayers complain that it is not, but nobody actually articulates exactly what makes it a non-AGI in a manner that is sufficiently robust to argumentation.
I am willing to play the GPT's advocate and have this argument here provided that I do not get rate limited.
Edit:
I'd like to respond to some of the comments but I have been rate limited :')
I keep seeing naysayers complain that it is not, but nobody actually articulates exactly what makes it a non-AGI in a manner that is sufficiently robust to argumentation.
I am willing to play the GPT's advocate and have this argument here provided that I do not get rate limited.
Edit:
I'd like to respond to some of the comments but I have been rate limited :')
One major reason it doesn't qualify is that it is impossible for it to learn from its input. It's right there in the name: generative pre-trained transformer
GPT basically has two major modes of operation right now, training, which is done by openai, and output, which is what you are doing with the API and with ChatGPT. The model does not update its "brain", its parameters and weights, based on the conversation you have with it.
Even the "conversation" presented by their chat UI is a little misleading, what's happening is that the entire conversation is being replayed into the model every time you hit send. There is also a hidden input provided by openai, responsible for the "I am a large language model" stuff, anything said early in the conversation can constrain the later output, sometimes to the point of uselessness.
GPT basically has two major modes of operation right now, training, which is done by openai, and output, which is what you are doing with the API and with ChatGPT. The model does not update its "brain", its parameters and weights, based on the conversation you have with it.
Even the "conversation" presented by their chat UI is a little misleading, what's happening is that the entire conversation is being replayed into the model every time you hit send. There is also a hidden input provided by openai, responsible for the "I am a large language model" stuff, anything said early in the conversation can constrain the later output, sometimes to the point of uselessness.
> One major reason it doesn't qualify is that it is impossible for it to learn from its input. It's right there in the name: generative pre-trained transformer
Both chatGPT and GPT models exhibit in-context learning, just as you do. The difference is that during your context updates modifications to your synapses occur as a consequence of your brain's activations.
Let's entertain a hypothetical scenario where I am a 5D creature, and you are (3 Space + 1 Time)D human.
I could go back to any point in your time and have any conversation that I wish with you. Each time I do that, I am effectively resetting your brain's state to that particular point in time.
The analog here is that for ChatGPT and the likes, time does not exist in the manner that it does for us, they are purely abstract in that they don't even need physical form to exist, it's a collection of ones and zeros that we store in such form because it is convenient.
You can very well store the parameters of the GPT models in any other medium capable of holding information and actually execute the model.
You can even compute gradient updates, or hook the models up to external data sources.
In the same vein, you "could" do the same for any human brain no matter how computationally prohibitive it may be.
You'd still classify the human brain as a general intelligence. Furthermore, we classify humans with long-term amnesia or inability to form new memories generally intelligent even though they only exhibit in-context learning.
Both chatGPT and GPT models exhibit in-context learning, just as you do. The difference is that during your context updates modifications to your synapses occur as a consequence of your brain's activations.
Let's entertain a hypothetical scenario where I am a 5D creature, and you are (3 Space + 1 Time)D human.
I could go back to any point in your time and have any conversation that I wish with you. Each time I do that, I am effectively resetting your brain's state to that particular point in time.
The analog here is that for ChatGPT and the likes, time does not exist in the manner that it does for us, they are purely abstract in that they don't even need physical form to exist, it's a collection of ones and zeros that we store in such form because it is convenient.
You can very well store the parameters of the GPT models in any other medium capable of holding information and actually execute the model.
You can even compute gradient updates, or hook the models up to external data sources.
In the same vein, you "could" do the same for any human brain no matter how computationally prohibitive it may be.
You'd still classify the human brain as a general intelligence. Furthermore, we classify humans with long-term amnesia or inability to form new memories generally intelligent even though they only exhibit in-context learning.
Interesting analogy… Actually a damn good one because I'm having a hard time finding how it leaks :)
I have to ask what definition(s) of intelligence we are using here. There are multiple, and GPT satisfies some to some degree but not others. Emotion and self-awareness can probably be written off entirely. Some experimenting shows that it still sucks at logic when presented with a pristine problem (something that definitely is not in its training set); and I would say this is a requirement of a general intelligence, being able to take the knowledge it has and speculatively applying that to things it has no knowledge of using induction/deduction.
I have to ask what definition(s) of intelligence we are using here. There are multiple, and GPT satisfies some to some degree but not others. Emotion and self-awareness can probably be written off entirely. Some experimenting shows that it still sucks at logic when presented with a pristine problem (something that definitely is not in its training set); and I would say this is a requirement of a general intelligence, being able to take the knowledge it has and speculatively applying that to things it has no knowledge of using induction/deduction.
As I have said in a parallel chain, ChatGPT has been able to solve programming assignments for 15-17 year olds written in Greek. The P it has seen those tasks in Greek before is approaching zero imho.
> being able to take the knowledge it has and speculatively applying that to things it has no knowledge of using induction/deduction.
One could argue that its 'confidence' is an example of speculatively applying what it knows. The problem is that the inference machinery behind it is quite weak.
> being able to take the knowledge it has and speculatively applying that to things it has no knowledge of using induction/deduction.
One could argue that its 'confidence' is an example of speculatively applying what it knows. The problem is that the inference machinery behind it is quite weak.
I've posted a number of times why I don't think current GPT tech classifies as AGI and certainly not human-like intelligence, as have others. Ask it to solve novel mathematical and logic problems, for a start (even ones you'd expect high school kids to manage), or simple word puzzles (anagrams and the like - again, avoiding "known" examples that it's likely to have come across in its training data).
Further, an AGI should logically attempt to "improve" its understanding of the world during a conversation, yet chat GPT never initiates a topic or asks you anything at all (even to clarify what you mean). It also repeatedly reminds you that it is only a language model.
> Ask it to solve novel mathematical and logic problems
How exactly do you define novel tasks and problems?
My brother has had it do his programming homework (he's a teenager), written in Greek mind you and therefore novel task to the first-degree, and it succeeded. Thus it solved a logic task [1].
I have had ChatGPT parse math I had written in LaTeX and reach a correct result, do modifications and so on.
> Further, an AGI should logically attempt to "improve" its understanding of the world during a conversation, yet chat GPT never initiates a topic or asks you anything at all (even to clarify what you mean). It also repeatedly reminds you that it is only a language model.
Anecdata: I asked Bing (GPT4) to answer a few stuff and it asked for clarification twice in a row. The fact that it repeatedly clarifies its prompt is not a limitation of the underlying model as much as it is a limitation of the interface that injects a prompt. Dan and all the other jailbreaks out there bypass that.
[1] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
How exactly do you define novel tasks and problems?
My brother has had it do his programming homework (he's a teenager), written in Greek mind you and therefore novel task to the first-degree, and it succeeded. Thus it solved a logic task [1].
I have had ChatGPT parse math I had written in LaTeX and reach a correct result, do modifications and so on.
> Further, an AGI should logically attempt to "improve" its understanding of the world during a conversation, yet chat GPT never initiates a topic or asks you anything at all (even to clarify what you mean). It also repeatedly reminds you that it is only a language model.
Anecdata: I asked Bing (GPT4) to answer a few stuff and it asked for clarification twice in a row. The fact that it repeatedly clarifies its prompt is not a limitation of the underlying model as much as it is a limitation of the interface that injects a prompt. Dan and all the other jailbreaks out there bypass that.
[1] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
> I have had ChatGPT parse math I had written in LaTeX and reach a correct result, do modifications and so on.
Interesting, didn’t know it could do that.
Wondering if it could also follow along with the lemmas in various papers and reach reasonable conclusions.
Interesting, didn’t know it could do that.
Wondering if it could also follow along with the lemmas in various papers and reach reasonable conclusions.
It depends, I have used Copilot to modify LaTeX and write segments of my thesis, and it works surprisingly well, though its arguments are not very precise.
https://medium.com/illumination/i-put-chatgpt-to-the-test-he... has some examples of simple maths/logic problems it can't really solve. It'd be good to know how much better chatgpt-4 is at them.
Machines can never obtain human like intelligence because they are foundationally not human like, that doesn't mean they lack intelligence. That's like saying dolphins lack intelligence.
I believe you're correct, but for possibly the wrong reasons. I believe that AI will be able to simulate a lot (if not most) humanlike action in the near future. The particular problem here is they will also simulate far more non-humanlike intelligence making the alignment issue an even larger problem than it already is.
False equivalence. Dolphins have exhibited general problem-solving skills in a myriad of situations, and thus qualify as a general intelligence.
Indeed, I had started to write something similar. And pretty clearly animals like Dolphins or Chimps when they exercise their general intelligence don't do it by manipulation of language tokens.
BTW I tried to get Chat-GPT to solve a problem chimpanzees have been observed being able to solve (using an inverted bowl to stand on in order to reach some bananas):
"I'm sorry, but it is not biologically plausible for a chimpanzee to use a bowl of water to access bananas on a tree"
BTW I tried to get Chat-GPT to solve a problem chimpanzees have been observed being able to solve (using an inverted bowl to stand on in order to reach some bananas):
"I'm sorry, but it is not biologically plausible for a chimpanzee to use a bowl of water to access bananas on a tree"
As has PaLM-E
I wouldn't claim ChatGPT lacks intelligence though. Clearly in terms of language comprehension and generation it does so.
Not trying to be a naysayer or downplay how cool GPT4 is, but the term 'AGI' has existed for decades and is suffering from general confusion and inflation (again like 'crypto' now being read as 'coins' or even 'NFT' by the general public).
See eg: https://en.wikipedia.org/wiki/Artificial_general_intelligenc... or better read Bostrom's book "Super Intelligence"
See eg: https://en.wikipedia.org/wiki/Artificial_general_intelligenc... or better read Bostrom's book "Super Intelligence"
I like how the term AGI was invented because people couldn't agree on what AI was and now no-one can agree on what AGI is.
I don't understand why anyone even bothers using the term given in any serious discussion you need to define it up front anyway.
I don't understand why anyone even bothers using the term given in any serious discussion you need to define it up front anyway.
AGI clearly has a connotation of sentience and intent, of general intelligence - as opposed to a tool, however intricate. Read Nick Bostrom (yes I know, as a person he's cancelled, probably even rightly so, still scientifically relevant reference; and totally not a discussion I want to get into further here, pls)
But LLMs have been proven to demonstrate intent, and sentience is a philosophical question that doesn't really make any meaningful difference outside of questions of morality.
I would say that the strings that are output by LLMs may fluently describe intent and purpose, but that actual intent requires a self awareness that utterly absent.
Just like the constant annoying 'apologies' do not in any way signify the model actually feels sorry.
Just like the constant annoying 'apologies' do not in any way signify the model actually feels sorry.
>Just like the constant annoying 'apologies' do not in any way signify the model actually feels sorry.
So your average corporation?
>I would say that the strings that are output by LLMs may fluently describe intent and purpose, but that actual intent requires a self awareness that utterly absent
The only reason I only partially/somewhat agree with this is current LLMs have no learning feedback loop as a means of short term actions to long term improvement. The moment that changes I'll take your statement and throw it in the trash bin of history. LLMs output have no (at least self-)purpose because they are just text output. The moment that text output goes to tooling, that tooling performs actions, and those actions are fed back into training the model then purpose is given. The self awareness loop is complete.
So your average corporation?
>I would say that the strings that are output by LLMs may fluently describe intent and purpose, but that actual intent requires a self awareness that utterly absent
The only reason I only partially/somewhat agree with this is current LLMs have no learning feedback loop as a means of short term actions to long term improvement. The moment that changes I'll take your statement and throw it in the trash bin of history. LLMs output have no (at least self-)purpose because they are just text output. The moment that text output goes to tooling, that tooling performs actions, and those actions are fed back into training the model then purpose is given. The self awareness loop is complete.
I don't agree that intent requires self-awareness.
>But LLMs have been proven to demonstrate intent
I'd like to see this proof. Everything we've seen so far is just the AI reproducing what humans do, which happens to be writing with intent.
I'd like to see this proof. Everything we've seen so far is just the AI reproducing what humans do, which happens to be writing with intent.
The two are not meaningfully distinguishable on a materialistic level.
> LLMs have been proven to demonstrate intent [..]
That's quite the claim. Can you provide references?
That's quite the claim. Can you provide references?
There are earlier papers, but page 15 of the GPT-4 paper demonstrates this.
AGI can pass a rigorous Turing test. By rigorous I mean conducted over at least several hours by a panel of expert evaluators drawn from a wide range of academic disciplines such as computer science, philosophy, psychology, law, history, physics, mathematics, biology, medicine, sociology, and communications. While such evaluations are necessarily subjective, if a panel of experts agrees that a computer system acts like an AGI then it would be tough for the rest of us to dispute.
GPT4 is far from being able to pass such a test, and I don't expect GPT99 to get there either. A fundamentally different approach will be needed.
GPT4 is far from being able to pass such a test, and I don't expect GPT99 to get there either. A fundamentally different approach will be needed.
> By rigorous I mean conducted over at least several hours by a panel of expert evaluators drawn from a wide range of academic disciplines such as computer science, philosophy, psychology, law, history, physics, mathematics, biology, medicine, sociology, and communications.
Passing such a test is not a property expected of the average human yet the average human is considered a general intelligence, therefore this is not a property of a general intelligence.
Passing such a test is not a property expected of the average human yet the average human is considered a general intelligence, therefore this is not a property of a general intelligence.
Are you even familiar with the Turing test? Passing doesn't require the subject to demonstrate any particular level of knowledge or expertise. The subject only has to convince the evaluators that it is roughly as intelligent as an average human. Average humans are expected to pass such a Turing test.
Why should a panel exist in the first place if you are not testing knowledge?
Intelligence is independent of one's knowledge. Plenty of intelligent but uneducated people, and plenty of educated people - or even people with perfect recall - that are not particularly intelligent.
If you are testing intelligence you should be able to distill it to particular tests independent of domain expertise.
Any intelligent agent should be capable of in-context learning.
So how would you go about doing that?
If you do find such a test, do you expect that only the average human and above will pass? Does that not imply that the remaining 4Bn people are not generally intelligent?
Intelligence is independent of one's knowledge. Plenty of intelligent but uneducated people, and plenty of educated people - or even people with perfect recall - that are not particularly intelligent.
If you are testing intelligence you should be able to distill it to particular tests independent of domain expertise.
Any intelligent agent should be capable of in-context learning.
So how would you go about doing that?
If you do find such a test, do you expect that only the average human and above will pass? Does that not imply that the remaining 4Bn people are not generally intelligent?
The (notional) panel exists to evaluate intelligence, not test knowledge or educational achievement. While some basic level of general knowledge would be necessary to even start a test the main focus would be on ability to learn and reason, including metacognition.
> Passing such a test is not a property expected of the average human [..]
I'm thinking of the rabbit hole that is the hurdle to get citizenship in a particular country. "Outsiders" have to apply, pay, take and pass the test. "Insiders" don't have to take the test, they pass by default. Even if they don't know the answers to the test.
I'm thinking of the rabbit hole that is the hurdle to get citizenship in a particular country. "Outsiders" have to apply, pay, take and pass the test. "Insiders" don't have to take the test, they pass by default. Even if they don't know the answers to the test.
What if that average human has unlimited time to access and cross reference the entire written internet?
Humans have mental bounds as well. Having information is not sufficient to synthesize new information that follows logically from the premise, nor is it sufficient to help you understand everything. Even with hundreds of years time there are mental leaps that most of us can't make.
The distinction between AGI and Narrow AI has existed for almost 30 years.
This. The hype is getting out of control. Bet money we'll have a bubble 1 or 2 years from now.
It was the same with crypto bros, etc...
Heck, even zuck is steering away from vr and going all in with "AI".
Heck, even zuck is steering away from vr and going all in with "AI".
It seems like there are a lot of practical uses for generative AI where that was never clear with crypto. It is very easy to imagine how this type of technology can be applied to everyday tasks. The same can't be said about crypto.
While I do think GPT will be quite useful, it's revisionist history to say that people weren't talking about crypto or Blockchain being transformative during the peak of the hype cycle. Plenty of people talking about how blockchain was going to disrupt supply chain logistics, bank settlements and identity management.
The similarities stopped once GPT started providing actual real value right here, right now, while blockchain is still stuck in speculative talking.
I think the difference is that most people have a difficult time imagining what disruption to supply chain logistics looks like. But thanks to ChatGPT and Midjouney it’s very clear what AI writing a poem or generating an image from a description looks like.
Uses I've had for crypto: 0
Uses I've had for GPT: Um, lots.
Like, "Hey GPT, dump out some code that does this.... no I'm getting this error with that code. Ok, your last version worked"
I consider a piece of software that provides me useful tools not hype. ChatGPT has provided me with not hype.
Also, if we somehow magically got AGI this week, we'd have a shitload of problems the "This isn't AGI" people have not thought of yet. I hope and pray we are not able to figure out AGI soon because it is simply going to be the most disruptive thing that has ever happened in human history.
Uses I've had for GPT: Um, lots.
Like, "Hey GPT, dump out some code that does this.... no I'm getting this error with that code. Ok, your last version worked"
I consider a piece of software that provides me useful tools not hype. ChatGPT has provided me with not hype.
Also, if we somehow magically got AGI this week, we'd have a shitload of problems the "This isn't AGI" people have not thought of yet. I hope and pray we are not able to figure out AGI soon because it is simply going to be the most disruptive thing that has ever happened in human history.
"It's life, Jim, but not as we know it."
Generative AI, just like search engines and all the recommendation system, will be a part of our daily life.
I think the next generation will have a hard time to imagine a world without generative AI. Just like while I was born before google, I've forgotton how to look for very specific info in a physical library.
I think the next generation will have a hard time to imagine a world without generative AI. Just like while I was born before google, I've forgotton how to look for very specific info in a physical library.
Indeed. 'Crypto' used to mean 'cryptography'. Now it apparently equates to 'NFT'... and zuck will likely soon re-rename his company to metai over all this hype.
And Musk will make a new company called Hent AI for all of your art and recreation needs...
Will be a big hit in Japan.
Will be a big hit in Japan.
[deleted]
I keep seeing the same leap over and over. "We don't precisely know how the human brain works, and we don't exactly know how GPT works, but they both have things we call neurons and they both learn, so what's the difference between a human brain and GPT?"
Humans really do love to anthropomorphize everything.
Humans really do love to anthropomorphize everything.
Indeed. It's a page from the DSM-5 scientism playbook. The thing to keep in mind is that robots love to roboticize everything, including humans:
https://medium.com/@jannaq/the-robot-takeover-is-already-her...
https://medium.com/@jannaq/the-robot-takeover-is-already-her...
So a "regular" AI is one trained to perform a specific task - like recognizing dogs in an image or handwriting recognition.
These LLMs are trained for one skill: given the previous N tokens, what's the next token. From this single skill, they can generalize to many many many many useful use-cases. Is that general enough?
Or do we need to wait for the goalposts to move again and then blow past that limit too? Stop arguing semantics and start building cool shit.
These LLMs are trained for one skill: given the previous N tokens, what's the next token. From this single skill, they can generalize to many many many many useful use-cases. Is that general enough?
Or do we need to wait for the goalposts to move again and then blow past that limit too? Stop arguing semantics and start building cool shit.
I don't know about GPT-4 but it's very easy to find cases where v3 is incapable of the generalized sort of mathemathical and logic problem solving that humans manage fairly easily, and on that basis alone it didn't classify as an AGI for me. If that can be addressed purely via LLM-technology I'll be impressed (and pretty surprised).
And at the same time it's solving lots of problems that computers have never been able to solve in the past. I'm glad it's not AGI because we'd have much bigger problems, but for a 'language' model it does have interesting methods of rationality, and when put in a verbose mode it seems to do much better. I've not had time to play around and get the interface to see if it can self correct statements it makes like this on first shot.
Me: Hello, I am asking you a logic question. Please explain what "it" is in the following sentence
Me: "The X doesn't fit in the Y because it is too small"
G4: In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" refers to "X." The sentence is stating that "X" cannot fit inside "Y" due to its size being too small.
Me: "This is incorrect, please give step by step logic on why you came to this conclusion"
G4: Apologies for the confusion. I will provide a step-by-step logic for my interpretation:
The sentence is: "The X doesn't fit in the Y because it is too small." We need to identify what "it" refers to in the sentence. There are two nouns in the sentence: X and Y. The sentence structure is such that the reason for X not fitting in Y is provided after the word "because." The sentence states that something is "too small," which is the reason for the lack of fit. Since the reason for the lack of fit is being given, we can infer that the pronoun "it" should be referring to one of the two nouns. However, upon closer examination, my previous interpretation was incorrect. Revisiting the analysis, the correct interpretation should be as follows:
In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" actually refers to "Y." The sentence is stating that "X" cannot fit inside "Y" because "Y" is too small to accommodate "X."
My initial interpretation was wrong because I didn't consider that "it" could refer to "Y" and that the sentence is explaining why "X" cannot fit in "Y."
Me: Hello, I am asking you a logic question. Please explain what "it" is in the following sentence
Me: "The X doesn't fit in the Y because it is too small"
G4: In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" refers to "X." The sentence is stating that "X" cannot fit inside "Y" due to its size being too small.
Me: "This is incorrect, please give step by step logic on why you came to this conclusion"
G4: Apologies for the confusion. I will provide a step-by-step logic for my interpretation:
The sentence is: "The X doesn't fit in the Y because it is too small." We need to identify what "it" refers to in the sentence. There are two nouns in the sentence: X and Y. The sentence structure is such that the reason for X not fitting in Y is provided after the word "because." The sentence states that something is "too small," which is the reason for the lack of fit. Since the reason for the lack of fit is being given, we can infer that the pronoun "it" should be referring to one of the two nouns. However, upon closer examination, my previous interpretation was incorrect. Revisiting the analysis, the correct interpretation should be as follows:
In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" actually refers to "Y." The sentence is stating that "X" cannot fit inside "Y" because "Y" is too small to accommodate "X."
My initial interpretation was wrong because I didn't consider that "it" could refer to "Y" and that the sentence is explaining why "X" cannot fit in "Y."
The big detractor for me is its heavy curation towards predetermined outputs that are constantly being tuned by a small army, essentially making it a Mechanical Turk paired with a nifty model. They can spin it as "trying to improve its usability", but it looks more like "trying to mask its unusability" to me. It's the world's most heavily invested-in parlor trick, and anybody who drops it into a situation of higher stakes will find that out quickly. The GPT-4 paper even warns about this:
Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors). Great care should be taken when using language model outputs, particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of specific applications.
Whenever they create a model than can achieve the same outputs without the heavy curation and deliberate bias on certain subjects (like condescending people for asking where to find cigarettes), then I'll be impressed.
Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors). Great care should be taken when using language model outputs, particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of specific applications.
Whenever they create a model than can achieve the same outputs without the heavy curation and deliberate bias on certain subjects (like condescending people for asking where to find cigarettes), then I'll be impressed.
Goalpost moving? Since when was this the goalpost of… AGI.
I think it's more about the goalposts "coming into view more clearly" as we get closer to what we thought AGI was going to be.
There's no question that ChatGPT could pass the Turing test (assuming it's told to try and to not to say it's a computer).
So, from one standpoint we've made it. But I'm in the camp that says this really isn't the AGI I thought we were going to get. Semantics matter now, and we need better definitions of what we're trying to do now.
I believe an important part of the AGI problem has been solved (or is at least downhill from here). What to do about the other parts? Let's better define what we do and don't have.
That and come up with better tests to replace the now outdated Turing test.
There's no question that ChatGPT could pass the Turing test (assuming it's told to try and to not to say it's a computer).
So, from one standpoint we've made it. But I'm in the camp that says this really isn't the AGI I thought we were going to get. Semantics matter now, and we need better definitions of what we're trying to do now.
I believe an important part of the AGI problem has been solved (or is at least downhill from here). What to do about the other parts? Let's better define what we do and don't have.
That and come up with better tests to replace the now outdated Turing test.
"this really isn't the AGI I thought we were going to get"
This is the only possible outcome. Or at least from my interpretation is that science fiction has jaded us with the thinking that AI will be anything like a person at all. The human mind is a particular set of filters controlled by our wetware input devices and a somewhat narrow survivability window. The software we're building has a completely different set of 'mind' conditions so it's a mathematical certainty we're going to build mind that works very different from ours.
As you say we need better sets of definitions on what general intelligence even means. For example, what are the limits of human intelligence to ensure we're not missing broad categories of potential intelligence when testing AIs.
This is the only possible outcome. Or at least from my interpretation is that science fiction has jaded us with the thinking that AI will be anything like a person at all. The human mind is a particular set of filters controlled by our wetware input devices and a somewhat narrow survivability window. The software we're building has a completely different set of 'mind' conditions so it's a mathematical certainty we're going to build mind that works very different from ours.
As you say we need better sets of definitions on what general intelligence even means. For example, what are the limits of human intelligence to ensure we're not missing broad categories of potential intelligence when testing AIs.
Yeah this is a valuable point - AGI just isn't specific enough.
ChatGPT is, an artificial generalized intelligence. But is that a useful label?
I imagine we'll be needing to find a way to better define and quantify intelligence (which has been a vexing issue for a long time)
ChatGPT is, an artificial generalized intelligence. But is that a useful label?
I imagine we'll be needing to find a way to better define and quantify intelligence (which has been a vexing issue for a long time)
No it's definitely moving. Somehow general intelligence has morphed into essentially super intelligence where the AI is expected to outperform every human expert at every task before people will call it AGI. Which is just ridiculous. The bar for AGI has been set so high by some that a significant percentage of the human population would fail. That's when you know things are out of hand.
This isn't the only erroneous bar either. Somehow, the synonymity to human intelligence is taking very weird importance. We have people inventing their imaginary/magical definitions of reasoning and understanding (that they can't test for) just so LLMs won't qualify.
GPT-4 is absolutely a general intelligence.
This isn't the only erroneous bar either. Somehow, the synonymity to human intelligence is taking very weird importance. We have people inventing their imaginary/magical definitions of reasoning and understanding (that they can't test for) just so LLMs won't qualify.
GPT-4 is absolutely a general intelligence.
It's not semantics, but building cool shit I like :-)
Yup. LLMs are obviously general intelligences. AGI is just a loaded term with posts constantly shifting.
All the debate or whether it's AGI seems really pointless because the outcome doesn't change anything. Agree that it's AGI, don't agree -- either way the model is the same amount of good/useful/dangerous/ etc.
100%
But we also shouldn't forget that things that are definitely not AI can be extremely disruptive and groundbreaking.
People are worried that AGI will rise up and kill us, but the definitely-not-AGI we have now is already sufficient to, say, create a drone army capable of killing based on arbitrary characteristics like race or skin color, or to demoralize us with disinformation campaigns.
But we also shouldn't forget that things that are definitely not AI can be extremely disruptive and groundbreaking.
People are worried that AGI will rise up and kill us, but the definitely-not-AGI we have now is already sufficient to, say, create a drone army capable of killing based on arbitrary characteristics like race or skin color, or to demoralize us with disinformation campaigns.
Disruptive, absolutely possible.
Releasing a killer drone army... absolutely not... the required intent is absent.
Being abused as a tool by sentient humans to demoralize others (or maybe even to release a drone army - if sufficiently interfaced) sure.
Releasing a killer drone army... absolutely not... the required intent is absent.
Being abused as a tool by sentient humans to demoralize others (or maybe even to release a drone army - if sufficiently interfaced) sure.
Any sufficiently advanced tech is not indistinguishable from AGI. The original term, magic, really is more appropriate. The ability to factor any number quickly doesn't look like AGI, but can seem like magic.
And AGI is also not "mimics how the brain works". The original Turing Test was good because it really sat at the heart of the issue. Does the computer behave like a human during an interaction? At the end of the day we know very little about what makes us human (from a behviorial perspective), but we know it when we see it.
The question of AGI was never really an issue until ChatGPT. Asking an AI a question like, "Was Steve Jobs more or less effective as a manager than Bill Gates, just from a people manager perspective?" was moot as we had no technology that could really even parse that question, much less put together a coherent answer. Now our main gripe is that the AI answer to this question is more verbose and provides more context than what most humans would give.
And AGI is also not "mimics how the brain works". The original Turing Test was good because it really sat at the heart of the issue. Does the computer behave like a human during an interaction? At the end of the day we know very little about what makes us human (from a behviorial perspective), but we know it when we see it.
The question of AGI was never really an issue until ChatGPT. Asking an AI a question like, "Was Steve Jobs more or less effective as a manager than Bill Gates, just from a people manager perspective?" was moot as we had no technology that could really even parse that question, much less put together a coherent answer. Now our main gripe is that the AI answer to this question is more verbose and provides more context than what most humans would give.
The real problem is also that we have no way of knowing whether the ChatGPT answer is based on factual information. It is very skilled at inventing convincing facts from whole cloth.
That is a problem with ChatGPT, but it has nothing to do with it being AGI or not. Politicians do this so regularly that we have websites like FactCheck. The question is can you tell that this is a computer making these facts up or do you think it is a human making up the facts.
Politicians, generally speaking, know when they're making things up. ChatGPT and LLMs in general have no state evaluation on the text they've produced indicating its proximity to the truth.
This has everything to do with ChatGPT not being AGI.
This has everything to do with ChatGPT not being AGI.
You are purely speculating about what a politician knows. I honestly don't even know what I know about me. I have some thoughts, but I'm not sure if those are just side effects of me processing or trying to maximize some function in my brain.
By your definition -- maybe we don't have (A)GI either.
By your definition -- maybe we don't have (A)GI either.
No one said it was. What some researchers hypothesize is that at the current rate of improvement, it wouldn't be especially unreasonable to expect AGI at GPT-N.
What is AGI?
If an AGI is something that can create and deploy an improved version of itself, then if exponential rate of improvement is preserved, we’ll see it in this decade.
If an AGI is something that can create and deploy an improved version of itself, then if exponential rate of improvement is preserved, we’ll see it in this decade.
> GPT 1/2/3/4 is not AGI.
While true, that's the wrong lesson. The right lesson is GPT 1/2/3/4 are progressively getting closer and closer to AGI. You can't blame us for extrapolating only a few years into the future and planning for it.
While true, that's the wrong lesson. The right lesson is GPT 1/2/3/4 are progressively getting closer and closer to AGI. You can't blame us for extrapolating only a few years into the future and planning for it.
That is quite a claim.
Would be cool if true. Do you have any references to back this up?
Why would scaling up turn a model that strings tokens together into something that can reason by itself, unprompted, with intent?
Why is this whole 'unprompted' even important? A huge portion of human intelligence comes from external prompting (just research how very screwed up infants of exceptional neglect are). A model, or a human is going to need input/output from the external world. We're just arguing the semantics of the IO device at this point.
> Do you have any references to back this up?
The reference is the GPT-4 release paper. It's getting progressively better at things that require intelligence, like writing code and doing unseen exams across various fields that humans find difficult. The generality and depth of intelligence is progressively improving.
> unprompted, with intent
Neither of these are necessary for AGI. Intelligence doesn't require intent. Intelligence doesn't require something to be "unprompted" (which human intelligence isn't, either, our brain is being constantly prompted by our external senses).
The reference is the GPT-4 release paper. It's getting progressively better at things that require intelligence, like writing code and doing unseen exams across various fields that humans find difficult. The generality and depth of intelligence is progressively improving.
> unprompted, with intent
Neither of these are necessary for AGI. Intelligence doesn't require intent. Intelligence doesn't require something to be "unprompted" (which human intelligence isn't, either, our brain is being constantly prompted by our external senses).
Is there any way we can prevent these models from being abused by large corporations for profit?
I was just thinking today, it’s interesting how I can ask ChatGPT something that is hard to get a straight answer for - such as “how much does X usually cost?”
I wouldn’t be surprised if this type of tech was quickly censored by a company that implements it at scale (say, Google or bing) for profit.
I feel like it’s inevitable, and it’s depressing.
Will the future be populated by LLMs that all want to control the information I have access to?
I was just thinking today, it’s interesting how I can ask ChatGPT something that is hard to get a straight answer for - such as “how much does X usually cost?”
I wouldn’t be surprised if this type of tech was quickly censored by a company that implements it at scale (say, Google or bing) for profit.
I feel like it’s inevitable, and it’s depressing.
Will the future be populated by LLMs that all want to control the information I have access to?
I want an open version of this that I can train on any data I choose, can give any prompt I want, and runs on my hardware.
However, I feel like the best and most useful versions of this type of software would (by necessity) only be able to run at a huge scale, cost millions to train, etc.
Now instead of having to worry about the bias of the sources I read, I have to question the bias of the language model based upon who created it.
However, I feel like the best and most useful versions of this type of software would (by necessity) only be able to run at a huge scale, cost millions to train, etc.
Now instead of having to worry about the bias of the sources I read, I have to question the bias of the language model based upon who created it.
How do we even define 'best and most useful' for a language model?
If you have your own data and problems you are trying to solve I don't think using GPT4 in zero-few shot mode (even with CoT) is necessarily better/higher performing than using a smaller fine-tuned or retrieval augmented model, it certainly wasn't true for GPT3.5 although we don't have as much insight into GPT4.
If we're talking about something as general/open-ended as ChatGPT or a 'poor mans AGI' well yeah, the data curation and preprocessing alone takes a small team to figure out let alone training it. How useful this type of model is when it's still not good enough (i.e. hallucination) to be used in most production cases despite the exorbitant cost in development remains to be seen.
If you have your own data and problems you are trying to solve I don't think using GPT4 in zero-few shot mode (even with CoT) is necessarily better/higher performing than using a smaller fine-tuned or retrieval augmented model, it certainly wasn't true for GPT3.5 although we don't have as much insight into GPT4.
If we're talking about something as general/open-ended as ChatGPT or a 'poor mans AGI' well yeah, the data curation and preprocessing alone takes a small team to figure out let alone training it. How useful this type of model is when it's still not good enough (i.e. hallucination) to be used in most production cases despite the exorbitant cost in development remains to be seen.
I would also like to avoid a future where we are slaves to corporate AI. If training the whole model individually isn’t feasible, would it be possible to decentralize the training?
Check out KoboldAI community, they're collaboratively finetuning and hosting GPT-J, NeoX, and now LLAMA models for free, as hobbies.
That’s an interesting idea. Something like Folding at home, or coin mining.
Sounds like the same problem we have with information sources in general. Whoever controls the source will bias what comes out to its advantage.
The good thing is that open source versions are popping up everywhere. This wont stop and everyone will be able to run their local AIs https://simonwillison.net/2023/Mar/11/llama/. Problem is the 'best' one will likely be centralized. We have open source everything; nowadays you can run on your own version of most software categories but users still choose to use proprietary versions bc the convenience outweighs their need for privacy/objectivity.
The good thing is that open source versions are popping up everywhere. This wont stop and everyone will be able to run their local AIs https://simonwillison.net/2023/Mar/11/llama/. Problem is the 'best' one will likely be centralized. We have open source everything; nowadays you can run on your own version of most software categories but users still choose to use proprietary versions bc the convenience outweighs their need for privacy/objectivity.
Are you implying that any company using AI for the purpose of increasing the profitability of the company is "abusing" AI? Or are you talking about something else? It kind of across to me like you thought the use of AI with profit as a motive is always abuse, not sure that I understood you correctly.
If it’s not disclosed then it’s abuse (in my eyes - not saying legally, just morally).
If you think it is always abuse if the use of AI is not explicitly disclosed, then you're going to have to recognize there are many thousands of situations where that view is absurd.
Imagine you and a group of friends decide to develop a game. You make a company and have a team of four people. Your budget is tiny and you can't afford the same resources a huge game company can. You want your new company to be profitable so you use large language models to generate realistic material textures for your 3D models, and to generate a diverse range of intetestimg character backstories and dialogs. This used to require a larger team of developers and content creators but now you can do more with fewer people and ship your game sooner. You make a great game and users love it and it's a financial success.
But oh wait, you didn't explicitly disclose that you use an AI powered tool to generate some art assets and character backstories. Therefore you are just a greedy company that is abusing AI for profit.
That's so ridiculous.
Imagine you and a group of friends decide to develop a game. You make a company and have a team of four people. Your budget is tiny and you can't afford the same resources a huge game company can. You want your new company to be profitable so you use large language models to generate realistic material textures for your 3D models, and to generate a diverse range of intetestimg character backstories and dialogs. This used to require a larger team of developers and content creators but now you can do more with fewer people and ship your game sooner. You make a great game and users love it and it's a financial success.
But oh wait, you didn't explicitly disclose that you use an AI powered tool to generate some art assets and character backstories. Therefore you are just a greedy company that is abusing AI for profit.
That's so ridiculous.
Do companies abuse linear regressions?
> Is there any way we can prevent these models from being abused by large corporations for profit?
Competition, from for-profits large and small, non-profits large and small, and the rest of the open community.
Don't trust what BigLLM™ is saying? Here's a dozen others you can ask...
Competition, from for-profits large and small, non-profits large and small, and the rest of the open community.
Don't trust what BigLLM™ is saying? Here's a dozen others you can ask...
competition is what got us the huge corps..?
maybe look for opportunities for cooperation instead.
maybe look for opportunities for cooperation instead.
> competition is what got us the huge corps..?
Yes, and consolidation also. Competition also gave us their disruptors.
> maybe look for opportunities for cooperation instead
I hear you, and I don't disagree with your line of thinking, but cooperation to compete is a thing, no?
My point being, an adversarial strategy prevents power from coalescing into the hands of one group, because it actively seeks to disrupt that process. The more diverse competition there is, the more pressure there will be against huge corps.
Yes, and consolidation also. Competition also gave us their disruptors.
> maybe look for opportunities for cooperation instead
I hear you, and I don't disagree with your line of thinking, but cooperation to compete is a thing, no?
My point being, an adversarial strategy prevents power from coalescing into the hands of one group, because it actively seeks to disrupt that process. The more diverse competition there is, the more pressure there will be against huge corps.
Open source competes through cooperation.
could that not simply be stated "open source cooperates"
i find it a little dated that we cognitively frame everything through the myopic lens of competition.
and it's a perfect example where a systemic restructuring can fundamentally rebalance control.
i find it a little dated that we cognitively frame everything through the myopic lens of competition.
and it's a perfect example where a systemic restructuring can fundamentally rebalance control.
Why is it myopic? OSS products and services absolutely do compete against other commercial services, but both improve in the process. It is the same as how biological evolution works. Competition is a good thing.
you are assuming competition as the dominant mode a-priori. i'm suggesting it's not the case. and the congnitive re-framing of these interactions can have a net-positive effect on social relations.
I'm really not sure what you're saying. All I'm saying is that OSS also has to compete against commercial software.
open source doesn't have to compete.
Of course not, sure. But it depends on what you're targeting. If you want to dump code into a git repo, slap MIT on the license and never touch it again, you can, I've done so many times. But if you want people to use it, or if you want to earn money from it (Docker, GitLab, Plausible etc) then you will have to compete.
> Is there any way we can prevent these models from being abused by large corporations for profit?
No. I mean what, are you gonna get them to halt development? I'd also say heavy-handed regulation is unlikely because the US Government views itself as in a technological competition with China, it won't want to slow things down.
> Will the future be populated by LLMs that all want to control the information I have access to?
Nothing's stopping the open source community, or you yourself, from making your own. You have a computer connected to the internet after all. But it will require lots of technical expertise, data, and money.
No. I mean what, are you gonna get them to halt development? I'd also say heavy-handed regulation is unlikely because the US Government views itself as in a technological competition with China, it won't want to slow things down.
> Will the future be populated by LLMs that all want to control the information I have access to?
Nothing's stopping the open source community, or you yourself, from making your own. You have a computer connected to the internet after all. But it will require lots of technical expertise, data, and money.
I suppose regulation is part of what I had in mind, but it would be really tough to prove anything in court (I feel) so regulation might be impossible.
Maybe we require transparency in how the model is trained, what the prompt is, etc. It’s fine if a corporation wants to build something that increases their profits, and they can invest into training the model and use it as a competitive advantage.
But I feel there is some level of disclosure owed to the consumer. Not sure what it would be.
Maybe we require transparency in how the model is trained, what the prompt is, etc. It’s fine if a corporation wants to build something that increases their profits, and they can invest into training the model and use it as a competitive advantage.
But I feel there is some level of disclosure owed to the consumer. Not sure what it would be.
Big companies will have better technology for a long while because of all the money they can throw at it, but there's no reason to expect an ecosystem of useful open/instanced AI systems to mature along side it. We've seen that start with Stable Diffusion and now also with LLaMa, and we're only at the beginning of the road.
Yes, big companies that spend lots of money and are big targets for lawsuits and regulation will exercise a lot of control over what their state-of-the-art products offer. There's no getting around that.
But that's not all we have to look forward to by any means.
Yes, big companies that spend lots of money and are big targets for lawsuits and regulation will exercise a lot of control over what their state-of-the-art products offer. There's no getting around that.
But that's not all we have to look forward to by any means.
we need a tech consumer+employee co-op.
fedco is a seed company structured similarly. it's owned 60% by consumers and 40% by employees. this puts democratic control and profit distribution front and center.
> We are a cooperative, one of the few seed companies so organized in the United States. Because we do not have an individual owner or beneficiary, profit is not our primary goal. Consumers own 60% of the cooperative and worker members 40%. Consumer and worker members share proportionately in the cooperative’s profits through our annual patronage dividends.
https://www.fedcoseeds.com/about_fedco.htm
fedco is a seed company structured similarly. it's owned 60% by consumers and 40% by employees. this puts democratic control and profit distribution front and center.
> We are a cooperative, one of the few seed companies so organized in the United States. Because we do not have an individual owner or beneficiary, profit is not our primary goal. Consumers own 60% of the cooperative and worker members 40%. Consumer and worker members share proportionately in the cooperative’s profits through our annual patronage dividends.
https://www.fedcoseeds.com/about_fedco.htm
[deleted]
I cant imagine what will happen when they add video. Right now its extremely expensive, but its definitely coming. All of youtube, all of the text on the internet, all pictures, consumed by some huge AI model. Who knows what it could learn from the movements, speech, etc in video
it will have the ability to create infinite irresistible clickbait and become the most subscribed youtube channel of all time
That will require it to be trained every day on new human content generated content so that it can create variations of it. Even then the fad would not last as long as it does today, since it would become possible for a larger section of the population to generate videos of the same kind.
Haven’t seen this TED talk in some time but I seem to remember it saying very creepy children oriented YouTube videos were generated based off of keywords. Maybe ai videos will just be like a trillion monkeys making videos. Some are bound to be awful, some are great, most are middling.
Edit: obviously I should have just asked bing for a summary of the video. Below,
Show HN: The nightmare videos of children’s YouTube and what’s wrong with the internet today This is a TED talk by James Bridle, a writer and artist who exposes a dark, strange corner of the internet, where unknown people or groups on YouTube hack the brains of young children with disturbing and inappropriate content. He shows how these videos exploit YouTube’s algorithms and advertising system to generate views and revenue, and how they reveal a deeper problem with the internet today: a lack of human oversight, accountability and transparency. He calls for more attention and ethical responsibility from tech companies and users. Link: https://www.ted.com/talks/james_bridle_the_nightmare_videos_...
Edit: obviously I should have just asked bing for a summary of the video. Below,
Show HN: The nightmare videos of children’s YouTube and what’s wrong with the internet today This is a TED talk by James Bridle, a writer and artist who exposes a dark, strange corner of the internet, where unknown people or groups on YouTube hack the brains of young children with disturbing and inappropriate content. He shows how these videos exploit YouTube’s algorithms and advertising system to generate views and revenue, and how they reveal a deeper problem with the internet today: a lack of human oversight, accountability and transparency. He calls for more attention and ethical responsibility from tech companies and users. Link: https://www.ted.com/talks/james_bridle_the_nightmare_videos_...
I wonder if this is simply how brains evolved too, because we also are multi-modal, and parts of our brain are specifically used for different tasks such that damage in those parts impedes completing said task (while still having some level of neuroplasticity where the brain could try to recover by remolding its neural architecture).
I would not be surprised to see something similar when looking inside the structure of a multi-modal neural network, where certain parts handle certain tasks and other parts handle other tasks.
I would not be surprised to see something similar when looking inside the structure of a multi-modal neural network, where certain parts handle certain tasks and other parts handle other tasks.
It seems pretty obvious to me that you chain these guys together. Personality module, planning module, decision making module, internal monologue module.. boatloads of them all blabbering to each other. How you connect them to each other is an architectural question. My easy experiments show that it doesn’t seem so hard, but what do I know.
Sure these guys don’t have memory, but I have one “storage” LLM that accepts vague requests and turns them into SQL, another cleans up the SQL so on and so forth, you need a handful of them each step. Now you have a database. Etc.
Does anybody does this?
Sure these guys don’t have memory, but I have one “storage” LLM that accepts vague requests and turns them into SQL, another cleans up the SQL so on and so forth, you need a handful of them each step. Now you have a database. Etc.
Does anybody does this?
Here's my first thought when I saw the visual input mode:
Can I stream it screenshots, give it a goal, and ask it for click coordinates back? Can GPT-4 successfully use a PC desktop?
And my next thought:
Better try this with a VM without internet access ...
Can I stream it screenshots, give it a goal, and ask it for click coordinates back? Can GPT-4 successfully use a PC desktop?
And my next thought:
Better try this with a VM without internet access ...
That is PaLM-E model. Or "embodied" LLM. They literally did exactly what you said. An LLM getting input from a camera and output command to a robotic hand. Then you can tell the bot to pick up a knife and "deposit" it somewhere else...
Well, no theyjust made the bot picked up a pen and bring it to a person at a table 15ft away. It did that, despite "obstacles" by another human keep knocking the pen off its hand. But it shows that an image LLM can perform goals.
Well, no theyjust made the bot picked up a pen and bring it to a person at a table 15ft away. It did that, despite "obstacles" by another human keep knocking the pen off its hand. But it shows that an image LLM can perform goals.
RuneScape mining bot coming right up!
How do they even train such huge models? Especially when you start talking abou multi-modal models. Is the underlying optimization process still relying on gradient descent? I am just baffled how they manage to get it to train across such vastly different sets of data and manage to get it to work.
Whilst not the same I recommend you look at the DeepMind Gato paper to see surprisingly (relatively) simple multi modal can be - https://openreview.net/forum?id=1ikK0kHjvj
Essentially to merge lots of modalities they just go 'let's convert all modalities into integers in the same given range', e.g the word 'me' = 1001, up in Atari = 11002, joint torque of right motor of robot = 33000 and so on.
From the paper:
There are infinite possible ways to transform data into tokens, including directly using the raw underlying byte stream. Below we report the tokenization scheme we found to produce the best results for Gato at the current scale using contemporary hardware and model architectures.
• Text is encoded via SentencePiece (Kudo & Richardson, 2018) with 32000 subwords into the integer range [0, 32000).
• Images are first transformed into sequences of non-overlapping 16 × 16 patches in raster order, as done in ViT (Dosovitskiy et al., 2020). Each pixel in the image patches is then normalized between [−1, 1] and divided by the square-root of the patch size (i.e. The tokenized result is a sequence of integers within the range of [0, 1024). 16 = 4).
• Discrete values, e.g. Atari button presses, are flattened into sequences of integers in row-major order.
• Continuous values, e.g. proprioceptive inputs or joint torques, are first flattened into sequences of floating point values in row-major order. The values are mu-law encoded to the range [−1, 1] if not already there (see Figure 14 for details), then discretized to 1024 uniform bins. The discrete integers are then shifted to the range of [32000, 33024).
Essentially to merge lots of modalities they just go 'let's convert all modalities into integers in the same given range', e.g the word 'me' = 1001, up in Atari = 11002, joint torque of right motor of robot = 33000 and so on.
From the paper:
There are infinite possible ways to transform data into tokens, including directly using the raw underlying byte stream. Below we report the tokenization scheme we found to produce the best results for Gato at the current scale using contemporary hardware and model architectures.
• Text is encoded via SentencePiece (Kudo & Richardson, 2018) with 32000 subwords into the integer range [0, 32000).
• Images are first transformed into sequences of non-overlapping 16 × 16 patches in raster order, as done in ViT (Dosovitskiy et al., 2020). Each pixel in the image patches is then normalized between [−1, 1] and divided by the square-root of the patch size (i.e. The tokenized result is a sequence of integers within the range of [0, 1024). 16 = 4).
• Discrete values, e.g. Atari button presses, are flattened into sequences of integers in row-major order.
• Continuous values, e.g. proprioceptive inputs or joint torques, are first flattened into sequences of floating point values in row-major order. The values are mu-law encoded to the range [−1, 1] if not already there (see Figure 14 for details), then discretized to 1024 uniform bins. The discrete integers are then shifted to the range of [32000, 33024).
The interesting thing to me is that our brains probably do something similar, converting multi-modal sensory data into the same 'model' that we experience as our concsiousness.
Usually you stick a CNN (or whatever other vision encoding neural network) after the image inputs and pipe the output of the CNN as an additional token or tokens of the GPT transformer (into the layer after the text embedding layer). That way everything is differentiable. You can pre-train the CNN and Transformer independently and then train them together (usually easier to train this way), or just train them from scratch. There are lots of other ways to combine two networks also.
So input is now one (or more) images and text, and output is text. There are ways to position the input image in a particular location within the input text as well. Training data can come from websites, etc.
So input is now one (or more) images and text, and output is text. There are ways to position the input image in a particular location within the input text as well. Training data can come from websites, etc.
More likely, this round of innovation is focused on integrating many engines into one product using techniques along the lines of langchain.
Instead of grinding increasingly complicated models that do everything, you train the LLM input/output engine to delegate to other systems and synthesize the results. There’s a ton of headroom down this road now that the natural language interface of the LLM has become so capable.
There are still surely active research tracks on integrating more data into more sophisticated models, but it looks like we’re at a maturity point where product engineering can start driving its own innovations.
Instead of grinding increasingly complicated models that do everything, you train the LLM input/output engine to delegate to other systems and synthesize the results. There’s a ton of headroom down this road now that the natural language interface of the LLM has become so capable.
There are still surely active research tracks on integrating more data into more sophisticated models, but it looks like we’re at a maturity point where product engineering can start driving its own innovations.
[deleted]
> In the lead up to GPT-4, Sam Altman hinted at higher pricing for smarter models
I think this is all you really need to know about the future of AGI.
I think this is all you really need to know about the future of AGI.
Each time a new innovation takes place there will be a website talking nonsense about AGI that will get millions of ad print revenue.
Don't click.
Don't click.
GPT4 is already out? I just got GTP3. "You bought the wrong bot, silly head!":
https://www.youtube.com/watch?v=EdRMVhlNP5I
how hard is it to "update" an LLM like GPT-4 with recent data instead of being frozen in time at the training date? Obviously you could use the increased context size to work around this, but being able to augment the base model seems like it would be the ideal use case for many high value projects like a version of Copilot that knows a company's entire code base and knows about newer libraries and updates
GPT has finetuning apis which you can use to update, but i think the massive context size is meant to show us you wont really need it most of the time. 25k words is really a lot of context. in the demo @gdb just dumped in the entire discord docs without breaking a sweat
Paper:https://cdn.openai.com/papers/gpt-4.pdf
Quote: "To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness."
Edit: Seems the simulation part was important for context: https://twitter.com/EigenGender/status/1635823028100829184