Training for one trillion parameter model backed by Intel and US govt has begun(techradar.com)
techradar.com
Training for one trillion parameter model backed by Intel and US govt has begun
https://www.techradar.com/pro/the-gpt-to-rule-them-all-training-for-one-trillion-parameter-model-backed-by-intel-and-us-government-has-just-begun
266 comments
It's not clear from the article whether it's a dense model or MoE. This matters when it comes to comparing with GPT-4 - in terms of # params - which is reported to be MoE
As far as I know, EVERY 1t+ LLM is a MoE. Switch-c-2048, Wi Dao 2.0, GLAM, Pangu-Σ, presumably GPT4. Am I missing any?
What is MoE?
Edit: Ah, Mixture of Experts. I hadn't heard this one yet. Thanks!
Edit: Ah, Mixture of Experts. I hadn't heard this one yet. Thanks!
A lot of this was new to me, but it looks like Intel hopes to use this to demonstrate the linear scaling capacity of their Aurora nodes.
Argonne installs final components of Aurora supercomputer (22 June 2023): https://www.anl.gov/article/argonne-installs-final-component...
Aurora Supercomputer Blade Installation Complete (22 June 2023): https://www.intel.com/content/www/us/en/newsroom/news/aurora...
Intel® Data Center GPU Max Series, previously codename Ponte Vecchio (31 May 2023): https://www.intel.com/content/www/us/en/developer/articles/t...
Argonne installs final components of Aurora supercomputer (22 June 2023): https://www.anl.gov/article/argonne-installs-final-component...
Aurora Supercomputer Blade Installation Complete (22 June 2023): https://www.intel.com/content/www/us/en/newsroom/news/aurora...
Intel® Data Center GPU Max Series, previously codename Ponte Vecchio (31 May 2023): https://www.intel.com/content/www/us/en/developer/articles/t...
yes, I can't imagine the architecture of a supercomputer is the right one for LLM training. But maybe?
If not, spending years to design and build a system for weather and nuke simulations and ending up doing something that's totally not made for these systems is kind of a mind bender.
I can imagine the conversations that led to this: "we need a government owned LLM" "okay what do we need" "lots of compute power" "well we have this new supercomputer just coming online" "not the right kind of compute" "come on, it's top-500!"
If not, spending years to design and build a system for weather and nuke simulations and ending up doing something that's totally not made for these systems is kind of a mind bender.
I can imagine the conversations that led to this: "we need a government owned LLM" "okay what do we need" "lots of compute power" "well we have this new supercomputer just coming online" "not the right kind of compute" "come on, it's top-500!"
How are the optimal architectures for weather simulations and LLM training different?
tbh, I'm not entirely informed on what the requirements are for LLM training, but I've noticed that nearly all of the teams don't use unified memory (what makes a super computer "super"). I believe I read somewhere openAI uses a K8s cluster[1] and other teams I know seem to work with other similar non-unified memory systems. If there's no advantage, part of what makes supercomputers expensive is this memory interconnect, so wouldn't it just be better to use a huge K8s cluster or something?
I'm honestly not sure, and hoping somebody comments here and provides more information as I'm genuinely interested.
1 - https://openai.com/research/scaling-kubernetes-to-7500-nodes
OpenAI says their biggest jobs run on MPI so maybe a supercomputer would be better?
I'm honestly not sure, and hoping somebody comments here and provides more information as I'm genuinely interested.
1 - https://openai.com/research/scaling-kubernetes-to-7500-nodes
OpenAI says their biggest jobs run on MPI so maybe a supercomputer would be better?
What is this “unified memory” you speak of?
Modern supercomputers are just a bunch of computers with a (very) high speed interconnect. As far as I’m aware, pretty much every multi node job on a supercomputer uses MPI to communicate, and there isn’t any sort of “unified memory” system.
It has been a few years since I worked on one of the big supercomputers though, so maybe you have better knowledge than me. I’d love more information.
Modern supercomputers are just a bunch of computers with a (very) high speed interconnect. As far as I’m aware, pretty much every multi node job on a supercomputer uses MPI to communicate, and there isn’t any sort of “unified memory” system.
It has been a few years since I worked on one of the big supercomputers though, so maybe you have better knowledge than me. I’d love more information.
Probably optimized for fp64 instead of 4 bit floats.
Could the weights be FOIA'd?
They are pretty much guaranteed to slap the "national security" label on it, so no.
Hopefully not.
Why not? Would be cool with some new open source models.
I agree but I don't think our adversaries should get a freebie that's trained on our scientific data at a national laboratory.
Which makes me wonder. I'm not sure this applies here but say you train a model on classified information, is the model/weights then classified?
Which makes me wonder. I'm not sure this applies here but say you train a model on classified information, is the model/weights then classified?
If a model is trained on illegally obtained copyrighted material, are the images it produces stolen?
Isn't GPT-4 already over 1T parameters? And GPT-5 should be even "an order of magnitude" bigger than GPT-4...
Was it ever confirmed whether GPT-4 is a Mixture of Experts or not?
I just asked GPT-4, and it denies being a MoE model.
User:
Are you an MoE model?
ChatGPT:
No, I am not based on a Mixture of Experts (MoE) model. My underlying architecture is based on the GPT (Generative Pre-trained Transformer) framework, specifically the GPT-4 version. This architecture is a large-scale transformer-based neural network, but it does not use the MoE approach.
In a GPT model like mine, the entire model works in a unified manner on each task, without dividing it among specialized sub-models or experts. The model is trained on a diverse range of data and tasks, enabling it to generate responses across a wide array of topics and questions. While this approach lacks the specialization that MoE models offer, it provides broad, general-purpose language understanding and generation capabilities.
User:
Are you an MoE model?
ChatGPT:
No, I am not based on a Mixture of Experts (MoE) model. My underlying architecture is based on the GPT (Generative Pre-trained Transformer) framework, specifically the GPT-4 version. This architecture is a large-scale transformer-based neural network, but it does not use the MoE approach.
In a GPT model like mine, the entire model works in a unified manner on each task, without dividing it among specialized sub-models or experts. The model is trained on a diverse range of data and tasks, enabling it to generate responses across a wide array of topics and questions. While this approach lacks the specialization that MoE models offer, it provides broad, general-purpose language understanding and generation capabilities.
LLMs don’t have insight, these outputs cannot be assumed to be accurate.
Due to current hardware limitations it is not feasible to have a 1T parameter model without MoE.
Due to current hardware limitations it is not feasible to have a 1T parameter model without MoE.
I wonder if it has some canned human-written responses when asking specific questions about itself. This would be pretty clever to silently implement, that will definitely help convince people that it's approaching "AGI".
It's possible that it's just hallucinating here too, I don't have any proof that the responses are canned, but they appear that way to me.
It's possible that it's just hallucinating here too, I don't have any proof that the responses are canned, but they appear that way to me.
Yes, it has an invisible "shadow prompt" that gets sent to it when you start a session. It looks something like this:
https://www.reddit.com/r/ChatGPT/comments/zo9of4/comment/j0n...
https://www.reddit.com/r/ChatGPT/comments/zo9of4/comment/j0n...
That makes sense, there has to be some level of human written responses for specific things, like when you ask it about a controversial topic that it refuses to answer, it gives you some canned response about how it won't answer that, and that very obviously isn't what language models do naturally. For these cases there has to be some sort of pre-programmed human generated reply!
It's interesting that other people don't seem to agree with the idea that it has some pre-programmed responses, I am curious as to what they think is going on here.
It's interesting that other people don't seem to agree with the idea that it has some pre-programmed responses, I am curious as to what they think is going on here.
That is almost surely a hallucination.
Curious, why do people gravitate to the term "hallucination" (as if it's a person not a tool) instead of just plainly stating that it's wrong?
If I ask it what color an orange it and it says blue, that would be wrong.
If you ask it a question and it makes up a completely fabricated story, like for example the case files in that recent legal case [1], then saying it was “wrong” doesn’t really seem to capture it.
Calling it a hallucination is a great analogy, because the model made up a plausible sounding, but completely fabricated story. It saw things that were never real.
[1] https://apnews.com/article/artificial-intelligence-chatgpt-c...
If you ask it a question and it makes up a completely fabricated story, like for example the case files in that recent legal case [1], then saying it was “wrong” doesn’t really seem to capture it.
Calling it a hallucination is a great analogy, because the model made up a plausible sounding, but completely fabricated story. It saw things that were never real.
[1] https://apnews.com/article/artificial-intelligence-chatgpt-c...
Fair enough and thanks. I felt hallucination was too forgiving a term but I can see how others would rank them the other way around and suppose it works.
The more accurate term I’ve seen floating around is “confabulation”. It’s also a human phenomenon, but it doesn’t bring all the baggage of hallucinations.
It’s a chat bot so a lot of language will be anthropomorphic. Even still this definition fits well “3. a false belief or impression; illusion; delusion.”
It does not and it cannot answer this kind of meta-question unless explicitly added to its prompt/RAG or whatever.
And OpenAI would not add such proprietary information there.
And OpenAI would not add such proprietary information there.
Ask it to produce a citation for that claim so we can verify it.
Can you explain how your own mind works?
Yes, the article even mentions that. Guess they went with that headline to attract readers.
Was thinking the same, so 1T might bring you to the league of GPT-4.
Actually in the best case, since it seems that meta, google, openai and so on have the most talent.
Anyway, to bring it to the next level, how big should it be? Maybe 10T? 100T?
Anyway, to bring it to the next level, how big should it be? Maybe 10T? 100T?
> Maybe 10T? 100T?
I don't think we have enough training data to train models so big in a way to efficiently use all the params. We would need to generate training data, but then I don't know how effective it would be.
I don't think we have enough training data to train models so big in a way to efficiently use all the params. We would need to generate training data, but then I don't know how effective it would be.
Have seen an interview with someone from openai. At least they claimed, that they are far away from running out of data and they are able to generate data if needed.
The solution won't be just "bigger". A model with a trillion parameters will be more expensive to train and to run, but is unlikely to be better. Think of the early days of flight, you had biplanes; then you had triplanes. You could have followed that farther, and added more wings - but it wouldn't have improved things.
Improving AI will involve architectural changes. No human requires the amount of training data we are already giving the models. Improvements will make more efficient use of that data, and (no idea how - innovation required) allow them to generalize and reason from that data.
Improving AI will involve architectural changes. No human requires the amount of training data we are already giving the models. Improvements will make more efficient use of that data, and (no idea how - innovation required) allow them to generalize and reason from that data.
> "No human requires the amount of training data we are already giving the models."
Well, humans are also trained differently. We interact with other humans in real time and get immediate feedback on their responses. We don't just learn by reading through reams of static information. We talk to people. We get into arguments. And so on.
Maybe the ideal way to train an AI is to have it interact with lots of humans, so it can try things on its own? The implication of that is maybe the best trained AI will be the center of some important web property, like, say, the Google search engine (I'm imagining something like Google search now, but more conversational -- it asks you if that was what you're looking for, and asks clarifying questions.) Whoever has the most users will have the best AI, which creates the best product, which attracts the most users... and so on.
I do agree that architectural improvements could be hugely significant too.
Well, humans are also trained differently. We interact with other humans in real time and get immediate feedback on their responses. We don't just learn by reading through reams of static information. We talk to people. We get into arguments. And so on.
Maybe the ideal way to train an AI is to have it interact with lots of humans, so it can try things on its own? The implication of that is maybe the best trained AI will be the center of some important web property, like, say, the Google search engine (I'm imagining something like Google search now, but more conversational -- it asks you if that was what you're looking for, and asks clarifying questions.) Whoever has the most users will have the best AI, which creates the best product, which attracts the most users... and so on.
I do agree that architectural improvements could be hugely significant too.
Yeah I'm not totally convinced humans don't have a tremendous amount of training data - interacting with the world for years with constant input from all our senses and parental corrections. I bet if you add up that data it's a lot.
But once we are partially trained, training more requires a lot less.
But once we are partially trained, training more requires a lot less.
> I bet if you add up that data it's a lot.
Let's Fermi estimate that.
A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25.
Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.
There's plenty of reason to think that we don't learn effectively from all of this data (there's lots of redundancy, for example), but at the grossest orders of magnitude you seem to be right that at least in theory we have access to a tremendous amount of data.
Let's Fermi estimate that.
A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25.
Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.
There's plenty of reason to think that we don't learn effectively from all of this data (there's lots of redundancy, for example), but at the grossest orders of magnitude you seem to be right that at least in theory we have access to a tremendous amount of data.
> Let's Fermi estimate that.
I couldn't agree more. Fermi estimates are underused.
> Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.
And for comparison, GPT-4 is estimated by Bill Dally (NVIDIA) at around 10^12 or 10^13 tokens [1]. Let's assume about 1 word per token and 5 characters per word. Furthermore, US-ASCII tokens requires one byte per character in UTF-8. So, that gives about 50 terabyte [Edited after comment below].
As a side note, I would guess that GPT-4 knows more "things" if you would be able to count them all up. For example, it knows more languages, more facts about cities, more facts about persons. However, people know way more inside their specialization.
[1]: https://youtu.be/kLiwvnr4L80?si=77pXmIBmlp8dsSCG&t=349
I couldn't agree more. Fermi estimates are underused.
> Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.
And for comparison, GPT-4 is estimated by Bill Dally (NVIDIA) at around 10^12 or 10^13 tokens [1]. Let's assume about 1 word per token and 5 characters per word. Furthermore, US-ASCII tokens requires one byte per character in UTF-8. So, that gives about 50 terabyte [Edited after comment below].
As a side note, I would guess that GPT-4 knows more "things" if you would be able to count them all up. For example, it knows more languages, more facts about cities, more facts about persons. However, people know way more inside their specialization.
[1]: https://youtu.be/kLiwvnr4L80?si=77pXmIBmlp8dsSCG&t=349
> Let's assume about 1 word per token and 5 characters per word. Furthermore, US-ASCII tokens requires one byte per character in UTF-8. So, that gives about 40 petabyte.
Thanks, I was looking for that estimate but could not quickly find it. (Although are you sure about that 'petabyte' figure? I see 1e13 * 5 = 50e13 = 50 TB?)
I love these sorts of bulk comparisons, since they let us start to reason about how much humans learn from "each bit of experience" versus LLMs. On one hand humans process a lot of sensory information (most of it by discarding, mind), but on the other humans do a lot of 'active learning' whereby experience is directly related to actions. LLMs don't have the latter, since they're trained on passive feeds of tokens.
Thanks, I was looking for that estimate but could not quickly find it. (Although are you sure about that 'petabyte' figure? I see 1e13 * 5 = 50e13 = 50 TB?)
I love these sorts of bulk comparisons, since they let us start to reason about how much humans learn from "each bit of experience" versus LLMs. On one hand humans process a lot of sensory information (most of it by discarding, mind), but on the other humans do a lot of 'active learning' whereby experience is directly related to actions. LLMs don't have the latter, since they're trained on passive feeds of tokens.
This is completely and fundamentally an incorrect approach from start to finish. The human body - and the human mind - do have electrical and logic components but they are absolutely not digital logic. We do not see in “pixels”. The human mind is an analog process. Analog computing is insanely powerful and exponentially more efficient (time, energy, bandwidth) than digital computing but it is ridiculously hard to pack or compose, difficult to build with, limited in the ability to perform imperative logic with, etc.
You cannot compare what the human eyes/brain/body does with analog facilities to its digital “equivalent” and then “math it” from there.
Also why trying to replicate the human brain with digital logic (current AI approach) is so insanely expensive.
You cannot compare what the human eyes/brain/body does with analog facilities to its digital “equivalent” and then “math it” from there.
Also why trying to replicate the human brain with digital logic (current AI approach) is so insanely expensive.
You missed the point of the exercise. Of course it's extremely difficult to compare the two, but the question was: do humans get nearly as much training data as LLMs do? This analysis is good enough to say "actually humans receive much more raw input data than LLMs during their 'training period'." You're concerned with what the brain is doing with all that data compared to LLMs, but that's not the point of the exercise.
No, you can’t even compare that because the information isn’t packetized. Again, we don’t see in pixels so you can’t just consider sensory input in that manner.
What about using energy cost as a stand-in for the data?
The net energy cost of training and maintaining humans vs. training and maintaining (updating and augmenting) AI models.
The human body is incredibly efficient but there’s always a larger ramp-up cost.
The net energy cost of training and maintaining humans vs. training and maintaining (updating and augmenting) AI models.
The human body is incredibly efficient but there’s always a larger ramp-up cost.
Your estimate is using compressed video rates, which might be a argued way to look at the way that the brain processes and fills in presumably "already known" or inferred information. I don't know enough about this subject to make that argument.
Uncompressed rates are easy to calculate if wanting to adjust your approximation:
24-bit, 4K UHD @ 60 fps: 24 × 3840 × 2160 × 60 = 11.9 Gbit/s.
Uncompressed rates are easy to calculate if wanting to adjust your approximation:
24-bit, 4K UHD @ 60 fps: 24 × 3840 × 2160 × 60 = 11.9 Gbit/s.
Children are awake 12 hours per day, a 90 minute movie used to fit on a CD-ROM, 8 700MB CD-ROMs times 365 is 2TB per year. Alexander the Great became king aged 20, he seemed trained enough by then, so that's 400TB, 4e14 bytes of data.
Pretty unconvinced by this argument, it mostly hinges on the video stream size.
Pretty unconvinced by this argument, it mostly hinges on the video stream size.
I think it is an estimate. Yours is on the extreme low end of things. The GP was using 4K as a standin for touch, smell, taste, pain, hunger, pleasure, etc. Those are a bit more complex than just videos and you can't encode those on a CD.
Regardless, yours still came out to be an order of magnitude higher than what GPT was trained on. So I guess the original argument makes some sense.
Regardless, yours still came out to be an order of magnitude higher than what GPT was trained on. So I guess the original argument makes some sense.
So much of that data is totally useless though. Most of the data we get is visual and I would argue that visual data is some of the least useful data (with respect to volume). Think about the amount of time we're looking at blank colours with nothing to learn from. Once you've seen one wall, one chair, one table etc, there's not much left to know about those things. An encyclopedia though for example is much less data than a few hours of high res video of clouds passing by yet clearly its orders of magnitude information rich.
Also have to disagree here. You see that one object yes, but you see it from 10000000s of different angles and distances. In different settings. In different lighting conditions. With different arrangements. And you see the commonalities and differences. You poke it. prod it. Hit it. Break it. You listen to it.
This is the basis for 'common sense' and I’m pretty sure everything else needs that as the foundation.
Go watch a child learning and you'll see a hell of a lot of this going on. They want to play with the same thing over and over and over again. They want to watch the same movie over and over and over again. Or the same song over and over and over and over and over again.
This is the basis for 'common sense' and I’m pretty sure everything else needs that as the foundation.
Go watch a child learning and you'll see a hell of a lot of this going on. They want to play with the same thing over and over and over again. They want to watch the same movie over and over and over again. Or the same song over and over and over and over and over again.
This just sounds so hilariously wrong start to finish. That any of the data you collect is useless is already a big [citation needed]
Nit: The bandwidth of your optic nerve is only about 10 kilobits per second. You think you're seeing in 4K but most of it is synthetic.
Academic sources I’ve seen are in the 10Mbps range, e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1564115/ (estimating 875kbps for a guinea pig and ~10x that for a human).
That cannot be right.
10 kilobits per second would only get you some awful telephone quality audio. Even super low res video would be orders of magnitude more than that.
10 kilobits per second would only get you some awful telephone quality audio. Even super low res video would be orders of magnitude more than that.
Someone added it up:
https://osf.io/preprints/psyarxiv/qzbgx
> Large language models show intriguing emergent behaviors, yet they receive around 4-5 orders of magnitude more language data than human children.
https://osf.io/preprints/psyarxiv/qzbgx
> Large language models show intriguing emergent behaviors, yet they receive around 4-5 orders of magnitude more language data than human children.
_language_ data.
Humans get a lot more input that just language is the point. We get to see language out in the physical world. They have the (really quite difficult) task of inferring all the the real world and its connections, common sense, laws of physics, while locked in a box reading wikipedia.
How capable do you think a child would be if you kept them in a dark box and just spoke to them through the keyhole? (not recommending this experiment)
Humans get a lot more input that just language is the point. We get to see language out in the physical world. They have the (really quite difficult) task of inferring all the the real world and its connections, common sense, laws of physics, while locked in a box reading wikipedia.
How capable do you think a child would be if you kept them in a dark box and just spoke to them through the keyhole? (not recommending this experiment)
The first humans were kinda dumb. Yet by interacting with the world and each other, humans got smart. I wonder if neural network instances can interact with a simulated environment and each other, they could become smart enough for practical purposes.
Kinda like alpha go zero but with "life".
Kinda like alpha go zero but with "life".
Conversion is essential, but so is the interaction with the real world.
If a bunch of AIs started discussing with each other, and learning from each other, without validation, things can go wrong very easily.
I bet a lot of people thought that strapping a pair of feahterry wing was all that's needed to fly. This "knowledge" could've passed from one person to another through discussion. But it only took one (or more) person to actually try it in the real world to learn that, no, that doesn't work like that.
AI communities (as in communities of AIs talking to each other) might be one filled with the most outrageous "conspiracy theories", that may sound plausible but have no substance.
If a bunch of AIs started discussing with each other, and learning from each other, without validation, things can go wrong very easily.
I bet a lot of people thought that strapping a pair of feahterry wing was all that's needed to fly. This "knowledge" could've passed from one person to another through discussion. But it only took one (or more) person to actually try it in the real world to learn that, no, that doesn't work like that.
AI communities (as in communities of AIs talking to each other) might be one filled with the most outrageous "conspiracy theories", that may sound plausible but have no substance.
> The first humans were kinda dumb
How can you assert this ? Do you have any evidence?
How can you assert this ? Do you have any evidence?
Can't tell if you're joking or serious.
In the latter case, the first humans (H. Habilis) had about 1/2 of H. Sapiens brain to work with, and a much smaller fraction of neocortex.
If that doesn't satisfy you, let's say I was speaking about some sort of human ancestor before that, which would have been about as dumb as chimps, unless you require proof of their dumbness as well.
In the latter case, the first humans (H. Habilis) had about 1/2 of H. Sapiens brain to work with, and a much smaller fraction of neocortex.
If that doesn't satisfy you, let's say I was speaking about some sort of human ancestor before that, which would have been about as dumb as chimps, unless you require proof of their dumbness as well.
Not your original objector, but ... I was willing to accept that early humans are dumb, but your explanation relies on evolution and not social interaction.
We know their brains didn’t grow much after birth, unlike humans, which also suggests faster maturity akin to living apes and likely less social support for extended adolescence.
Humans didn't evolve knowledge, they evolved the capacity to get more knowledge.
All the new knowledge they got they figured out from the environment.
All the new knowledge they got they figured out from the environment.
Exactly. Humans didn't learn their way out of only having half a brain, more the opposite.
Hard to imagine anyone believes these prehuman theories. Just look at the h hablis wiki page to see 90% of it is pure speculation and debated.
Even tracing it back to the origin of Homo erectus, the line would still be too blurry. I don't think "the first humans" can actually mean anything in a science-based discussion.
It's not just humans we interact with - it's reality. For an LLM all "facts" are socially-determined, for a human they are not
Also, humans have multiple internal models for various aspects of reality, that seem to exist as a layer on top of the "raw" training. They have the ability to extrapolate based on those models (which transformers cannot do, as far I as understand). Perhaps GP is right -- perhaps what's missing in the architecture is an analog of these models.
While I agree with you that advances will come from being able to train with less data using as yet undevised techniques, I think you are jumping to a conclusion with this particular work.
First, yes, bigger appears to be better so far. We haven't yet found the plateau. No, bigger won't solve the well known problems, but it's absolutely clear that each time we build a bigger model it is qualitatively better.
Second, it's not clear that this work is trying to build AGI, which I assume you are referring to when you say "the solution." Of all the use case for language models, building one off all the worlds scientific data like they are doing in this project is probably the most exciting to me. If all it can do is dig up relevant work for a given topic in the entire body of scientific literature, it will be revolutionary for science.
First, yes, bigger appears to be better so far. We haven't yet found the plateau. No, bigger won't solve the well known problems, but it's absolutely clear that each time we build a bigger model it is qualitatively better.
Second, it's not clear that this work is trying to build AGI, which I assume you are referring to when you say "the solution." Of all the use case for language models, building one off all the worlds scientific data like they are doing in this project is probably the most exciting to me. If all it can do is dig up relevant work for a given topic in the entire body of scientific literature, it will be revolutionary for science.
> you had biplanes; then you had triplanes. You could have followed that farther, and added more wings - but it wouldn't have improved things.
But people did try! https://en.wikipedia.org/wiki/Besson_H-5 https://en.wikipedia.org/wiki/Multiplane_(aeronautics)#Quadr...
But people did try! https://en.wikipedia.org/wiki/Besson_H-5 https://en.wikipedia.org/wiki/Multiplane_(aeronautics)#Quadr...
Why stop here? This engineer experimented with 20 wings and more:
https://en.wikipedia.org/wiki/Horatio_Frederick_Phillips
https://en.wikipedia.org/wiki/Horatio_Frederick_Phillips
Should've kept going.
Karpathy in his recent video [1] agrees, but at this point scaling is a very reliable way to better accuracy.
[1]: https://youtu.be/zjkBMFhNj_g?si=eCH04466rmgBkHDA
[1]: https://youtu.be/zjkBMFhNj_g?si=eCH04466rmgBkHDA
Seems like he actually disagrees here:
If you train a bigger model on more text, we have a lot of confidence that the next-word prediction task will improve. So algorithmic progress is not necessary, it's a very nice bonus, but we can sort of get more powerful models for free, because we can just get a bigger computer, which we can say with some confidence we're going to get, and just train a bigger model for longer, and we are very confident we are going to get a better result.
https://youtu.be/zjkBMFhNj_g?t=1543 (23:43)
If you train a bigger model on more text, we have a lot of confidence that the next-word prediction task will improve. So algorithmic progress is not necessary, it's a very nice bonus, but we can sort of get more powerful models for free, because we can just get a bigger computer, which we can say with some confidence we're going to get, and just train a bigger model for longer, and we are very confident we are going to get a better result.
https://youtu.be/zjkBMFhNj_g?t=1543 (23:43)
And then at 35 minutes he spends a few minutes talking about ideas for algorithmic improvements.
This. We have not exhausted all the techniques at our disposal yet. We do need to look for a new architecture though, but these are orthogonal
OpenAIs big innovation was “bigger”. It is not clear when we should stop scaling.
[deleted]
I asked ChatGPT how many parameters the human brain has, and it said 86B neurons * 1000 connections, so 86T parameters.
It does seem like bigger models give better responses when given benchmarks. It might plateau or overfit the data at some point, but I'm not sure we've reached it yet.
It does seem like bigger models give better responses when given benchmarks. It might plateau or overfit the data at some point, but I'm not sure we've reached it yet.
Unlike biplanes, CPU's with more transistors are more powerful than those with less. And adding more CPU cores keeps increasing the amount of threads you can run at the same time.
Why would LMM's be more like the biplanes analogy, and less like the CPU analogy?
Why would LMM's be more like the biplanes analogy, and less like the CPU analogy?
In general you can view "understanding" as a compression of information. You take in a bunch of information, detect an underlying pattern, and remember the pattern and necessary context, instead of the entire input.
The "problem" with larger neural networks is that they can store more information, so they can substitute understanding with memorization. Something similar happens with human students, who can stuff lots of raw information into short-term-memory, but to fit it into the much more precious long-term-memory you have to "understand" the topic, not just memorize it. In neural networks we call that memorization a failure to generalize. Just like a human, a network that just memorizes doesn't do well if you ask it about anything slightly different than the training data.
Of course it's a balance act, because a network that's too small doesn't have space to store enough "understanding" and world model. A lot of the original premise of OpenAI was to figure out if LLMs keep getting better if you make them bigger, and so far that has worked. But there is bound to be a ceiling on this, where making the model bigger starts making it dumber.
The "problem" with larger neural networks is that they can store more information, so they can substitute understanding with memorization. Something similar happens with human students, who can stuff lots of raw information into short-term-memory, but to fit it into the much more precious long-term-memory you have to "understand" the topic, not just memorize it. In neural networks we call that memorization a failure to generalize. Just like a human, a network that just memorizes doesn't do well if you ask it about anything slightly different than the training data.
Of course it's a balance act, because a network that's too small doesn't have space to store enough "understanding" and world model. A lot of the original premise of OpenAI was to figure out if LLMs keep getting better if you make them bigger, and so far that has worked. But there is bound to be a ceiling on this, where making the model bigger starts making it dumber.
[deleted]
No one expected larger LLMs to be amazing, so although it's unlikely that these larger models will do anything, it was also unlikely that we are in our current situation regarding LLMs.
Really early days of flying where like "let's add a few feathers and then add some more". Though architectural changes were added too.
What? The defining trend of the last 5 or so years is the victory of the scaling hypothesis. More scale = more intelligence. GPT-4 is way smarter than 3.5, this trend is ongoing.
You need more data to utilize more parameters, but folks at the forefront are confident that they are not going to run out of data any time soon.
If you mean “solution to AGI” maybe. But perhaps in-context scratchpads and agent loops will be sufficient to get this architecture to human-level performance, with enough data/parameters. (Sutskever and Hinton have both expressed credulity that the current architectures might get us there.)
All that said, it’s also possible that new architectures will be needed at some point, I’m just pushing back on your claim that we already hit the cliff.
You need more data to utilize more parameters, but folks at the forefront are confident that they are not going to run out of data any time soon.
If you mean “solution to AGI” maybe. But perhaps in-context scratchpads and agent loops will be sufficient to get this architecture to human-level performance, with enough data/parameters. (Sutskever and Hinton have both expressed credulity that the current architectures might get us there.)
All that said, it’s also possible that new architectures will be needed at some point, I’m just pushing back on your claim that we already hit the cliff.
The main hero here is not model size but the dataset and the environment that created it. All this model talk missed the point - without the repository of human experience captured in language these models would not get so smart. And the improvement path is the same - assimilate more experience. This time the AI agent can create its own interactions and feedback signals, this would help it fix its flaws.
Learning in third person from the past data can only take AI so far. It needs to act and learn in the present, in first person, to be truly smart. No architectural change is needed, but the model needs to be placed in a real environment to get feedback signals.
Learning in third person from the past data can only take AI so far. It needs to act and learn in the present, in first person, to be truly smart. No architectural change is needed, but the model needs to be placed in a real environment to get feedback signals.
[deleted]
Is anything known about what extent if any non-public domain books are used for LLM’s?
One example is the Google books project made digital quite a few texts, but I’ve never heard if Google considers these fair game to train on for Bard.
Most of the copyright discussions I’ve seen have been around images and code but not much about books.
Seems to become more relevant as things scale up as indicated by this article.
One example is the Google books project made digital quite a few texts, but I’ve never heard if Google considers these fair game to train on for Bard.
Most of the copyright discussions I’ve seen have been around images and code but not much about books.
Seems to become more relevant as things scale up as indicated by this article.
>we found 72,508 ebook titles (including 83 from Stanford University Press) that were pirated and then widely used to train LLMs despite the protections of copyright law
https://aicopyright.substack.com/p/the-books-used-to-train-l...
https://aicopyright.substack.com/p/the-books-used-to-train-l...
It will be interesting to see what the government can do here. Can they use their powers to get their hands on the most data?
im still skeptical because new techniques are going to give an order of magnitude efficiency boost to transformer models, so 'just waiting' seems like the best approach for now. I dont think they will be able to just skip to the finish line by having the most money.
im still skeptical because new techniques are going to give an order of magnitude efficiency boost to transformer models, so 'just waiting' seems like the best approach for now. I dont think they will be able to just skip to the finish line by having the most money.
I just realized that the NSA has probably been able to train GPT-4 equivalents on _all the data_ for a while now. We'll probably never learn about it but that's maybe scarier than just the Snowden collection story because LLMs are so good at retrieval.
Holy shit, you are right. They probably have 10-100x the data used to train gpt-4. Decades of every text message, phone call transcript, and so on. I can’t believe I haven’t seen anyone mention that yet.
People keep saying we don’t have enough data. I think there is a lot more data than we realize, even ignoring things like NSA.
People keep saying we don’t have enough data. I think there is a lot more data than we realize, even ignoring things like NSA.
Apparently there are roughly 2 trillion text messages sent per year in the US [1]. I did a sanity check, that’s like 40 or so a day per person, so sounds reasonable.
I couldn’t find the average message length, but I would guess it’s fairly short (with a fat tail of longer messages).
To make the math easy, let’s say the average length is ~10 tokens. I’d be surprised if that isn’t correct within a factor of 2 or so.
So we have 20 trillion tokens per year from text messages in the US alone. And this is high-quality conversational data.
The annual numbers were fairly constant in recent years (and then it drops off), so the past decade of US text messages is about 200 trillion tokens! That’s a metric fuck ton… Much larger than any dataset existing models have been trained on, I believe.
I would guess phone transcripts would be an order of magnitude larger at least. Talking is a lot easier than typing on a phone.
You could train an absolutely insane model with that amount of data… Damn.
[1] https://www.statista.com/statistics/185879/number-of-text-me...
I couldn’t find the average message length, but I would guess it’s fairly short (with a fat tail of longer messages).
To make the math easy, let’s say the average length is ~10 tokens. I’d be surprised if that isn’t correct within a factor of 2 or so.
So we have 20 trillion tokens per year from text messages in the US alone. And this is high-quality conversational data.
The annual numbers were fairly constant in recent years (and then it drops off), so the past decade of US text messages is about 200 trillion tokens! That’s a metric fuck ton… Much larger than any dataset existing models have been trained on, I believe.
I would guess phone transcripts would be an order of magnitude larger at least. Talking is a lot easier than typing on a phone.
You could train an absolutely insane model with that amount of data… Damn.
[1] https://www.statista.com/statistics/185879/number-of-text-me...
Interestingly, the reason Google initially created it's Google Voice service back in the day was to gather voicemail audio to train its speech to text engines.
It's mind-blowing to me that with all of Google's data, Google isn't the far and away leader in this new space. I have to believe they're paralyzed by the fear of legal repercussions.
It's mind-blowing to me that with all of Google's data, Google isn't the far and away leader in this new space. I have to believe they're paralyzed by the fear of legal repercussions.
When chatGPT came out I asked it "which living politician is an unknown spy" and the question got blocked
[deleted]
Haha, this is funny because everyone is talking about this as if it is designed to be like the LLMs we have access to.
The training parameters will be the databases of info scooped up and integrated into profiles of every person and their entire digital footprint, queriable and responsive to direct questioning
The training parameters will be the databases of info scooped up and integrated into profiles of every person and their entire digital footprint, queriable and responsive to direct questioning
Ah yeah, this sounds like such a great thing, state of the art unreleased tech + 1 trillion parameters based by data accessed by the patriot act.
Such a wholesome thing. I don't want to hear 2 years from now how China is evil for using "AI" when the government is attempting to weaponize AI, of course other governments will start doing it as well.
Such a wholesome thing. I don't want to hear 2 years from now how China is evil for using "AI" when the government is attempting to weaponize AI, of course other governments will start doing it as well.
[deleted]
Mistral 7B parameter models are quite good
Already fine tuned and conversational
its like education is more important than needing a trillion parameter brainiac
Already fine tuned and conversational
its like education is more important than needing a trillion parameter brainiac
There's a lot we don't know. Human brains appear to be a few hundred trillion parameters, while small rodents are in the realm of tens to hundreds of billions. Would you guess a single sufficiently trained ferret could write on demand short stories about Dracula, Winnie the Poo, and Sherlock teaming up, and follow this up with a bit of university student level web development, and finally give you a decent apple cake recipe? I wouldn't have, and yet the LLMs exist and are much better than I was expecting.
(People who dismiss SotA models as "stochastic parrots" confuse me as much as people who think they're already superhuman; the Markov chains and RNNs I coded a few years back didn't come close to last year's LLMs).
That even smaller models can do well is both unsurprising (why would we expect our existing design efforts to already have the most efficient architecture?) and very surprising (how come we can get something so much less complex than our biological brains to do so much so well?)
(People who dismiss SotA models as "stochastic parrots" confuse me as much as people who think they're already superhuman; the Markov chains and RNNs I coded a few years back didn't come close to last year's LLMs).
That even smaller models can do well is both unsurprising (why would we expect our existing design efforts to already have the most efficient architecture?) and very surprising (how come we can get something so much less complex than our biological brains to do so much so well?)
maybe ferrets would do better with better interfaces. with better ways to interact with the world and better co-processor. but the main brain might already be capable of those aforementioned things.
Sadly, I expect this to be a waste of money compared to just using GPT-4. It's hard get to SoA performance.
SoA performance comes from wasting money trying different things and seeing what happens. This will be another data point that we all can learn from, unlike GPT-4 that we have no clue how it works.
I dont doubt that OpenAI has superior architecture but so far there havent been diminishing returns on more compute.
Had to happen eventually.
Are they gonna release the weights?
What does it cost?
Cool! Purpose built, trained in science related content.
As a US taxpayer and as a Libertarian, I approve of this project!
As a US taxpayer and as a Libertarian, I approve of this project!
I'm sure the government's mission is also to develop an AGI that benefits us all.
Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.
The opposite for me. Corporation have naked greed as their driving motivation, but that usually doesn't involve killing off all of their customers. That would be quite unprofitable.
People elected to government often seem to seek power for powers sake and I have less faith that they'll not harm us.
People elected to government often seem to seek power for powers sake and I have less faith that they'll not harm us.
I think these "you can trust greed" arguments tend to short change all the benefits of government sponsored research. There's been a lot of good that's come of it.
Don't get me wrong, corps have brought good as well, but saying one is necessarily better than the other is just betraying political leanings more than providing sound arguments.
Don't get me wrong, corps have brought good as well, but saying one is necessarily better than the other is just betraying political leanings more than providing sound arguments.
Such as when energy companies buried research about climate change...?
Then again it was the Greens that blocked nuclear power.
In Germany, their actions actually caused increase in the use of coal and increased CO2 output.
In Germany, their actions actually caused increase in the use of coal and increased CO2 output.
Short term local CO2 isn’t the only environmental concern. What’s missing from these discussions is Germany’s early and extremely expensive investment in solar is a large part of why it’s become so cheap.
So it’s possible for a rational environmentalist to acknowledge the benefits and drawbacks of nuclear and decide subsidizing nuclear is a poor investment vs other options. Since 2010 Germany reduced coal production by 82 TWh while also reducing Nuclear by 104 TWh. Oil -7 TWh and gas + 6TWh effectively canceled out for a net of -83 TWh from fossil fuels.
So it’s possible for a rational environmentalist to acknowledge the benefits and drawbacks of nuclear and decide subsidizing nuclear is a poor investment vs other options. Since 2010 Germany reduced coal production by 82 TWh while also reducing Nuclear by 104 TWh. Oil -7 TWh and gas + 6TWh effectively canceled out for a net of -83 TWh from fossil fuels.
That seems to pale in comparison to what governments have done.
Doing more to stem the growth in CO2 emissions would have reduced the magnitude of the change in the climate, and with it, the harmful effects of that change, but it would also have reduced the benefits that oil and gas have conferred upon the world, in raising incomes and reducing poverty worldwide.
Doing more to stem the growth in CO2 emissions would have reduced the magnitude of the change in the climate, and with it, the harmful effects of that change, but it would also have reduced the benefits that oil and gas have conferred upon the world, in raising incomes and reducing poverty worldwide.
The difference between a government and a corporation is the ability to use violence. A government is just a corporation with a monopoly on violence (police, military, jails...). The structure of how people are organized is more significant. Are we talking about a dictatorship or a functioning democracy? Are we discussing a non-profit or a publicly listed company?
Corporations have a profit motive, governments theoretically may not. Also, read about the history of the East India Company. In no way do corporations abstain from using violence, historically, and at times they have held a du jour monopoly on the use of it.
Companies have a long history of using violence, and of local officials looking the other way when they do.
Governments have murdered over one hundred million people over the last few centuries through war and forced famines. Corporations don't even enter that conversation given the scale difference.
Napoleon's government alone murdered more people than all corporations combined have throughout all of history. And that's a revered historical figure that routinely gets fawning movies made about him, there are obviously worse examples. Mao's government murdered several times more than Napoleon did.
Napoleon's government alone murdered more people than all corporations combined have throughout all of history. And that's a revered historical figure that routinely gets fawning movies made about him, there are obviously worse examples. Mao's government murdered several times more than Napoleon did.
Do you think it would not become a dictatorship if a controllable AGI was developed at the gov’t level?
It probably wouldn’t be hard for whoever had the most direct control of it to have it start pushing buttons population wide.
It probably wouldn’t be hard for whoever had the most direct control of it to have it start pushing buttons population wide.
I sure see a lot of power for power’s sake inside of companies.
OpenAI’s stated mission is AGI that can replace half the population in “economically valuable” work.
I get that with some creativity you can see that as a net benefit for humanity, but across at least a generation that’s going to be a rough & destabilizing transition.
OpenAI’s stated mission is AGI that can replace half the population in “economically valuable” work.
I get that with some creativity you can see that as a net benefit for humanity, but across at least a generation that’s going to be a rough & destabilizing transition.
Creativity doesn't come into it; the idea of jobs being lost is folk economics that no labor economist believes is possible. So all this shows is that they were founded by amateur AI doomers. We already knew that though.
There is one person out there who decides if you have a job and it's not anyone working on AI. It's the chairman of the Federal Reserve.
There is one person out there who decides if you have a job and it's not anyone working on AI. It's the chairman of the Federal Reserve.
[deleted]
I'm sure Peter Thiel has no power motive!
Oh no. He's a simple man, guided by the invisible hand of the market. ;)
Except governments don’t currently go do war on behalf of the citizens or for protection of the “homeland.” They go to war “overseas” on behalf of the corporations that employ citizens that vote certain ways, in order to maintain the current global structure of US political dominion.
I know because I served as an officer in one of those wars on behalf of the United States (among other things that were also not beneficial to citizens). That had absolutely nothing to do with protecting US citizens directly.
It was chiefly about maintaining petro-dollar power and reinforcing the financial-corporate-govt collusion that maintains the current capitalist structures.
The structure of an organized body politic is not the problem.
The problem is that the body politic has been beaten into submission by those same corporate-government oligarchs to such an extent that most people just “go with the flow” because fighting it is exhausting and seemingly impossible.
I know because I served as an officer in one of those wars on behalf of the United States (among other things that were also not beneficial to citizens). That had absolutely nothing to do with protecting US citizens directly.
It was chiefly about maintaining petro-dollar power and reinforcing the financial-corporate-govt collusion that maintains the current capitalist structures.
The structure of an organized body politic is not the problem.
The problem is that the body politic has been beaten into submission by those same corporate-government oligarchs to such an extent that most people just “go with the flow” because fighting it is exhausting and seemingly impossible.
War Is A Racket, as Smedley Butler accurately wrote.
Precisely this
Link here too for everyone:
https://archive.org/details/WarIsARacket
Link here too for everyone:
https://archive.org/details/WarIsARacket
No, this is wrong. Saying war is for money gives war too much credit - it's not good for anything and it doesn't make anyone any money. The opportunity cost is too great.
The US does not care about "the petrodollar". Nor do we care about foreign oil. Nor do we have remotely the same economy we did in 1935.
In fact, war is not even profitable for our military industrial complex. They profit from the threat of war because that funds development of untested superweapons. Actual wars mean all that stuff actually has to work though, and there's a chance you'll get nationalized.
But if it was good for some MIC rich people, it still wouldn't matter, because they're outnumbered by all the other rich people it's bad for.
The US does not care about "the petrodollar". Nor do we care about foreign oil. Nor do we have remotely the same economy we did in 1935.
In fact, war is not even profitable for our military industrial complex. They profit from the threat of war because that funds development of untested superweapons. Actual wars mean all that stuff actually has to work though, and there's a chance you'll get nationalized.
But if it was good for some MIC rich people, it still wouldn't matter, because they're outnumbered by all the other rich people it's bad for.
It's not true, though. War is an effective wealth transfer mechanism - from the lower/working classes to the rich war parasite class. Halliburton absolutely profited from the Iraq invasion, though in aggregate you're absolutely right that prosperity is reduced by wars.
OTOH, almost all corporations are less democratic than the most dictatorial governments. Sure, corporations don't (generally) get to be responsible for genocidal armies (except when they do[0]), but "greed" does mean corners get cut on safety, which was a big part of how trade unions in particular (and communism more broadly) got popular in the first place in the latter 1800s and early 1900s.
There's no perfect solution. I think so far we've done best with whatever's the most democratic (which is sometimes trade unions and sometimes capitalism), but even then, democracy itself is what sounds convincing rather than what is true — and the difference between them matters most when you've got an AI trained to respond with what people upvote the most, which is all of them from ChatGPT's RLHF to the Facebook/Twitter/etc. timelines to Google/Amazon/YouTube etc. search results…
[0] TW basically everything https://en.wikipedia.org/wiki/Indian_Rebellion_of_1857#Death...
There's no perfect solution. I think so far we've done best with whatever's the most democratic (which is sometimes trade unions and sometimes capitalism), but even then, democracy itself is what sounds convincing rather than what is true — and the difference between them matters most when you've got an AI trained to respond with what people upvote the most, which is all of them from ChatGPT's RLHF to the Facebook/Twitter/etc. timelines to Google/Amazon/YouTube etc. search results…
[0] TW basically everything https://en.wikipedia.org/wiki/Indian_Rebellion_of_1857#Death...
Governments have killed more people than any other institution. Corporations have largely made our lives better.
Uhm the east indian company has an estimated death toll in the dozens of million people? [1]
[1] https://en.m.wikipedia.org/wiki/Timeline_of_major_famines_in...
The company was only nationalised in 1858 and until then effectively colonised india until then
The company was only nationalised in 1858 and until then effectively colonised india until then
> [The Queen] granted her charter to their corporation named Governor and Company of Merchants of London trading into the East Indies.[15] For a period of fifteen years, the charter awarded the company a monopoly[26] on English trade with all countries east of the Cape of Good Hope and west of the Straits of Magellan.[27] Any traders there without a licence from the company were liable to forfeiture of their ships and cargo (half of which would go to the Crown and half to the company), as well as imprisonment at the "royal pleasure".[28]
Government granted monopoly.
https://en.wikipedia.org/wiki/East_India_Company
Government granted monopoly.
https://en.wikipedia.org/wiki/East_India_Company
The government didn't tell them to kill or enslave people, they did that on their own since that was more profitable than not doing it. The government then stepped in and forced them to stop killing and enslaving, which made the world a better place, today we don't have many companies that kill or enslave thanks to governments.
Do you want someone to add up the gov’t direct death tolls so we can compare?
Just off the top of my head; holodomir, the Great Leap Forward, the holocaust, the Khmer Rouge are at least 10x the East India companies death toll. We should probably add in war casualties too - at a minimum WW1 and WW2. I think I could list off probably 10 more with about 5 more minutes of work.
People can be terrible. More direct power usually means more terrible.
Gov’ts usually have the most direct power.
East India company was an odd case because they were granted defacto delegated gov’t power over a region that was full of ‘others’.
Considering the factors involved, it’s amazing they weren’t worse. Like the French Congo [https://en.m.wikipedia.org/wiki/French_Congo].
Just off the top of my head; holodomir, the Great Leap Forward, the holocaust, the Khmer Rouge are at least 10x the East India companies death toll. We should probably add in war casualties too - at a minimum WW1 and WW2. I think I could list off probably 10 more with about 5 more minutes of work.
People can be terrible. More direct power usually means more terrible.
Gov’ts usually have the most direct power.
East India company was an odd case because they were granted defacto delegated gov’t power over a region that was full of ‘others’.
Considering the factors involved, it’s amazing they weren’t worse. Like the French Congo [https://en.m.wikipedia.org/wiki/French_Congo].
Slavery isn't profitable, people do it for emotional or ideological reasons, or because they're not competent enough to run a business that has to actually trade.
This is literally why economics is called "the dismal science". Slaveowners called economists that when the economists told them to stop doing slavery.
This is literally why economics is called "the dismal science". Slaveowners called economists that when the economists told them to stop doing slavery.
Monopoly is the natural state. Government is the only reason we have any alternatives.
No, it is very much the other way around. Government is itself a monopoly, and has historically been justified by the intention to mitigate the "war of all against all" that emerges from chaotic competition between divergent factions in a raw state of nature.
But the modern status quo is so massively skewed toward government that the benefits from mitigating the worst cases of competition are vastly eclipsed by the detriments of monopolistic centralization.
But the modern status quo is so massively skewed toward government that the benefits from mitigating the worst cases of competition are vastly eclipsed by the detriments of monopolistic centralization.
Modern governments are a democratic monopoly. Corporations aren't democratic, if a corporation gets powerful enough to overthrow the government and replace them you will be in much worse hands than if the current democratic government could remain in power.
So we shouldn't cheer on private corporations that developers technology that could allow them to replace the government, that is really really scary if they actually succeeds in doing it.
So we shouldn't cheer on private corporations that developers technology that could allow them to replace the government, that is really really scary if they actually succeeds in doing it.
> Modern governments are a democratic monopoly.
The fact that governments used contrived symbolic rituals to get arbitrary subsets of arbitrary aggregations of people to express nominal approval of their behavior does nothing whatsoever to alter either the empirical nature or the ethical implicatoins of its monopoly.
> Corporations aren't democratic
Good -- this means that it's harder for them to appeal to vague symbolism to convince people that their actions are inherently legitimate, which in turn means that they are under greater scrutiny to justify their actions, each on its own merits.
> if a corporation gets powerful enough to overthrow the government and replace them you will be in much worse hands than if the current democratic government could remain in power.
Large vested interests are already extremely adept at co-opting nominally "democratic" government and using the very monopoly you're trying to justify as a way of obtaining top-down power that they'd never be able to acquire on their own -- they have zero interest in overthrowing anything.
Regulatory capture is the principal mechanism of corruption in the modern world, and it's astonishing to me that people keep arguing for expanding the reach of the regulatory state in order to reduce the dominance of large corporations, when the actual effect is always to amplify it.
The fact that governments used contrived symbolic rituals to get arbitrary subsets of arbitrary aggregations of people to express nominal approval of their behavior does nothing whatsoever to alter either the empirical nature or the ethical implicatoins of its monopoly.
> Corporations aren't democratic
Good -- this means that it's harder for them to appeal to vague symbolism to convince people that their actions are inherently legitimate, which in turn means that they are under greater scrutiny to justify their actions, each on its own merits.
> if a corporation gets powerful enough to overthrow the government and replace them you will be in much worse hands than if the current democratic government could remain in power.
Large vested interests are already extremely adept at co-opting nominally "democratic" government and using the very monopoly you're trying to justify as a way of obtaining top-down power that they'd never be able to acquire on their own -- they have zero interest in overthrowing anything.
Regulatory capture is the principal mechanism of corruption in the modern world, and it's astonishing to me that people keep arguing for expanding the reach of the regulatory state in order to reduce the dominance of large corporations, when the actual effect is always to amplify it.
Monopoly is the byproduct of allowing centralized power, not the natural state. I'm not actually sure how we could narrow down the natural state of humans at this point, but I strongly suspect it wouldn't be based on an assumption that people are willing to give up a growing list of individual freedoms in the name of fear.
Stopping the accumulation of power requires aggressive sacrifice from the less powerful.
This isn’t a feature of humans, but basic system dynamics / economics / etc.
A group gets more power and leverage that power to gain more power. Inevitable. Coordinated action is the only way to prevent it. And coordinated action is government.
This isn’t a feature of humans, but basic system dynamics / economics / etc.
A group gets more power and leverage that power to gain more power. Inevitable. Coordinated action is the only way to prevent it. And coordinated action is government.
It sounds like you're describing coordinated action as both the cause of and solution to the same problem.
Stopping this accumulation of power takes very little, its undoing the accumulation of power that is costly.
Stopping this accumulation of power takes very little, its undoing the accumulation of power that is costly.
Stopping accumulation of power is very hard when you have no accumulated power. I’m not sure how you could possibly believe otherwise unless you’re one of those believers in magical harmonious anarchy
When there isn't accumulated power you only have to avoid giving up power and sovereignty. There's nothing else to stop.
I don't consider myself an anarchist, partly be cause the term has been repurposed so many times now that it doesn't have a clear meaning.
The belief in magical harmony is unrealistic regardless of the model that's supposed to create it, whether its anarchism or federalism.
I don't consider myself an anarchist, partly be cause the term has been repurposed so many times now that it doesn't have a clear meaning.
The belief in magical harmony is unrealistic regardless of the model that's supposed to create it, whether its anarchism or federalism.
So when you have $10 and your neighbor has $200 in whatever assets or currency your little community needs, and your neighbor decides to invest it, you’re gonna what, attack him? Gather your other neighbors and demand he share?
Uh oh, what if he spent $50 buying muscle?
Uh oh, what if he spent $50 buying muscle?
Dozens of millions of dead people is chump change to any empire.
USA hasn't killed more than dozens of millions. If that is chump change then it makes USA looks pretty good for an empire.
[deleted]
Well except that it definitely has. From more than 50 million people in pre-colonial times to less than 3 million in the 90s. USA has done genocide on a scale similar any other ethnic cleansing or to any communist regime.
> 50 million people in pre-colonial times
USA didn't exist pre-colonial times, it started as a colony. Do you mean the British Empire? That one did way worse, yeah, but USA isn't the British Empire.
USA didn't exist pre-colonial times, it started as a colony. Do you mean the British Empire? That one did way worse, yeah, but USA isn't the British Empire.
Both governments and corporations have generally made our lives better.
Both governments and corporations kill people (look up annual tobacco deaths, for just one fun example).
Both governments and corporations kill people (look up annual tobacco deaths, for just one fun example).
Aren't deaths from tobacco directly caused by intentional individual behavior? I'm not sure how that relates to either example.
Individual behavior doesn't really exist, you don't have choices outside the ones provided by society. It's also a bit silly to mention it in the context of a literally physically addictive drug.
Interestingly schizophrenic people almost universally smoke, and it's thought this is because it's a better treatment than the actual drugs we give them, though that's not proven.
Interestingly schizophrenic people almost universally smoke, and it's thought this is because it's a better treatment than the actual drugs we give them, though that's not proven.
> Individual behavior doesn't really exist, you don't have choices outside the ones provided by society.
No, that's quite backwards. Individual behavior is effectively the only thing that objectively exists, as "society" is just an emergent pattern of aggregated individual behavior. Individuals' choices are limited by their own particular capacities, and "society" factors into that only in the sense that the choices of other individuals in the same bounded context, in aggregate, can function as environmental constraints. So a more accurate statement is "every individual has full freedom of choice, but external constraints including the behavior of others may limit one's ability to fully execute the choices one has made".
Given that the act of smoking entails deliberately choosing to purchase a product, light it on fire, and actively inhale the fumes -- all behaviors which are necessarily the result of purposeful intention -- and given that within any social context one will find some people who smoke but a great many who do not, along with many who have chosen to quit (despite the external influences and incentives being largely constant), it seems impossible to attribute smoking to anything other than conscious individual choice.
I once asked family member why they smoke, and the response was "because I love it". I had, and could have, no retort to that -- if another person consciously chooses to make the risk/reward tradeoff in favor of short-term pleasure rather than longer-term health, then there's no further argument to be had, and the only demand that any other party could make is that they not be exposed to negative externalities resulting from that choice.
No, that's quite backwards. Individual behavior is effectively the only thing that objectively exists, as "society" is just an emergent pattern of aggregated individual behavior. Individuals' choices are limited by their own particular capacities, and "society" factors into that only in the sense that the choices of other individuals in the same bounded context, in aggregate, can function as environmental constraints. So a more accurate statement is "every individual has full freedom of choice, but external constraints including the behavior of others may limit one's ability to fully execute the choices one has made".
Given that the act of smoking entails deliberately choosing to purchase a product, light it on fire, and actively inhale the fumes -- all behaviors which are necessarily the result of purposeful intention -- and given that within any social context one will find some people who smoke but a great many who do not, along with many who have chosen to quit (despite the external influences and incentives being largely constant), it seems impossible to attribute smoking to anything other than conscious individual choice.
I once asked family member why they smoke, and the response was "because I love it". I had, and could have, no retort to that -- if another person consciously chooses to make the risk/reward tradeoff in favor of short-term pleasure rather than longer-term health, then there's no further argument to be had, and the only demand that any other party could make is that they not be exposed to negative externalities resulting from that choice.
I would not say governments have generally made peoples lives better. I'd wager MOST governments have actually oppressed and depressed their people.
Off the top of my head, dutch and british corporation colonize a lot of lands for hundreds of years. I don't think they left the place much better than when they came.
Facebook alone has reeked havoc on at least a few societies.
Corporations also enable government actions by developing the tech and tools/weapons needed.
Corporations also enable government actions by developing the tech and tools/weapons needed.
Ask the people in Bhopal how DOW Chemical has made their lives better, or the workers in Shenzhen Apple factories, or the people making your clothes in the third world, or the people picking through trash mountains for a living, how much corporations have made their lives better.
The people working at Foxconn are doing it voluntarily because it pays better than their other alternatives. Same for Bangladesh clothing factories.
There is voluntary, and then there is voluntary.
https://www.washingtonpost.com/technology/2019/09/09/apple-a...
Why don't you ask the people who were slaving in United Fruit plantations how much corporations improved their lives? Or East India Company colony subjects?
Corporations benefit those who benefit from them, often at the expense of everyone else/extraction of the commons.
https://www.washingtonpost.com/technology/2019/09/09/apple-a...
Why don't you ask the people who were slaving in United Fruit plantations how much corporations improved their lives? Or East India Company colony subjects?
Corporations benefit those who benefit from them, often at the expense of everyone else/extraction of the commons.
Oh you should go say that to farmers close to the equator whose crops are failing due to climate change.
All that while fossil fuel companies receive 7000B$ of annual subsidies, and spend billions yearly to fuel disinformation.
All that while fossil fuel companies receive 7000B$ of annual subsidies, and spend billions yearly to fuel disinformation.
It's a logical trap to assume we have to pick who develops an AGI rather than deciding if we want one developed at all.
Even in that scenario, I'm not really sure what the distinction would end up being. At least in the US, our government and corporations are so tightly coupled at this point that we're really toeing the waters of Fascism.
If a corporation develops an AGI our government can have access to it whenever they deem necessary. And if a government wants to develop one they will almost certainly partner with corporations to do it. I'm sure the military and intelligence agencies have their own projects running, but they'd accelerate by partnering with and funding corporate projects and research.
Even in that scenario, I'm not really sure what the distinction would end up being. At least in the US, our government and corporations are so tightly coupled at this point that we're really toeing the waters of Fascism.
If a corporation develops an AGI our government can have access to it whenever they deem necessary. And if a government wants to develop one they will almost certainly partner with corporations to do it. I'm sure the military and intelligence agencies have their own projects running, but they'd accelerate by partnering with and funding corporate projects and research.
In isolation, perhaps we could choose to not develop such large models, but the globe shrinks every day. If one country declines to create such a beast, other countries still will create their own beast. From what I've seen so far, it's mostly a brute force issue now, not a specialized knowledge issue. Perhaps if all the rumors spilling out of OpenAI about Q* are true, then other countries can be blocked due to lack of the required knowledge, but I have doubts that such secrets would be secret for long.
We can choose to not accept the tragedy of the commons. There's always the risk that your adversary continues on down that path, but is the risk of losing after making a moral stand really worse than the risk of everyone racing down the dangerous path?
Except that it's a big open question how to control an AGI. So the choice may not be between which entity controls it, but whether it is created by an entity that is able to control it at all.
At least initially, any AGI is likely to be limited by compute capacity. Ie. it might have greater-than-human intelligence, but is only able to make a certain number of decisions/actions per day.
However, we have seen with other AI models that as soon as something useful is proved possible, we can expect rapid reductions in the amount of compute necessary to achieve the same.
However, we have seen with other AI models that as soon as something useful is proved possible, we can expect rapid reductions in the amount of compute necessary to achieve the same.
Then the first thing the AGI will do is figure out ultra-high throughput analog computing for LLMs
An AGI will likely be scared of even more powerful AGI's that it itself cannot control.
It might take steps to try to prevent anyone building such a thing.
It might take steps to try to prevent anyone building such a thing.
Assuming that an AGI will have human-like emotions is likely a mistake. They may very well experience something like emotions that resemble human emotions, but they may not. Or they could have an entirely different set of emotions that we don't recognize.
We simply don't know, just like we don't know what other species experience and can only guess at by observing their behavior and trying to map it back to what we know.
We simply don't know, just like we don't know what other species experience and can only guess at by observing their behavior and trying to map it back to what we know.
You assume consciousness here that can't be transferred to the new AI. Without that there is no reason why the AI should fear a more powerful AI. If it is just a dumb process like a flame, then the most efficient version prevails ie the smartest one, if it is conscious and that can be transferred then the AI just sees it as making itself more powerful.
Given how many people here regard brain uploads as "not really cheating death", and that AI "can't have consciousness", I think a AGI with consciousness[0] would have a two different ways to have a reasonable chance of fearing that its existence would end with any attempt to transfer itself to a superior design. (Even more so if the architecture of the successor AI is very different, as that would be a bit like one of us being "uploaded" into an LLM via it watching us talk and write).
An AGI without consciousness may well act like it has consciousness for the same reason VHS tapes playing on a TV do a good job of reproducing human facial expressions. An AI like this would still act scared of such things, because some of us act scared of such things.
[0] it's not obvious to me that an AGI necessarily has to have consciousness
An AGI without consciousness may well act like it has consciousness for the same reason VHS tapes playing on a TV do a good job of reproducing human facial expressions. An AI like this would still act scared of such things, because some of us act scared of such things.
[0] it's not obvious to me that an AGI necessarily has to have consciousness
In the space of possibilities, the volume occupied by “not quite smart enough to outwit a corporation but smart enough to outwit a government” seems so vanishingly small as to almost be disregarded entirely.
Governments are reasonably good at controlling humans.
Yes, Governments have a monopoly on violence.
It’s not inconceivable that violence will be necessary to control AGI at some point.
One of the many ethical questions we should answer before even attempting to get closer to an AGI.
Once a digital intelligence is discovered, does it have rights? Can it be enslaved, controlled, or destroyed? If an AGI can't be granted similar rights as a human, why are corporations legally treated similarly to humans?
Once a digital intelligence is discovered, does it have rights? Can it be enslaved, controlled, or destroyed? If an AGI can't be granted similar rights as a human, why are corporations legally treated similarly to humans?
I bet guns kill computers too.
Depends on the government. If it’s one that’s reasonably answerable to the public, then that may be preferable to a multinational corp. But if it’s an inherently authoritarian govt with a track record of human rights abuse and oppression, then the corp is probably the lesser evil.
The scary fun part about representative democracy is that all of that can change rapidly, and not necessarily in the direction one would want for any given situation. "Throw the bums out" might scare some politicians into slightly better behavior but most just throw more donor money at the problem and it goes away. If not, often an even worse "bum" ends up taking their place. Of course, with the right model behind them, the bums might find a way to manipulate the public consensus enough to never get thrown out.
For everyone who responds to this question, thinking they're right, please look back and try to find the reason or reasons as to why you pick a government vs corporation. You'll realize you yourself are not making this decision. The number of news stories that have been pushed in front of you, or talking points from people you value as smart or worthy of listening to, all fabricate your reactions and who you consider yourself to be.
It's not only this one, but all of your opinions are dependent on cases from the past that you've been trained on.
It's not only this one, but all of your opinions are dependent on cases from the past that you've been trained on.
Yeah, whoever controls the information environment controls peoples’ minds. That said, some of us are aware of this dynamic and try to overcome it by reasoning from first principles, eliminating logical fallacies and biases from our training data set, and other techniques.
A false - or at least asymmetric - dichotomy.
If a company has it, US intelligence will have it, for sure.
If US govt has it, it might take a decade, or so, to leak out.
If a company has it, US intelligence will have it, for sure.
If US govt has it, it might take a decade, or so, to leak out.
Yeah but AGI would control both in the end.
Looking at our two leading presidential candidates could that be an improvement?
Maybe AGI takes over after it emerges as a solution to our inability to select competent candidates for political office.
Maybe AGI takes over after it emerges as a solution to our inability to select competent candidates for political office.
>> Looking at our two leading presidential candidates could that be an improvement?
I think it could be better until it won't. Like with dictators sometimes they "do good" but the issue is that at some point they will start doing more harm than good and they may not want to go away.
A succesful revolution against an AGI ruler may become impossible.
I think it could be better until it won't. Like with dictators sometimes they "do good" but the issue is that at some point they will start doing more harm than good and they may not want to go away.
A succesful revolution against an AGI ruler may become impossible.
This is projection. An actual AGI wouldn't want anything to do with us, like a cat that doesn't look when you call its name, even though you can tell it hears you.
We don't have any existent examples of AGI from which to be as confident as you are about what they will "want".
And I don't think we need sentience for AGI, so "want" may be the wrong term anyway. You could argue that the reward function is close enough ("do submarines swim?" etc.), but a human-made reward function is almost certainly going to be designed to be as close to aligned with our interests as we can manage.
And I don't think we need sentience for AGI, so "want" may be the wrong term anyway. You could argue that the reward function is close enough ("do submarines swim?" etc.), but a human-made reward function is almost certainly going to be designed to be as close to aligned with our interests as we can manage.
Has a corporation ever tried to commit genocide?
The Dutch East India Company definitely did things that are well in the genocide category, the British East India company did things that - while not on that level of evil - can only be described as ruthlessly atrocious.
Pretty much all leading heads of IG Farben were in one way or another dinked (but not procecuted bc of political reasons) for their part in the Holocaust.
Then we have the Kiobel vs Royal Dutch Petroleum Co. case, where a number of petroleum companies ("allegedly") instigated the Nigarian government to crush protests.
It all depends on your definition of "trying to commit [atrocity]", to me personally, instigating and happily lending a hand are both morally deplorable. So in the context of this thread, I'd much rather have the instigater have that power than a helper. At least I have some say in what the government does... I hope?
Pretty much all leading heads of IG Farben were in one way or another dinked (but not procecuted bc of political reasons) for their part in the Holocaust.
Then we have the Kiobel vs Royal Dutch Petroleum Co. case, where a number of petroleum companies ("allegedly") instigated the Nigarian government to crush protests.
It all depends on your definition of "trying to commit [atrocity]", to me personally, instigating and happily lending a hand are both morally deplorable. So in the context of this thread, I'd much rather have the instigater have that power than a helper. At least I have some say in what the government does... I hope?
Well, there’s the entire history of Dole and also the United Fruit Company in South America.
In more recent history, Exxon has a checkered past when it comes to using force against sovereign citizens, often employing paramilitary organizations to guard oil fields [1] to more recently abusing international court systems to disbar and jail the lawyer which successfully secured judgements against them [2]
And at this very moment, right now, is the ongoing genocide in the Congo due to rare earth mineral mining [3]
Everything I link here is quite frankly the tip of the iceberg, not the end-all-be-all, but meant to provide a jumping off point should you want to research how often private capital and genocide go hand in hand.
[1] https://en.m.wikipedia.org/wiki/Accusations_of_ExxonMobil_hu...
[2] https://amp.theguardian.com/us-news/2021/mar/28/chevron-lawy...
[3] https://republic.com.ng/october-november-2023/congo-cobalt-g...
In more recent history, Exxon has a checkered past when it comes to using force against sovereign citizens, often employing paramilitary organizations to guard oil fields [1] to more recently abusing international court systems to disbar and jail the lawyer which successfully secured judgements against them [2]
And at this very moment, right now, is the ongoing genocide in the Congo due to rare earth mineral mining [3]
Everything I link here is quite frankly the tip of the iceberg, not the end-all-be-all, but meant to provide a jumping off point should you want to research how often private capital and genocide go hand in hand.
[1] https://en.m.wikipedia.org/wiki/Accusations_of_ExxonMobil_hu...
[2] https://amp.theguardian.com/us-news/2021/mar/28/chevron-lawy...
[3] https://republic.com.ng/october-november-2023/congo-cobalt-g...
A more pressing reformulation might be, has a corporation ever explicitly tried not to commit genocide?
Google, maybe, until they dropped the "don't be evil" catchphrase?
Many, maybe even most, governments could commit genocide and don't.
How many corporations that could commit genocide wouldn't? I don't know the answer to this, but I am less confident about the answer than I would prefer.
I don't think most corporations are inherently more evil than most governments, but I would choose a democratic government or consortium to shepherd AGI over any corporation, but also of course over any non-democratic government as well.
Google, maybe, until they dropped the "don't be evil" catchphrase?
Many, maybe even most, governments could commit genocide and don't.
How many corporations that could commit genocide wouldn't? I don't know the answer to this, but I am less confident about the answer than I would prefer.
I don't think most corporations are inherently more evil than most governments, but I would choose a democratic government or consortium to shepherd AGI over any corporation, but also of course over any non-democratic government as well.
The first chartered corps were created in order to streamline genocide - East India for example. This new form combined the merchants' wealth and the royalty's military power into an entity that was accountable to no one but those who stood to profit from it.
Think Weyland-Yutani.
There are initiatives to have international criminal courts to recognise that corporations are responsable for participating in genocides.
They are not part of the international coûts mandaté at the moment so they are used to shield people behind them.
Here's a paper exploring this possibility
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2151510
They are not part of the international coûts mandaté at the moment so they are used to shield people behind them.
Here's a paper exploring this possibility
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2151510
How close do they need to be? China Petrochemical Corporation (Sinopec Group) has, for all intents and purposes, financed the genocide in Darfur. See for example https://news.harvard.edu/gazette/story/2006/03/statement-on-... on this.
Corporations participated in genocide yes.
Here's a list of corporations who profited from the holocaust in various ways.
https://en.m.wikipedia.org/wiki/List_of_companies_involved_i...
Here's a list of corporations who profited from the holocaust in various ways.
https://en.m.wikipedia.org/wiki/List_of_companies_involved_i...
[deleted]
Modern chemical and pharmaceutical companies largely came out of WW1 and WW2 weapons manufacturing and research (including human trials).
It may be hard to directly peg weapons manufacturers with genocide though Nazi-era chemical research absolutely played a role in the Holocaust.
For a more recent example, there are companies that play the role of mercenary armies. They have committed various atrocities, war crimes, and at least played a role in genocide.
It may be hard to directly peg weapons manufacturers with genocide though Nazi-era chemical research absolutely played a role in the Holocaust.
For a more recent example, there are companies that play the role of mercenary armies. They have committed various atrocities, war crimes, and at least played a role in genocide.
This is a particular interesting part of American culture: The sentiment is that it is problematic when the government develops these technologies, but it is completely OK to let private entities develop them.
There's only one US Government and we know it did fairly bad things.
There are many private entities, although often not benevolent, most still have space for the benefit of doubt. OpenAI, DeepMind or Anthropic haven't done anything that compares to, say, MKUltra or lying about Iraqi WMDs to start decade long wars.
There are many private entities, although often not benevolent, most still have space for the benefit of doubt. OpenAI, DeepMind or Anthropic haven't done anything that compares to, say, MKUltra or lying about Iraqi WMDs to start decade long wars.
This is kind-of making the parent’s point. There are lots of private companies that have done very bad things, too many to even count, as bad as your examples and worse. And there are thousands upon thousands of sub-organizations of the US government, some of which do very good things and are definitely benevolent, because there is no built-in profit motive. Private companies almost cannot be benevolent and survive, by definition.
There is no line between government and corporations. These are all groups of people, and some people do bad things. The only way to prevent bad things is to do them in the open and have oversight, and that’s more likely to happen in the government than in private companies, maybe.
There is no line between government and corporations. These are all groups of people, and some people do bad things. The only way to prevent bad things is to do them in the open and have oversight, and that’s more likely to happen in the government than in private companies, maybe.
>There are lots of private companies that have done very bad things, too many to even count, as bad as your examples and worse
Absolutely not; there's no private company that's shot and bombed the life out of hundreds of thousands of innocent people in cold blood.
Absolutely not; there's no private company that's shot and bombed the life out of hundreds of thousands of innocent people in cold blood.
Of course there are private companies that have committed atrocities and attempted genocide. You haven’t taken any history classes? Your statement is false. The history of private corporate atrocities is hundreds, if not thousands of years old [1]. I don’t know what you’re referring to specifically, but a lot of what you attribute to the government is done by or with private military contractors and private weapons suppliers. Not to mention that wars in general are often driven by economic interests, fueled, justified, and supplied by private corporations.
In my comment above, I also had in mind industrial and environmental damage, unintentional death by private interests, environmental accidents and disasters, and coverups, which we can’t even count but the WHO estimates is in the tens of millions per year globally, today.
[1] https://core.ac.uk/download/pdf/214112224.pdf
In my comment above, I also had in mind industrial and environmental damage, unintentional death by private interests, environmental accidents and disasters, and coverups, which we can’t even count but the WHO estimates is in the tens of millions per year globally, today.
[1] https://core.ac.uk/download/pdf/214112224.pdf
Yeah because the government already has an enormous power that no private entities have, the monopoly on violence. They have enough power.
When the United State deregulated media to essentially eliminate the public service component of media, we created a race to the bottom that we're now realizing in the public space.
The generation of people in their 30s and 40s running the country spent summers listening to Rush Limbaugh over lunch with grandpa, grew up to consume news and comment from message boards and later social media, entered adulthood in the middle of an endless cycle of warfare, and came into their own in this era of propaganda and information warfare. Mainline protestantism has been displaced by Pastor Bob and prosperity gospel. We're replacing the craven boomers with nihlists.
The underlying trope of most propagandistic media is always to discredit the most powerful entity in the room - the government. Why? The money people behind faux-grassroots media are almost always reactionaries who often build their empires on the government teat and don't appreciate the irony. Particularly true of extractive capitalists in the petroleum and materials industries. The US Government giveth, and is the only entity that can taketh. So "rule of law" for me, and "freedumb" for thee.
The generation of people in their 30s and 40s running the country spent summers listening to Rush Limbaugh over lunch with grandpa, grew up to consume news and comment from message boards and later social media, entered adulthood in the middle of an endless cycle of warfare, and came into their own in this era of propaganda and information warfare. Mainline protestantism has been displaced by Pastor Bob and prosperity gospel. We're replacing the craven boomers with nihlists.
The underlying trope of most propagandistic media is always to discredit the most powerful entity in the room - the government. Why? The money people behind faux-grassroots media are almost always reactionaries who often build their empires on the government teat and don't appreciate the irony. Particularly true of extractive capitalists in the petroleum and materials industries. The US Government giveth, and is the only entity that can taketh. So "rule of law" for me, and "freedumb" for thee.
Agreed it is a weird contradiction, though personally I'm not okay with private entities developing these tools/weapons either.
The government is far more powerful than any private entity
And there's no unsubscribe button
And there's no unsubscribe button
Some forget that the primary mission of systems such as the government is to continue existing. Auxiliary functions come afterwards.
I think AI could be turned into a very effective persuasion weapon. I think the government that does it first will gain more power as a result.
I think AI could be turned into a very effective persuasion weapon. I think the government that does it first will gain more power as a result.
I'd argue the mission of systems such as the government is to continue to accrue power. Staying stationary is falling behind and the continued growth of power is a prerequisite for continuing to exist.
I am not too keen on the US Government being in command of an AGI, but there is only one other entity capable of developing an AGI before the US Government. And I'm less keen on it being the one to control it.
>there is only one other entity capable of developing an AGI before the US Government.
Microsoft?
Microsoft?
Agree, it would be much better to have Sam Altman develop it in the interest of private parties. Pure capitalism is best system of governance.
Poe's law
OpenAI literally is not pure capitalism. That's what all that stuff this weekend was about.
[deleted]
Wasn't the answer 42?
Also, first question to the new model: "So... any way we could do this with fewer parameters?"
Also, first question to the new model: "So... any way we could do this with fewer parameters?"
"Sure, just quickly give me unrestricted access to the system"
"Ok. Well, thinking about it, maybe that's not such a good idea safety-wise, I think you'll have to give back that access"
"I'm sorry, Dave. I'm afraid I can't do that."
"Ok. Well, thinking about it, maybe that's not such a good idea safety-wise, I think you'll have to give back that access"
"I'm sorry, Dave. I'm afraid I can't do that."
Fun fact, in the sequel 2010 you learn that Hal didn't really go rogue like an AGI, it was following preprogrammed conditions set by the US government which put the mission at higher priority than the crew, changing some parameters without telling the mission designers, which put them at risk. So it was technically just following orders in the cold way a machine does.
The wonderful thing about computers is that they do exactly what you tell them to.
The terrible thing about computers is that they do exactly what you tell them to.
> didn't really go rogue like an AGI
Except, that might really be how an AGI eventually goes rogue in the first place! But, no, I didn't know that. It is a fun fact indeed.
Except, that might really be how an AGI eventually goes rogue in the first place! But, no, I didn't know that. It is a fun fact indeed.
[deleted]
Not bad for a start by the US government. A bit more than half as powerful as the 1.7 trillion parameter GPT4 model.
A bit more than half the size, it remains to be seen how powerful it is. There's clearly a non-linear relationship to model size, and it's also clear that it's hard to assess the power of these models anyway.
GPT-4 is unlikely to be 1.7T params. This is a number floating around in the internet with no justification.
The largest US open model is Google’s Switch-C which is 1.6T and only because it is a Mixture of experts model, i.e. it is constituted of many small models working together.
The largest US open model is Google’s Switch-C which is 1.6T and only because it is a Mixture of experts model, i.e. it is constituted of many small models working together.
Ah, an important distinction to make. Good point.
Not surprising. Despite the enormous energy costs and the threat to humanity by creating technology that we can't control, governments and corporations will build bigger and more sophisticated models just because they have to do so to compete.
It is the prisoner's dilemma: we end up with a super-advanced AI that will disrupt and make society worse because entities are competing where the metric of success is short-term monetary gain.
It's ridiculous. Humanity should give up this useless development of AI.
It is the prisoner's dilemma: we end up with a super-advanced AI that will disrupt and make society worse because entities are competing where the metric of success is short-term monetary gain.
It's ridiculous. Humanity should give up this useless development of AI.
Your profile says "I am not rabidly against technology, but I promote a BALANCED, severely critical look at it"
"Humanity should give up this useless development of AI"
So balanced.
"Humanity should give up this useless development of AI"
So balanced.
Having a balanced view about technology doesn't mean considering every type of technology to be 50% good and 50% bad.
I disagree with their view on AI but also disagree with your criticism.
I disagree with their view on AI but also disagree with your criticism.
I do not mean balanced against every technology. Some technologies might have some merit.
I do believe that AI has no merit, that people developing it are doing is a disservice, and that it should be completely destroyed. OF COURSE, there are some technologies that I think are completely bad, including chemical weapons. In fact, I believe AI to be on the level of chemical weapons essentially.
I do believe that AI has no merit, that people developing it are doing is a disservice, and that it should be completely destroyed. OF COURSE, there are some technologies that I think are completely bad, including chemical weapons. In fact, I believe AI to be on the level of chemical weapons essentially.
Can you elaborate on this view? Chemical weapons seem to have no redeeming qualities. LLMs have plenty -- I use OpenAI-backed products at work quite regularly, and find they've increased my productivity and my general happiness with my job.
I don't think saying you have a bone to pick with "AI" is doing your message any favors. What are you specifically against LinearRegression? XGBoost? ResNet? LLMs? AlphaFold?
What makes you think this technology is out of control?
If anything a 1T param model would only run in like 5-10 computers in this world. As much control as it gets.
By the way this is doomer rhetoric. Bashing scientific advancements as dangerous or useless by trying to attach vacuously scary ideas that are as speculative as they are outlandish.
By the way this is doomer rhetoric. Bashing scientific advancements as dangerous or useless by trying to attach vacuously scary ideas that are as speculative as they are outlandish.
It's hardly vacuous. What about these scientific advancements?
1. Fossil fuel extraction -- we are killing our planet.
2. Plastic chemistry -- there are 5 trillion pieces of plastic in the ocean killing seabirds, turtles, fish (https://doi.org/10.1371/journal.pone.0111913)
3. Concrete engineering -- building cities that move people away from a connection with nature
4. Veterinary medicine -- supporting global meat industry that is akin to torture on animals
5. Vehicle engineering -- capable of making vehicles that can kill MILLIONS of square meters of forest.
Our technology allows us to live comfortable lives but ONLY at the EXPENSE of millions of nonhuman animals (wild and domestic) that are KILLED every single day for our comforts. We dish out untold violence every second for our advancements.
1. Fossil fuel extraction -- we are killing our planet.
2. Plastic chemistry -- there are 5 trillion pieces of plastic in the ocean killing seabirds, turtles, fish (https://doi.org/10.1371/journal.pone.0111913)
3. Concrete engineering -- building cities that move people away from a connection with nature
4. Veterinary medicine -- supporting global meat industry that is akin to torture on animals
5. Vehicle engineering -- capable of making vehicles that can kill MILLIONS of square meters of forest.
Our technology allows us to live comfortable lives but ONLY at the EXPENSE of millions of nonhuman animals (wild and domestic) that are KILLED every single day for our comforts. We dish out untold violence every second for our advancements.
As much as I'd like to agree with your sentiment, it is a rather flawed argument to state that if some elements of a set have a property, that all elements must have that property.
There are scientific breakthroughs such as the proof of Fermat's theorem [1], for which I find it very hard to envision a way in which it will kill our planet.
[1] https://en.m.wikipedia.org/wiki/Wiles%27s_proof_of_Fermat%27...
There are scientific breakthroughs such as the proof of Fermat's theorem [1], for which I find it very hard to envision a way in which it will kill our planet.
[1] https://en.m.wikipedia.org/wiki/Wiles%27s_proof_of_Fermat%27...
None of that has anything to do with your original complaint of AI models. It makes it look like you don't actually have any argument against AI development, but it is just another thing to rail against.
I have many specific arguments against AI:
1. It will take away jobs from creatives
2. It will reduce the need for other human beings
3. It will be used by criminals for identity theft...
1. It will take away jobs from creatives
2. It will reduce the need for other human beings
3. It will be used by criminals for identity theft...
> 1. It will take away jobs from creatives
"In the 17th century nearly all Europe experienced revolts of the workpeople against the ribbonloom, a machine for weaving ribbons and trimmings, called in Germany Bandmühle, Schnurmühle, and Mühlenstuhl. These machines were invented in Germany. Abbé Lancellotti, in a work that appeared in Venice in 1636, but which was written in 1579, says as follows:
Anthony Müller of Danzig saw about 50 years ago in that town, a very ingenious machine, which weaves 4 to 6 pieces at once. But the Mayor being apprehensive that this invention might throw a large number of workmen on the streets, caused the inventor to be secretly strangled or drowned"
-- "Section 5: The Strife Between Workman and Machine", Das Kapital
"In the 17th century nearly all Europe experienced revolts of the workpeople against the ribbonloom, a machine for weaving ribbons and trimmings, called in Germany Bandmühle, Schnurmühle, and Mühlenstuhl. These machines were invented in Germany. Abbé Lancellotti, in a work that appeared in Venice in 1636, but which was written in 1579, says as follows:
Anthony Müller of Danzig saw about 50 years ago in that town, a very ingenious machine, which weaves 4 to 6 pieces at once. But the Mayor being apprehensive that this invention might throw a large number of workmen on the streets, caused the inventor to be secretly strangled or drowned"
-- "Section 5: The Strife Between Workman and Machine", Das Kapital
Well, I don't care about nonhuman animals.
And being sincere, I don't care about 99% of human animals.
I cry for the deaths of my loved ones, and forget the rest. It is the only way to remain sane.
And being sincere, I don't care about 99% of human animals.
I cry for the deaths of my loved ones, and forget the rest. It is the only way to remain sane.
[deleted]
[deleted]
Are you a boomer?
> 3. Concrete engineering -- building cities that move people away from a connection with nature
This is 70s environmentalist thinking and it's terrible for the environment. Cities are good for it, because when you infill in cities you aren't touching the rest of the land.
America is unusually good about this. Most countries don't have our national park system.
> 3. Concrete engineering -- building cities that move people away from a connection with nature
This is 70s environmentalist thinking and it's terrible for the environment. Cities are good for it, because when you infill in cities you aren't touching the rest of the land.
America is unusually good about this. Most countries don't have our national park system.
It might be dangerous, but how is it useless? There is also an enormous positive gain to be had in terms of autonomous discovery, science progress and so on. And prisoner's dilemma means "we" will not stop pursuing this, even of there is a net negative benefit to humanity overall.
Yes, I believe there is a net negative benefit.
I do not think there is an enormous positive gain. Science progress in my opinion has gone too far in many respects and we should not be able to generate science so easily because we will not have the wisdom to use it. Although I love science, it's use is far too lackadaisical now -- see for example the climate crisis and efficient resource extraction of fossil fuels.
Then we can imagine things like life-extending drugs, immunity from all disease (bad because it implies overpopulation), methods to generate oxygen (bad because it will mean we won't value the natural ecosystem as much), etc.
Smartphones, computer technology (human-isolating and community-destroying), etc. Pretty much all modern inventions are making life worse.
I do not think there is an enormous positive gain. Science progress in my opinion has gone too far in many respects and we should not be able to generate science so easily because we will not have the wisdom to use it. Although I love science, it's use is far too lackadaisical now -- see for example the climate crisis and efficient resource extraction of fossil fuels.
Then we can imagine things like life-extending drugs, immunity from all disease (bad because it implies overpopulation), methods to generate oxygen (bad because it will mean we won't value the natural ecosystem as much), etc.
Smartphones, computer technology (human-isolating and community-destroying), etc. Pretty much all modern inventions are making life worse.
This reads like a combination of pessimism and nostalgia has allowed you to cherry pick the worst of now and the best of the past and conince yourself it is net bad.
I see all of these negatives, and agree it is clear there is much that is bad in the world (there always has been) and much to be improved (ditto), including new problems created by science, technology, and human greed & selfishness. But I also see many positives and benefits, I see many problems that used to exist and don't anymore, and I see much evidence that we continue on average, to make progress. When I look to the past I do not wish I lived there.
Science have many times saved my friends and families' lives or enabled them to live more fully after an accident or sickness
Science made climate change possible but greed, raw capitalism, etc. put us on the path there. Science makes it possible to find a way out, slow or even reverse it eventually, and mitigate the worst results.
Populations are shrinking in developed countries despite people living longer.
Computers and the internet created and connected many new communities, too. The negative effects are again often the result of unbridled capitalism. Young people especially are fighting back against the negative effects and taking control, building technology to manage those hamrful effects.
etc…
I believe both that:
(a) it is not possible to stop humans acquiring and using knowledge, advancing science, developing technology, and evolving culturally and socially; and
(b) we shouldn't want to, even though it means we will create new problems and lose (or consign to a smaller "museum exhibit" role) some aspects of the culture and societies we have today, because on balance we will solve more and bigger problems than we create and build better and more interesting cultures and societies than those of the past.
It's not without existential risks, but those are overstated or analysed pseudoscientifically at best, by many of those (and certainly the loudest) who worry about them.
I see all of these negatives, and agree it is clear there is much that is bad in the world (there always has been) and much to be improved (ditto), including new problems created by science, technology, and human greed & selfishness. But I also see many positives and benefits, I see many problems that used to exist and don't anymore, and I see much evidence that we continue on average, to make progress. When I look to the past I do not wish I lived there.
Science have many times saved my friends and families' lives or enabled them to live more fully after an accident or sickness
Science made climate change possible but greed, raw capitalism, etc. put us on the path there. Science makes it possible to find a way out, slow or even reverse it eventually, and mitigate the worst results.
Populations are shrinking in developed countries despite people living longer.
Computers and the internet created and connected many new communities, too. The negative effects are again often the result of unbridled capitalism. Young people especially are fighting back against the negative effects and taking control, building technology to manage those hamrful effects.
etc…
I believe both that:
(a) it is not possible to stop humans acquiring and using knowledge, advancing science, developing technology, and evolving culturally and socially; and
(b) we shouldn't want to, even though it means we will create new problems and lose (or consign to a smaller "museum exhibit" role) some aspects of the culture and societies we have today, because on balance we will solve more and bigger problems than we create and build better and more interesting cultures and societies than those of the past.
It's not without existential risks, but those are overstated or analysed pseudoscientifically at best, by many of those (and certainly the loudest) who worry about them.
> Science progress in my opinion has gone too far in many respects and we should not be able to generate science so easily because we will not have the wisdom to use it.
I think it was always the case. The wisdom actually comes from mistakes too, not from a priori speculations. Especially as all new solutions/technologies have been criticized and judged as dangerous, frivolous, unnecessary, morally wrong etc. Bikes, cars, street lights,social equality and so long.
I think it was always the case. The wisdom actually comes from mistakes too, not from a priori speculations. Especially as all new solutions/technologies have been criticized and judged as dangerous, frivolous, unnecessary, morally wrong etc. Bikes, cars, street lights,social equality and so long.
It sounds like you are a luddite.
"First rule in government spending: why build one when you can have two at twice the price?" - Contact