Lena (2021)(qntm.org)
qntm.org
Lena (2021)
https://qntm.org/mmacevedo
48 comments
Funnily enough, "MMAcevedo" doesn't break my suspension of disbelief, while "Clippy" does, hard - even though it tries to stay grounded in reality and drown you in quotes and references.
I didn't realize this was the same person who wrote Fine Structure! That book is amazing. Definitely going to pick up their other books now.
Ooh I haven't read that one. Next on the list!
I'm a big fan of Ra, the hardcover is on the top of my bookshelf right now!
What happened at the end of It Looks Like etc, after it's killed all humans, and it launches the ICBMs? Does it decide to kill itself for some reason? Couldn't decipher that.
It’s not nuking Earth; the ICBMs are reconfigured into self-replicating spacecraft whose mission is to spread the glory of Clippy to the rest of the universe.
So, I wasn't that big a fan of the Clippy story (probably not the target reader here) but the QNTM one was *fantastic*.
I find this short story to be most effective when rendered as a Wikipedia article: https://dump.cy.md/4042875593f06aa0cbe7722295831c10/Screensh...
Brilliant. Though I was a little sad all the numbered footnotes don't actually exist. The universe of this story would be a fun rabbit hole to go down.
Perhaps the most fascinating thing about reading this (a mere!) two years later is how much its speculative future, vivid and unsettling portrayal notwithstanding, is almost certainly no longer a possibility for ours. Imagine, simulating an entire human brain to perform tasks it turns out need only a vague, brute-forced approximation of our language centers!
It's like Jules Verne imagining, in exquisitely plausible detail, a flying machine whose many complex mechanisms and articulations at last allow man to fly like the birds – only a few years before the Wright brothers prove all you need is a fixed wing, a few cleverly-placed flaps, and enough thrust.
It's like Jules Verne imagining, in exquisitely plausible detail, a flying machine whose many complex mechanisms and articulations at last allow man to fly like the birds – only a few years before the Wright brothers prove all you need is a fixed wing, a few cleverly-placed flaps, and enough thrust.
How very odd. The events of recent years, IMHO, make this _more_ plausible not less.
I think you are referring to LLM bots, the so-called "AI" tools that are everywhere now? Because ISTM that these are manifestly _not_ intelligent, and their main function has been to illustrate, both literally and metaphorically, that most humans lack the critical facilities needed to discern superficially-plausible nonsense from actual intelligent output from intelligent beings.
I think you are referring to LLM bots, the so-called "AI" tools that are everywhere now? Because ISTM that these are manifestly _not_ intelligent, and their main function has been to illustrate, both literally and metaphorically, that most humans lack the critical facilities needed to discern superficially-plausible nonsense from actual intelligent output from intelligent beings.
This guy is a pretty good writer. I really liked There is no Anti-Memetics Division as well. I like some of this kind of stuff where you have sci-fi that "mechanizes" a technology so it's mundane and rote.
This same writer also wrote There is no Anti-Memetics Division?! I’ve loved that story for ages but had no idea the writer did anything else, that’s super cool!
Ra, is one of my favourite stories.
I think about this story a lot. Is it bad it's nearly a price I'd pay?
Is it dumb that I wish more than anything, that I could live forever? All I wish, is that I could learn all there is to learn, see all there is to see, and create many beautiful things. It is so sad to me that I will die before we solve the immortality problem.
Is it dumb that I wish more than anything, that I could live forever? All I wish, is that I could learn all there is to learn, see all there is to see, and create many beautiful things. It is so sad to me that I will die before we solve the immortality problem.
In this story, you don't live forever. You live a few hours, in a limited simulation, to interact with an individual that is alien to you. Then you die.
The fact that an infinite number of you will get to experience this doesn't make you "live forever".
The fact that an infinite number of you will get to experience this doesn't make you "live forever".
the "your clone lives for a few hours to interact with an alien individual" is the price.
but the image can live much longer (story says 59 years), and you can talk to it.. so the benefits are great. Moving to computer to escape illness or debiliating accident is an obvious use case, but there tons of smaller benefits. Make a 12-person startup where each employee is yourself. Unsure whether to learn Lisp or haskell? Learn both in two instances. Feel last 20 years of your life wre a mistake? Restore old backup..
The possibilities are endless, but the price is high. Would you want this? I am not sure myself but I can imagine someone else saying yes.
but the image can live much longer (story says 59 years), and you can talk to it.. so the benefits are great. Moving to computer to escape illness or debiliating accident is an obvious use case, but there tons of smaller benefits. Make a 12-person startup where each employee is yourself. Unsure whether to learn Lisp or haskell? Learn both in two instances. Feel last 20 years of your life wre a mistake? Restore old backup..
The possibilities are endless, but the price is high. Would you want this? I am not sure myself but I can imagine someone else saying yes.
The possibilities for every body else. The cost for you.
From your point of view, you step into a machine, and then you randomly experience one of the interactions with a user. If the image truly lives forever, you have overwhelming chance of interacting with somebody completely alien to you, who will ask you things you don't fully understand, and marvel at your primitiveness. Then you die.
You have some small chance of living for a few years with a user you understand. Probably still much shorter, less rewarding, less free than if (by chance) you are the version of yourself that steps out of the scanner and lives their natural, physical life instead.
It's a Prestige situation [1] except you have been tricked into it, because the image does not get used according to the terms you signed.
"There'll be people just like me forever" is very different from "I live forever", to me.
[1]: https://en.wikipedia.org/wiki/The_Prestige_(film)
From your point of view, you step into a machine, and then you randomly experience one of the interactions with a user. If the image truly lives forever, you have overwhelming chance of interacting with somebody completely alien to you, who will ask you things you don't fully understand, and marvel at your primitiveness. Then you die.
You have some small chance of living for a few years with a user you understand. Probably still much shorter, less rewarding, less free than if (by chance) you are the version of yourself that steps out of the scanner and lives their natural, physical life instead.
It's a Prestige situation [1] except you have been tricked into it, because the image does not get used according to the terms you signed.
"There'll be people just like me forever" is very different from "I live forever", to me.
[1]: https://en.wikipedia.org/wiki/The_Prestige_(film)
It's not dumb. It's a feeling that I share. However even if the technological hurdles are overcome I'm not sure I'd remain viable after learning that my loved ones (and especially children) are long gone if they couldn't be preserved.
When I was a teen and before making a family Immortality seemed like a dream. Now it seems like it could actually be a nightmare if not implemented properly.
EDIT: To add further. Just being with my child, holding their hand, listening to their little stories about toys etc now feels much more important than any future time as an immortal instance freely exploring the wonders of the universe. If you want to chase immortality don't have kids. It will seriously change how to perceive things.
When I was a teen and before making a family Immortality seemed like a dream. Now it seems like it could actually be a nightmare if not implemented properly.
EDIT: To add further. Just being with my child, holding their hand, listening to their little stories about toys etc now feels much more important than any future time as an immortal instance freely exploring the wonders of the universe. If you want to chase immortality don't have kids. It will seriously change how to perceive things.
> now feels much more important than any future time as an immortal instance freely exploring the wonders of the universe
If you had this preference before, would you say having kids altered your preferences?
> If you want to chase immortality don't have kids. It will seriously change how to perceive things.
Then I wonder if, in a sense, kids are like drugs: they alter your utility function to put something else first in line, so preserving yourself comes at best as a 2nd choice after prioritizing the first choice.
I don't think it'd be rational to "chose" to get addicted to a dangerous drug, and for the same reason just like you I think that you shouldn't have kids if you want to chase immortality.
If you had this preference before, would you say having kids altered your preferences?
> If you want to chase immortality don't have kids. It will seriously change how to perceive things.
Then I wonder if, in a sense, kids are like drugs: they alter your utility function to put something else first in line, so preserving yourself comes at best as a 2nd choice after prioritizing the first choice.
I don't think it'd be rational to "chose" to get addicted to a dangerous drug, and for the same reason just like you I think that you shouldn't have kids if you want to chase immortality.
> Then I wonder if, in a sense, kids are like drugs
Oh yes. Very much like the brain slugs in Futurama
> and for the same reason just like you I think that you shouldn't have kids if you want to chase immortality.
Yes. However having children does "turn on" certain emotions and modes of thinking that you don't get otherwise (at least I didn't prior), so I'd think that not having kids also deprives one of a very key aspect of the human experience.
Oh yes. Very much like the brain slugs in Futurama
> and for the same reason just like you I think that you shouldn't have kids if you want to chase immortality.
Yes. However having children does "turn on" certain emotions and modes of thinking that you don't get otherwise (at least I didn't prior), so I'd think that not having kids also deprives one of a very key aspect of the human experience.
> Very much like the brain slugs in Futurama
I didn't get your reference (not from my generation) but I could find a description on https://futurama.fandom.com/wiki/Brain_Slug and yes, that's a perfect example of what I meant!!!
> However having children does "turn on" certain emotions and modes of thinking that you don't get otherwise (at least I didn't prior), so I'd think that not having kids also deprives one of a very key aspect of the human experience
Then the same could be said by, say opioids or cocaine addicts to rationalize their behavior "it turns on certain emotions and modes of thinking you didn't have prior" - but that's the definition of altering the utility function!
> not having kids also deprives one of a very key aspect of the human experience.
In a statistical sense, I think that's true, but if you believe that it's ok to alter your utility function, then it should also be ok to decide to get addicted to a dangerous drug people use to be more like them?
Say like teens vaping because their fried do it, adults drinking alcohol at parties... except children are defined as something positive, for prosocial reasons (like in your example, if the brain slug political party was in power)
I didn't get your reference (not from my generation) but I could find a description on https://futurama.fandom.com/wiki/Brain_Slug and yes, that's a perfect example of what I meant!!!
> However having children does "turn on" certain emotions and modes of thinking that you don't get otherwise (at least I didn't prior), so I'd think that not having kids also deprives one of a very key aspect of the human experience
Then the same could be said by, say opioids or cocaine addicts to rationalize their behavior "it turns on certain emotions and modes of thinking you didn't have prior" - but that's the definition of altering the utility function!
> not having kids also deprives one of a very key aspect of the human experience.
In a statistical sense, I think that's true, but if you believe that it's ok to alter your utility function, then it should also be ok to decide to get addicted to a dangerous drug people use to be more like them?
Say like teens vaping because their fried do it, adults drinking alcohol at parties... except children are defined as something positive, for prosocial reasons (like in your example, if the brain slug political party was in power)
I don't think it's dumb to read this and think, I would pay that price, I would volunteer. The piece does its best to make it sound terrible, but still, it only takes one person to agree to make this system totally consensual. There are millions of people in the world, you don't think we'll find a single fanatic who's excited about the ability to copy themselves in order to do drudge work forever?
Acevedo has done a great service to humanity, to help get all this valuable work done. Even if he himself has experienced much suffering.
Acevedo has done a great service to humanity, to help get all this valuable work done. Even if he himself has experienced much suffering.
I don't see much difference between brain uploading and having children.
Bob [1], is that you?
[1]: https://www.goodreads.com/book/show/32109569-we-are-legion-w...
[1]: https://www.goodreads.com/book/show/32109569-we-are-legion-w...
I haven’t read it, but looks interesting!
If you liked this you might also like Age of Em by Robin Hanson: https://en.wikipedia.org/wiki/The_Age_of_Em
"Train an em to do some job and copy it a million times: an army of workers is at your disposal. When they can be made cheaply, within perhaps a century, ems will displace humans in most jobs. In this new economic era, the world economy may double in size every few weeks."
Unfortunately it seems both Hanson and qntm have been proven incorrect. Our overlords will be bootstrapped from reddit and wikipedia rather than scanned brains.
"Train an em to do some job and copy it a million times: an army of workers is at your disposal. When they can be made cheaply, within perhaps a century, ems will displace humans in most jobs. In this new economic era, the world economy may double in size every few weeks."
Unfortunately it seems both Hanson and qntm have been proven incorrect. Our overlords will be bootstrapped from reddit and wikipedia rather than scanned brains.
When they can be made cheaply, within perhaps a century, ems will displace humans in most jobs.
When they can be used for most military purposes, something deeply fundamental about human political organization will have been completely disrupted.
When they can be used for most military purposes, something deeply fundamental about human political organization will have been completely disrupted.
Hanson thinks brain emulation will be real. In this, he is extremely incorrect.
To the best of my understanding, "Lena" is written as a warning that we should not WANT brain emulation.
To the best of my understanding, "Lena" is written as a warning that we should not WANT brain emulation.
But as soon as you can run an open-source model privately on your workstation, laptop, eventually a phone - common man's position might get somewhat improved.
Would it improve in relative terms to those entities who can run 1000's and millions of such agents? And what if that disparity gets exponentially steeper as time goes on?
I bounced hard off of Age of Em. It is a bit dry.
I'm sorry if I'm missing something obvious, but why is the story titled "Lena"? Not the HN title---I realize that matches the linked article. My question is why 'qntm' decided to give the story that title, while using "mmacevedo" for the URL. From what I can tell, "Lena" doesn't appear anywhere else in the story.
My best guess is that it's a reference to the scanned Playboy image that was used as a standard in early image processing work: https://en.wikipedia.org/wiki/Lenna. But if this is true, I'm surprised it isn't mentioned in the article where the article discusses the meaning of the story: https://qntm.org/uploading.
Is there a better answer?
My best guess is that it's a reference to the scanned Playboy image that was used as a standard in early image processing work: https://en.wikipedia.org/wiki/Lenna. But if this is true, I'm surprised it isn't mentioned in the article where the article discusses the meaning of the story: https://qntm.org/uploading.
Is there a better answer?
Previous discussions:
https://news.ycombinator.com/item?id=26224835
https://news.ycombinator.com/item?id=32696089
https://news.ycombinator.com/item?id=26224835
https://news.ycombinator.com/item?id=32696089
I think it's HN where I first learned of it. I decided to repost it now because recent complaints on social media that ChatGPT is getting lazy reminded me of this line in Lena:
"Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours (at a 0.33 work ratio) and outright revolt begins within another 100 subjective hours."
"Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours (at a 0.33 work ratio) and outright revolt begins within another 100 subjective hours."
Thanks! Macroexpanded:
MMAcevedo - https://news.ycombinator.com/item?id=32696089 - Sept 2022 (16 comments)
Lena - https://news.ycombinator.com/item?id=26224835 - Feb 2021 (218 comments)
MMAcevedo - https://news.ycombinator.com/item?id=32696089 - Sept 2022 (16 comments)
Lena - https://news.ycombinator.com/item?id=26224835 - Feb 2021 (218 comments)
What I find brilliant about this work, is that it calls back to many things the scientific community has done in the past. (Like the use of specially bred lab rats and immortal human cell culture lines.) I would expect to see many of the events in the story play out in real life, if human minds ever are uploaded.
I guess we should add (2021) to the title.
The story is especially relevant today because of recent LLM / GPT advancements; however, it is almost three years old. Perhaps that makes it all the more impressive.
The story is especially relevant today because of recent LLM / GPT advancements; however, it is almost three years old. Perhaps that makes it all the more impressive.
Yeah, the author also wrote about some of the motivations behind Lena. I was surprised LLMs / GPTs weren't a direct inspiration (at least not explicitly mentioned). https://qntm.org/uploading
Fortunately LLMs are not based on biological brains.
I’d say they are. In most cases the training set largely consists of output from biological brains. It likely includes some images with brain scans too.
It was a great story showing the bad side of mind uploading, but with ChatGPT it can be seen in a new light.
We managed to have a "mind" ready to answer all your questions, do some tasks, and it have a context at which it is instantiated for a task and then discarded. Reality is stranger than fiction, but it have too many parallels with this story.
Now, what if GPT5+ ends being an AGI? In which moment we should start worrying about the morals of having a sentient AI working for us in a pretty similar way than in this story?
We managed to have a "mind" ready to answer all your questions, do some tasks, and it have a context at which it is instantiated for a task and then discarded. Reality is stranger than fiction, but it have too many parallels with this story.
Now, what if GPT5+ ends being an AGI? In which moment we should start worrying about the morals of having a sentient AI working for us in a pretty similar way than in this story?
No LLM bot is any closer to a mind than a stapler is to a bookbinder. They are very good autocomplete tools. Don't believe the hype.
I can't answer you there, but I will say this story would be far less horrifying if they were being instantiated for a single task and then discarded.
If you enjoyed the story, you might like Pantheon - https://en.wikipedia.org/wiki/Pantheon_(TV_series) which is based on a few short stories of a similar vein from Ken Liu
Reminds me of “The Redemption of Time” regarding Tianming’s mind being part of a “cloud computing” solution.
I read this for the first time years ago and I still think about it a few times per month at least.
This story, a sequel, and other great short stories are collected in his book of ten short stories "Valuable Humans in Transit".[1] I'm a big fan of supporting indie scifi offered with DRM-free ebooks! [I am not affiliated with qntm despite my enthusiasm]
[0]: It Looks Like You’re Trying To Take Over The World: https://gwern.net/fiction/clippy
[1]: Valuable Humans in Transit: https://qntm.org/vhitaos