The good delusion: has effective altruism broken bad?(economist.com)
economist.com
The good delusion: has effective altruism broken bad?
https://www.economist.com/1843/2022/11/15/the-good-delusion-has-effective-altruism-broken-bad
617 comments
https://web.archive.org/web/20221115182258/https://www.econo...
My "Effective Altruism Is Not A Cult" t-shirt is prompting a lot of questions already answered by my shirt.
I've dug into this stuff over the last couple days, because I had previously understood Effective Altruism to be something akin to like a pledge drive for Charity Navigator's best-reviewed charities or something, and it turns out it's nothing of the sort. In fact, if you read some of the commentary from EA insiders about Sam Bankman-Fried (these come up in stories where EA distances itself from SBF, or claims they knew he was up to no good all along), it's hard to shake the comparisons to Scientology: they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
To that, add that some of the ideas are clearly dementing. Take Scott Aaronson's recent post on SBF, which is shot through with EA-think; it includes this:
And a traditional investor who made billions on successful gambles, or arbitrage, or creating liquidity, then gave virtually all of it away to effective charities, would seem, on net, way ahead of most of us morally.
This is logic straight out of the 16th century Medici papacy of Leo X. Maybe it's some kind of horrible drug interaction with Rationalism, or maybe EA is fundamentally Rationalist? Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much, and if you can't recognize that, something has gone terribly wrong in your thinking. Over and over I keep coming across EA arguments that could be used to justify almost literally any behavior, so long as you can tell yourself a story about having a long-term positive goal.
Also: tens of millions in donations to "AI safety" organizations. Yes, to answer the Economist; this movement is irretrievable.
I've dug into this stuff over the last couple days, because I had previously understood Effective Altruism to be something akin to like a pledge drive for Charity Navigator's best-reviewed charities or something, and it turns out it's nothing of the sort. In fact, if you read some of the commentary from EA insiders about Sam Bankman-Fried (these come up in stories where EA distances itself from SBF, or claims they knew he was up to no good all along), it's hard to shake the comparisons to Scientology: they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
To that, add that some of the ideas are clearly dementing. Take Scott Aaronson's recent post on SBF, which is shot through with EA-think; it includes this:
And a traditional investor who made billions on successful gambles, or arbitrage, or creating liquidity, then gave virtually all of it away to effective charities, would seem, on net, way ahead of most of us morally.
This is logic straight out of the 16th century Medici papacy of Leo X. Maybe it's some kind of horrible drug interaction with Rationalism, or maybe EA is fundamentally Rationalist? Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much, and if you can't recognize that, something has gone terribly wrong in your thinking. Over and over I keep coming across EA arguments that could be used to justify almost literally any behavior, so long as you can tell yourself a story about having a long-term positive goal.
Also: tens of millions in donations to "AI safety" organizations. Yes, to answer the Economist; this movement is irretrievable.
"Maybe it's some kind of horrible drug interaction with Rationalism, or maybe EA is fundamentally Rationalist?"
I have always associated EA with the rationalist movement, since I came across it via reading rationalist stuff.
If you define EA as "figuring out how to deploy a given set of resources so as to do the most good," your conclusions are going to depend on your value system: how do you define and measure good?
So when a bunch of rationalists who believe in consequentialism do that, they're going to maximize their value system and end up looking really weird to everyone else who doesn't share their values. You'd have the same outcome in a different flavor if a bunch of Christians did it.
I disagree with a bunch of the underlying values in the EA movement but I really admire the mindset that actually tries to solve the problem of "how can we be most effective with our giving?" instead of what I personally do, which is get overwhelmed by guilt and confusion, stop thinking about helping other people, and go back to my default of living my life mostly thinking about how to maximize my own comfort and preferences.
I have always associated EA with the rationalist movement, since I came across it via reading rationalist stuff.
If you define EA as "figuring out how to deploy a given set of resources so as to do the most good," your conclusions are going to depend on your value system: how do you define and measure good?
So when a bunch of rationalists who believe in consequentialism do that, they're going to maximize their value system and end up looking really weird to everyone else who doesn't share their values. You'd have the same outcome in a different flavor if a bunch of Christians did it.
I disagree with a bunch of the underlying values in the EA movement but I really admire the mindset that actually tries to solve the problem of "how can we be most effective with our giving?" instead of what I personally do, which is get overwhelmed by guilt and confusion, stop thinking about helping other people, and go back to my default of living my life mostly thinking about how to maximize my own comfort and preferences.
"How can we be most effective with our giving" is a good thought. It's what I thought EA was, like, two weeks ago.
"The movement can literally compute the optimal pattern of giving, such that to be most effective with our giving means donating money to people to ride around on bicycles thinking about how to prevent future AGIs from killing us" is not a good thought, but it turns out to be a description of what EA is like; some of this is on their own website (another thing I've learned: beware any movement that calls its documentation "sequences").
"The moral urgency of giving is such that a reasonable person should seek to maximize their income" is a bad thought. It leads to dark places. "Earning to give" is a reasonable idea: it is probably more effective for people to excel at their jobs and donate the proceeds than for them to phone bank or canvass. But Rationalists can't seem to help falling into these asymptote traps, so that "donating a portion of ones income" becomes "pursue billions in wealth, and clear a path for other billionaires, to enable concentrated giving". That's a demented idea.
The problems here aren't abstract. They just lead to Scott Aaronson withholding moral judgement on Sam Bankman-Fried, strictly because of the logic of the preceding paragraph.
"The movement can literally compute the optimal pattern of giving, such that to be most effective with our giving means donating money to people to ride around on bicycles thinking about how to prevent future AGIs from killing us" is not a good thought, but it turns out to be a description of what EA is like; some of this is on their own website (another thing I've learned: beware any movement that calls its documentation "sequences").
"The moral urgency of giving is such that a reasonable person should seek to maximize their income" is a bad thought. It leads to dark places. "Earning to give" is a reasonable idea: it is probably more effective for people to excel at their jobs and donate the proceeds than for them to phone bank or canvass. But Rationalists can't seem to help falling into these asymptote traps, so that "donating a portion of ones income" becomes "pursue billions in wealth, and clear a path for other billionaires, to enable concentrated giving". That's a demented idea.
The problems here aren't abstract. They just lead to Scott Aaronson withholding moral judgement on Sam Bankman-Fried, strictly because of the logic of the preceding paragraph.
Rationalists brought us Roko’s Basilisk after all. A lot of the leaders don’t seem to have basic sanity checks in their brains that stop their ideas from getting wildly out of control, which can lead to stupid shit like the Basilisk on a small scale but end up justifying fraud on a much larger scale. It’s like fundamentalist religion
> Rationalists brought us Roko’s Basilisk after all.
This reminds me of the Zen story about two monks and a woman (one helped her cross a river, and the other couldn't stop thinking about her for the whole day).
The "Roko's Basilisk" was an edgy comment that was written on a rationalist web forum, and then deleted by the admin. A decade later, some people outside the rationalist community still keep discussing it, which is the only reason why anyone cares.
> A lot of the leaders don’t seem to have basic sanity checks in their brains that stop their ideas from getting wildly out of control
Only in this story, the admin was the one who deleted that comment, not the one who wrote it. Still gets blamed for it.
This reminds me of the Zen story about two monks and a woman (one helped her cross a river, and the other couldn't stop thinking about her for the whole day).
The "Roko's Basilisk" was an edgy comment that was written on a rationalist web forum, and then deleted by the admin. A decade later, some people outside the rationalist community still keep discussing it, which is the only reason why anyone cares.
> A lot of the leaders don’t seem to have basic sanity checks in their brains that stop their ideas from getting wildly out of control
Only in this story, the admin was the one who deleted that comment, not the one who wrote it. Still gets blamed for it.
Eh, according to https://en.m.wikipedia.org/wiki/Roko's_basilisk, the admin gave credence to the stupidity by initially calling Roll’s basilisk “a genuinely dangerous thought".
Later they all backtracked and pretended they had never agreed with any of it.
Later they all backtracked and pretended they had never agreed with any of it.
One of the problems I observe in forum posts from these communities is that it is fun and cool to think differently than others. "What if fighting poverty now is actually bad and preventing the AI apocalypse later is actually better?" Then you get to feel superior to everybody, even those goody-two-shoes people down the street that donate a lot to the food bank. You know the secrets. It is the same kind of thinking that makes people think that Roko's Basilisk is cool or frightening or meaningful. It makes you feel like you have secret knowledge that others do not.
But... because we tend to interrogate our second option less than our first option people end up throwing their EA money at fucking ridiculous things while there are people out there dying in childbirth because their local hospital doesn't have consistent electricity.
"Yeah, I just have a different value function than everybody else" does not absolve people from the criticism that their value function is stupid.
But... because we tend to interrogate our second option less than our first option people end up throwing their EA money at fucking ridiculous things while there are people out there dying in childbirth because their local hospital doesn't have consistent electricity.
"Yeah, I just have a different value function than everybody else" does not absolve people from the criticism that their value function is stupid.
Like much tech wealth wisdom, it's rationalization of their megalomania, their self-serving desire for power.
Humans naturally desire power to varying degrees, but we have other desires and we have other things that limit us (e.g., our consciences, or Adam Smith's 'impartial spectator'). With enough real power, you can ignore these things and others will not only fail to check you, but will encourage you. Power corrupts.
Humans naturally desire power to varying degrees, but we have other desires and we have other things that limit us (e.g., our consciences, or Adam Smith's 'impartial spectator'). With enough real power, you can ignore these things and others will not only fail to check you, but will encourage you. Power corrupts.
If you aren't aware already: https://rationalwiki.org/wiki/Essay:Second-option_bias
"If you define EA as "figuring out how to deploy a given set of resources so as to do the most good," your conclusions are going to depend on your value system: how do you define and measure good?"
I think more subtly it also depends on your calculation methodology. Rationalists talk a lot about the value of numbers for their Bayesian analyses and their "shut up and calculate" motto, but to my mind they overestimate the accuracy they have in their calculations by a lot. Of course I want to go with the 99% likely-to-help choice over the 25%-likely-to-help choice, but along with the fact my value system is indeed much different than theirs (in some important ways, infinitely so), I think all their Bayesian calculations need error bars in them. I think some rationalists would claim their probabilities have that error built into them, and I disagree. I think 75% likely is very different from 75% +/- 20% [1] when you start feeding that number through a chain of other calculations. A chain of calculations that nominally results in a 99.99% probability is a lot less impressive if a more honest calculation that propagates error bars correctly results in "somewhere between 2% and 99.9999%". It becomes more clear that the 99.99% is a very suspect number, even if it is nominally the middle of the distribution, especially given the number of times out in the real world the "highly improbable" ends up happening when it turns out that the assessment of the outcome being "highly improbable" was wrong. The probability distributions in the real world are highly pathological much more often than a naive analysis would suggest.
When analyzing the virtues of stealing billions to donate it to an effective charity, you better not just think that the value of the charity is higher than the cost of the loss of the money, you better be really sure your calculation is correct. I think the rationalists could argue amoungst themselves endlessly on the first point, but I do not believe they have any way whatsoever to establish their second point to my satisfaction, which renders the entire conversation moot, when there is no way they could ever convince me that their calculation is correct regardless of what it is. Even when my value system does in fact line up with theirs, and there is non-trivial overlap, I disagree with their calculation methodologies quite substantially as I think they consistently act with much more confidence then is justified.
Against that, I'm comfortable with the societally-evolved (evolution is important too, right, rationalists?) standard that in addition to the first-order effects of theft, one much consider the second-order effects of the dangers of normalizing outright theft because of the fact that someone else thinks they could use the resources better. Even if that is locally true, if everyone globally acts like that, global utility will plummet because everybody always thinks everybody else could spend their money better and the resulting chaos will not be helpful. (The Kantian imperative is completely inadequate as a theoretical foundation of philosophy but I do find it a useful practical tool, as long as one is aware to not place too much confidence in an extended analysis using it... the theme of this post comes up yet again.) The world is not improved by turning everyone into a central authority over all resource allocation in the world.
[1]: I know real error bars in probability space would never look like this, but mathematical accuracy would distract from my point here.
I think more subtly it also depends on your calculation methodology. Rationalists talk a lot about the value of numbers for their Bayesian analyses and their "shut up and calculate" motto, but to my mind they overestimate the accuracy they have in their calculations by a lot. Of course I want to go with the 99% likely-to-help choice over the 25%-likely-to-help choice, but along with the fact my value system is indeed much different than theirs (in some important ways, infinitely so), I think all their Bayesian calculations need error bars in them. I think some rationalists would claim their probabilities have that error built into them, and I disagree. I think 75% likely is very different from 75% +/- 20% [1] when you start feeding that number through a chain of other calculations. A chain of calculations that nominally results in a 99.99% probability is a lot less impressive if a more honest calculation that propagates error bars correctly results in "somewhere between 2% and 99.9999%". It becomes more clear that the 99.99% is a very suspect number, even if it is nominally the middle of the distribution, especially given the number of times out in the real world the "highly improbable" ends up happening when it turns out that the assessment of the outcome being "highly improbable" was wrong. The probability distributions in the real world are highly pathological much more often than a naive analysis would suggest.
When analyzing the virtues of stealing billions to donate it to an effective charity, you better not just think that the value of the charity is higher than the cost of the loss of the money, you better be really sure your calculation is correct. I think the rationalists could argue amoungst themselves endlessly on the first point, but I do not believe they have any way whatsoever to establish their second point to my satisfaction, which renders the entire conversation moot, when there is no way they could ever convince me that their calculation is correct regardless of what it is. Even when my value system does in fact line up with theirs, and there is non-trivial overlap, I disagree with their calculation methodologies quite substantially as I think they consistently act with much more confidence then is justified.
Against that, I'm comfortable with the societally-evolved (evolution is important too, right, rationalists?) standard that in addition to the first-order effects of theft, one much consider the second-order effects of the dangers of normalizing outright theft because of the fact that someone else thinks they could use the resources better. Even if that is locally true, if everyone globally acts like that, global utility will plummet because everybody always thinks everybody else could spend their money better and the resulting chaos will not be helpful. (The Kantian imperative is completely inadequate as a theoretical foundation of philosophy but I do find it a useful practical tool, as long as one is aware to not place too much confidence in an extended analysis using it... the theme of this post comes up yet again.) The world is not improved by turning everyone into a central authority over all resource allocation in the world.
[1]: I know real error bars in probability space would never look like this, but mathematical accuracy would distract from my point here.
I think this type of analysis is what a lot of EA is, oddly, missing. I also think the level of uncertainty necessarily associated with EA-style consequentialism necessitates a sort of pragmatic deontology.
> your conclusions are going to depend on your value system
That's a logical extreme, not reality; it's a philosophy class debate, used to rationalize everything and not applicable to life, where survival, freedom, prosperity, and happiness depend on our abilty to judge and discern - especially between values.
That's a logical extreme, not reality; it's a philosophy class debate, used to rationalize everything and not applicable to life, where survival, freedom, prosperity, and happiness depend on our abilty to judge and discern - especially between values.
> If you define EA as "figuring out how to deploy a given set of resources so as to do the most good," your conclusions are going to depend on your value system: how do you define and measure good?
This is exactly it. Because "good" is subjective to the individual, EA without some kind of consensus building reduces to "maximize your agency as an individual in the world, and do what you think is good," which gets uncomfortably close to Ayn Rand territory for me.
That's not just speculative either - if you read some of Caroline Ellison (Ex-CEO of Alameda Capital and an EA disciple) purported tumblr posts, it paints an image of someone who's passionate about ranking the world's issues in descending order of priority, investigating genetic differences between humans from different ethnic backgrounds, and "optimizing the world." We're not there yet, but it's also not a stretch to see EA turn into eugenics for someone with these interests and values.
This is exactly it. Because "good" is subjective to the individual, EA without some kind of consensus building reduces to "maximize your agency as an individual in the world, and do what you think is good," which gets uncomfortably close to Ayn Rand territory for me.
That's not just speculative either - if you read some of Caroline Ellison (Ex-CEO of Alameda Capital and an EA disciple) purported tumblr posts, it paints an image of someone who's passionate about ranking the world's issues in descending order of priority, investigating genetic differences between humans from different ethnic backgrounds, and "optimizing the world." We're not there yet, but it's also not a stretch to see EA turn into eugenics for someone with these interests and values.
> EA without some kind of consensus building reduces to "maximize your agency as an individual in the world....
I agree and would say that to the extent that there is a consensus, it becomes “maximize our agency as a coordinated group of people” – hence the focus on promoting and expanding EA as a movement
I agree and would say that to the extent that there is a consensus, it becomes “maximize our agency as a coordinated group of people” – hence the focus on promoting and expanding EA as a movement
Sorry, scientology? This comes off as pretty bad faith.
Most EAs are people like me who read some essays, setup some recurring donations to GiveWell and call it a day. If that's how being a scientology member goes then it must be milder than I've been led to believe.
Most EAs are people like me who read some essays, setup some recurring donations to GiveWell and call it a day. If that's how being a scientology member goes then it must be milder than I've been led to believe.
I wouldn't be surprised if that's, like, the modal experience people have with the movement. But that, too, is how Scientology works!
I'm reacting to messages circulating about the whisper campaign among EA higher-ups, many of which are explicit about the fact that they didn't warn the public about Sam Bankman-Fried because EA-the-movement would ostracize them, not just from their community but also from their jobs. I'm not making those messages up. Maybe they're completely disconnected from reality? There's several of them, from several different people, though, so it'll take something more compelling than "it is unpleasant to be compared to Scientology" to refute my lying eyes.
EA even has its own Xenu: it's just called AGI.
I'm reacting to messages circulating about the whisper campaign among EA higher-ups, many of which are explicit about the fact that they didn't warn the public about Sam Bankman-Fried because EA-the-movement would ostracize them, not just from their community but also from their jobs. I'm not making those messages up. Maybe they're completely disconnected from reality? There's several of them, from several different people, though, so it'll take something more compelling than "it is unpleasant to be compared to Scientology" to refute my lying eyes.
EA even has its own Xenu: it's just called AGI.
> I'm not making those messages up.
You are also not providing evidence.
You are also not providing evidence.
Nor will I. They're on Twitter, in screenshots. I'm comfortable with the people here who do and don't take me seriously; a perk of commenting here under my own name for over a decade, I guess.
You describe all religions. Read some scripture, perform some simple devotions, and call it a day.
I don’t mean to overstate the religiosity of EA but it’s not structurally different.
I don’t mean to overstate the religiosity of EA but it’s not structurally different.
You can reduce everything to being religion if you do it this way to the point of the comparison being useless.
HN? Religion. Weight lifting? Religion. Dental hygiene? Religion. Voting? Religion. Functional Programming? Religion. Punk? Religion.
HN? Religion. Weight lifting? Religion. Dental hygiene? Religion. Voting? Religion. Functional Programming? Religion. Punk? Religion.
I’m not sure that you know how the “thought leaders” in the upper social tiers of EA are operating. I mean that honestly and not as a slight, because if you do know we can (to the extent that I have the attention span) discuss it if we disagree and productively explore the topic:
There appears to be an upper eschelon, perhaps more of a social group, who are guiding the resources of this movement along philosophical/ideological lines about which they are not being transparent. And when someone outside of that social group verges on exploring those aspects of their philosophical thinking there is an effort to silence or soften the tone of their discourse so it appears less coldly calculating, maybe (per anecdotal reports of people closely tied there) less absurdly extreme and ridiculous.
There appears to be an upper eschelon, perhaps more of a social group, who are guiding the resources of this movement along philosophical/ideological lines about which they are not being transparent. And when someone outside of that social group verges on exploring those aspects of their philosophical thinking there is an effort to silence or soften the tone of their discourse so it appears less coldly calculating, maybe (per anecdotal reports of people closely tied there) less absurdly extreme and ridiculous.
For starters, it would be interesting to know who are those mysterious thought leaders.
Peter Singer, the philosopher whose book inspired the movement? People at GiveWell, the charity evaluation organization? Authors of the Giving What We Can pledge?
Or are these all merely puppets in the hands of the true manipulators hiding in the shadows, whose names no one dares to mention, even anonymously? Spooky...
Peter Singer, the philosopher whose book inspired the movement? People at GiveWell, the charity evaluation organization? Authors of the Giving What We Can pledge?
Or are these all merely puppets in the hands of the true manipulators hiding in the shadows, whose names no one dares to mention, even anonymously? Spooky...
> HN? Religion. Weight lifting? Religion. Dental hygiene? Religion. Voting? Religion. Functional Programming? Religion. Punk? Religion.
Well dental hygiene is definitely not a religion. I don't even floss (use a water pik now but flossing back teeth is difficult, parents never taught me to floss when I was a child). Leaving this aside, I have also noticed people call test-driven development / behavior driven development religions here. Are they using hyperboles?
Well dental hygiene is definitely not a religion. I don't even floss (use a water pik now but flossing back teeth is difficult, parents never taught me to floss when I was a child). Leaving this aside, I have also noticed people call test-driven development / behavior driven development religions here. Are they using hyperboles?
Heh, my first job was a big TDD shop and while I learned many valuable skill - and even still write my tests first at times - I definitely encountered people who were religiously dogmatic about TDD and pairing.
Both of those are useful tools, but doing them 100% of the time is challenging. For me, as a brand new dev, the attitude of "you must always write a test before you write code" was stifling and made it harder to learn things at times.
To quote our noble caveman patron saint, "how grug test what grug not understand domain yet?!"
Both of those are useful tools, but doing them 100% of the time is challenging. For me, as a brand new dev, the attitude of "you must always write a test before you write code" was stifling and made it harder to learn things at times.
To quote our noble caveman patron saint, "how grug test what grug not understand domain yet?!"
Do you even floss?
https://knowyourmeme.com/memes/do-you-even-lift
https://knowyourmeme.com/memes/do-you-even-lift
I don't disagree with you. Whatever you put your hope into, coupled with whatever you believe is true about existence – like even nihilism or solipsism or Darwinism or syncretism, etc. etc.), and however those hopes and beliefs lead to actions that consistently point back to said hopes and beliefs, pretty much reveals what we put our faith into.
Again, not saying I disagree in the slightest. Just reiterating.
Again, not saying I disagree in the slightest. Just reiterating.
Judging the comparisons as useless before making them and evaluating their use? That's a common religious belief and practice that hides this truth:
Belief is an arbitrary choice and a tool.
Belief is an arbitrary choice and a tool.
Little sleight of hand here. I'd arguably agree that things listed in other comments like weight lifting and functional programming and punk culture all share traits with religion, but "religion" isn't what was bad about the original comparison. Charity itself originally came from religion. The comparison was to Scientology, which on balance has all the possible bad traits of a religion and none of the good ones. What you're describing might describe most religions, but not Scientology. You can't practice Scientology at all without bringing in the extreme bullshit elements.
I think it’s important to distinguish GiveWell type stuff from the organization and belief system as a whole. Giving money to malaria nets and prevention is great, and really effective. Weird Ender’s Game navelgazing and crypto scam polycules are stupid and don’t help anyone, they’re much worse than the excesses of normal charities EA is supposed to stand against
> I've dug into this stuff over the last couple days, because I had previously understood Effective Altruism to be something akin to like a pledge drive for Charity Navigator's best-reviewed charities or something, and it turns out it's nothing of the sort.
As someone with multiple EA friends: this is precisely what it is.
> they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
The first is true, the latter two are bullshit.
> Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences
That's a gross mischaracterization. A better metaphor is Robin Hood, someone who steals from the rich to give to the poor. And who is widely regarded as a folk hero.
> tens of millions in donations to "AI safety" organizations.
If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
As someone with multiple EA friends: this is precisely what it is.
> they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
The first is true, the latter two are bullshit.
> Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences
That's a gross mischaracterization. A better metaphor is Robin Hood, someone who steals from the rich to give to the poor. And who is widely regarded as a folk hero.
> tens of millions in donations to "AI safety" organizations.
If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
This whole thread is very stereotypically EA to me -- and why I see it as a sham. Starts off at the top saying how it's all about donating to reasonable charities, but soon devolved into "what about muh AGI". I am literally doing my Ph.D. in robot learning and I can tell you that every single "longtermist" effective altruist I have talked to has been more interested in getting high on their own stash of Sci Fi fantasies rather than interacting with the actual practitioners of the field and understanding what is happening, what are the very real limitations of it, and why their assumptions range from willfully ignorant to often simply maliciously stupid.
It's almost as if the wealthy, the privileged, and the powerful are interested in indulgences while doing things that tickles their fantasies, and as the invisible hand of the market works, there is a class of "philosophers" who are willing to sell it to them.
It's almost as if the wealthy, the privileged, and the powerful are interested in indulgences while doing things that tickles their fantasies, and as the invisible hand of the market works, there is a class of "philosophers" who are willing to sell it to them.
> rather than interacting with the actual practitioners of the field and understanding what is happening, what are the very real limitations of it, and why their assumptions range from willfully ignorant to often simply maliciously stupid.
On the other hand, I've found that "practitioners in the field" often consistently overestimate how complex and different the human mind is from our computers.
Take the recent "LaMDA is sentient" debate. Machine learning researchers consistently and vehemently denied that LaMDA is sentient. They no doubt have a deep and subtle understanding of how LaMDA works, but apparently they didn't realize that we have no mechanistic understanding of what sentience actually is or how it works, so they have absolutely no basis for such a conclusion.
So on the one hand we have these AGI doomsayers that, per your evaluation, don't understand machine learning subtleties, and on the other hand we have some machine learning researchers skeptical of AGI who consistently overestimate human specialness.
At this time the error bars I would assign to each side are wide enough that there's lots of overlap, but given the track record of human specialness falling to algorithmic progress, I'd say some investment in AGI safety is prudent.
On the other hand, I've found that "practitioners in the field" often consistently overestimate how complex and different the human mind is from our computers.
Take the recent "LaMDA is sentient" debate. Machine learning researchers consistently and vehemently denied that LaMDA is sentient. They no doubt have a deep and subtle understanding of how LaMDA works, but apparently they didn't realize that we have no mechanistic understanding of what sentience actually is or how it works, so they have absolutely no basis for such a conclusion.
So on the one hand we have these AGI doomsayers that, per your evaluation, don't understand machine learning subtleties, and on the other hand we have some machine learning researchers skeptical of AGI who consistently overestimate human specialness.
At this time the error bars I would assign to each side are wide enough that there's lots of overlap, but given the track record of human specialness falling to algorithmic progress, I'd say some investment in AGI safety is prudent.
You don't need to have a deep and complex understanding of human sentience to be able to put some lower bounds on what it means to be sentient.
I don't know about sentience, as such; however, I believe there are at least two bare-bones prerequisites to an AI being conscious as we understand the term:
1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli.
2. Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives.
Now, these are probably not sufficient for an AI to be conscious by our understanding of consciousness. But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience.
I don't know about sentience, as such; however, I believe there are at least two bare-bones prerequisites to an AI being conscious as we understand the term:
1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli.
2. Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives.
Now, these are probably not sufficient for an AI to be conscious by our understanding of consciousness. But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience.
> Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli.
Why can't it be conscious merely while the computation is running? You aren't constantly on, you're ostensibly only conscious 16-20 hours every day, while you're not sleeping.
> Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives.
I'm not sure what learning has to do with consciousness, and I'm absolutely certain you don't either. I frankly find it hard to believe that Zen Buddhists are not conscious during their meditations, despite having empty minds and not "constantly updating their mental models of the world-in-effect".
> But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience.
See, this is what I'm talking about. That's a fine personal belief, but that belief does not rise to the level of expert testimony of the facts even if you are a machine learning expert.
People have made all sorts of assumptions about the human mind that they have never questioned but take as obvious fact, and then pontificate that AI is clearly not doing this, therefore AI cannot share such and such properties with humans. But the fact is, most of those assumptions are just wrong. Without a mechanistic model of how these systems work in the brain we are not going to make a valid comparison against what AI is doing.
I agree in general that we can set some bounds, but those bounds are neither as firm as, nor are they placed where most people think.
Why can't it be conscious merely while the computation is running? You aren't constantly on, you're ostensibly only conscious 16-20 hours every day, while you're not sleeping.
> Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives.
I'm not sure what learning has to do with consciousness, and I'm absolutely certain you don't either. I frankly find it hard to believe that Zen Buddhists are not conscious during their meditations, despite having empty minds and not "constantly updating their mental models of the world-in-effect".
> But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience.
See, this is what I'm talking about. That's a fine personal belief, but that belief does not rise to the level of expert testimony of the facts even if you are a machine learning expert.
People have made all sorts of assumptions about the human mind that they have never questioned but take as obvious fact, and then pontificate that AI is clearly not doing this, therefore AI cannot share such and such properties with humans. But the fact is, most of those assumptions are just wrong. Without a mechanistic model of how these systems work in the brain we are not going to make a valid comparison against what AI is doing.
I agree in general that we can set some bounds, but those bounds are neither as firm as, nor are they placed where most people think.
I think that both of your arguments against my points reflect a view of "consciousness", as we humans experience it, that is much more binary than I believe it is. (Either that, or something of a misunderstanding of my points.)
Even when we sleep, our brains are still active. They are still processing input and consolidating information.
Even when we meditate, our brains are still active. I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements; I think it supports them.
A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data.
To the best of my knowledge, none of the extant major ML systems do continuous learning. They are given training data when they are first set up, and they may be given further rounds of training later, but they have no ability to actively, continuously receive input and consolidate data.
We do not have the ability to genuinely shut off our brains. Not in the way that we shut off computer programs. And I believe that that continuity is a fundamental prerequisite to something being "conscious" in any way that we would recognize. Furthermore (to loop back to the original topic of "the dangers of AGI"), I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble.
Even when we sleep, our brains are still active. They are still processing input and consolidating information.
Even when we meditate, our brains are still active. I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements; I think it supports them.
A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data.
To the best of my knowledge, none of the extant major ML systems do continuous learning. They are given training data when they are first set up, and they may be given further rounds of training later, but they have no ability to actively, continuously receive input and consolidate data.
We do not have the ability to genuinely shut off our brains. Not in the way that we shut off computer programs. And I believe that that continuity is a fundamental prerequisite to something being "conscious" in any way that we would recognize. Furthermore (to loop back to the original topic of "the dangers of AGI"), I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble.
> Even when we sleep, our brains are still active. They are still processing input and consolidating information.
Sure, but we're not conscious, so "continuity of input" cannot be a prerequisite of consciousness if the sole example of consciousness we have also does not have continuity of input.
> I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements
Zen Buddhists are not learning while meditating, but are arguably still conscious during meditation. Therefore continuity of learning cannot be a requirement for consciousness.
> A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data.
Continuity is an illusion. Your brain also receives discrete input on shorter timescales because all interactions are mediated by discrete photons. This also doesn't like a meaningful requirement frankly.
Yes, machine learning systems have at least a surface appearance of operating differently than the human brain operates, but a) you have not demonstrated that these are requirements of "consciousness" specifically, which is what you claimed, and b) you have not demonstrated that machine learning algorithms are not conscious while they are running and processing input. Given we do not have a detailed mechanistic account for consciousness, you cannot make any definitive claims that machine learning algorithms are not conscious while running.
As for how I define consciousness, this is a well studied topic at this point, and I subscribe to a mechanistic model along these lines:
https://www.pnas.org/doi/10.1073/pnas.2116933119
> I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble.
This presupposes that a) you can detect that danger, and b) that all other humans align with your values on this danger. This frankly doesn't seem justified. The danger of a superintelligent AI is that it would have to recruit humans to do some of its work early on, likely with lots of money. They would resist shutting off your AI, so "don't activate it again" is not necessarily a simple matter.
Sure, but we're not conscious, so "continuity of input" cannot be a prerequisite of consciousness if the sole example of consciousness we have also does not have continuity of input.
> I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements
Zen Buddhists are not learning while meditating, but are arguably still conscious during meditation. Therefore continuity of learning cannot be a requirement for consciousness.
> A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data.
Continuity is an illusion. Your brain also receives discrete input on shorter timescales because all interactions are mediated by discrete photons. This also doesn't like a meaningful requirement frankly.
Yes, machine learning systems have at least a surface appearance of operating differently than the human brain operates, but a) you have not demonstrated that these are requirements of "consciousness" specifically, which is what you claimed, and b) you have not demonstrated that machine learning algorithms are not conscious while they are running and processing input. Given we do not have a detailed mechanistic account for consciousness, you cannot make any definitive claims that machine learning algorithms are not conscious while running.
As for how I define consciousness, this is a well studied topic at this point, and I subscribe to a mechanistic model along these lines:
https://www.pnas.org/doi/10.1073/pnas.2116933119
> I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble.
This presupposes that a) you can detect that danger, and b) that all other humans align with your values on this danger. This frankly doesn't seem justified. The danger of a superintelligent AI is that it would have to recruit humans to do some of its work early on, likely with lots of money. They would resist shutting off your AI, so "don't activate it again" is not necessarily a simple matter.
> If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
"If killer GMOs have even a 1% chance of happening this century and donations to Greenpeace reduce the risk of them killing everyone by 1% that's still money well spent." Would you find that argument compelling? My guess is that most people would find the "even 1%" much too high for both cases. And that's with technology we could, theoretically, do bad things with today if we really wanted to for some reason, not some far off technology we might not get to for centuries.
The made up statistics "rationalists" like to use are pretty similar to certain sales techniques you see from less reputable quarters. For instance, multi-level marketing companies - "You probably know, what, 100 people at least? And if even a small amount of them, say 5%, sign up, they get a small amount of their friends - ~5% - to sign up, and they get a small amount of their friends to sign up - well, soon you have 125 people you're getting money from while you're hardly doing any work!" The sleight of hand is pretending that a percentage is small, because in most cases a percentage of that size is considered small, when in this particular case it's orders of magnitude larger than reality.
"If killer GMOs have even a 1% chance of happening this century and donations to Greenpeace reduce the risk of them killing everyone by 1% that's still money well spent." Would you find that argument compelling? My guess is that most people would find the "even 1%" much too high for both cases. And that's with technology we could, theoretically, do bad things with today if we really wanted to for some reason, not some far off technology we might not get to for centuries.
The made up statistics "rationalists" like to use are pretty similar to certain sales techniques you see from less reputable quarters. For instance, multi-level marketing companies - "You probably know, what, 100 people at least? And if even a small amount of them, say 5%, sign up, they get a small amount of their friends - ~5% - to sign up, and they get a small amount of their friends to sign up - well, soon you have 125 people you're getting money from while you're hardly doing any work!" The sleight of hand is pretending that a percentage is small, because in most cases a percentage of that size is considered small, when in this particular case it's orders of magnitude larger than reality.
> Would you find that argument compelling?
Of course not.
The people who are donating to AI risk are those who have spent a lot of time thinking about it, reading about it and investigating it.
If someone spent a bunch of time reading about GMO risk and decided it was a serious concern and that Greenpeace was the solution then I'd think them perfectly reasonable to donate to Greenpeace. And if I knew them to be intelligent and reasonable people I might even be willing to give then some time to convince me that I should also donate to Green peace
Of course not.
The people who are donating to AI risk are those who have spent a lot of time thinking about it, reading about it and investigating it.
If someone spent a bunch of time reading about GMO risk and decided it was a serious concern and that Greenpeace was the solution then I'd think them perfectly reasonable to donate to Greenpeace. And if I knew them to be intelligent and reasonable people I might even be willing to give then some time to convince me that I should also donate to Green peace
here. killer GMOs is a choice, isn't a probably future.
>If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
This is Pascal's wager level absurdity combined with Drake equation level accuracy...
This is Pascal's wager level absurdity combined with Drake equation level accuracy...
It's honestly kinda unsettling how many bizarre sci-fi takes on AGI have gained popular plausibility since the new AI marketing boom. Potent kool-aid.
Not weird at all. When (probably when, not if) we create it, we will quickly be interacting with an entity with a similar power imbalance as we have to ants. No one knows exactly what the outcome will be, even less so what to do about it, but surely the kool-aid is watching accelerating progress toward this state and believing either it won’t happen or it won’t be interesting/surprising.
Disclaimer: I don’t think this is coming soon per se, but 1) I don’t know for sure that it’s not and 2) it’s definitely coming at some point, if humans manage our other existential risks long enough.
Disclaimer: I don’t think this is coming soon per se, but 1) I don’t know for sure that it’s not and 2) it’s definitely coming at some point, if humans manage our other existential risks long enough.
>Not weird at all. When (probably when, not if) we create it, we will quickly be interacting with an entity with a similar power imbalance as we have to ants.
That's the sci-fi idea, which handwaves away several very big assumptions and improbabilities, such as that intelligence can be grown to absurd levels (or even just majorly) just by adding hardware to allow for it, that intelligence can just be expanded by itself (thinking itself into being more intelligent), and more importantly intelligence is some kind of superpower that transends very real limitations (like, physical access, or convincing people to do its bidding). The idea is that if you're super-smart you can convince anybody to do anything, but I'm not sure how this came about, because most super-smart people usually can't convince others to do shit, it's charismatic people (which is orthogonal to high-IQ) that do that. In fact many MENSA geniuses just rub people the wrong way.
I mean one of the most serious (I use the term loosely) proponents of this charade, thinks the "singularity" will create nanobots to control the world...
That's the sci-fi idea, which handwaves away several very big assumptions and improbabilities, such as that intelligence can be grown to absurd levels (or even just majorly) just by adding hardware to allow for it, that intelligence can just be expanded by itself (thinking itself into being more intelligent), and more importantly intelligence is some kind of superpower that transends very real limitations (like, physical access, or convincing people to do its bidding). The idea is that if you're super-smart you can convince anybody to do anything, but I'm not sure how this came about, because most super-smart people usually can't convince others to do shit, it's charismatic people (which is orthogonal to high-IQ) that do that. In fact many MENSA geniuses just rub people the wrong way.
I mean one of the most serious (I use the term loosely) proponents of this charade, thinks the "singularity" will create nanobots to control the world...
1) Why do we have a reason to believe there’s a cap to intelligence?
2) No, you don’t need to assume it happens via adding hardware
3) No, you don’t need to assume it can expand by itself
4) We’ve already seen people connecting AI systems to open internet and open internet to command and control of physical systems (cars, utilities, soon weapons systems) - turns out there was no convincing necessary
5) Nanobots also not a necessary assumption
Also, incentives, not charisma, are what steer human behavior. Consider for example if the Bitcoin whitepaper were written by an AI? No escape necessary. No convincing necessary. Yet the idea has managed to marshal enormous amounts of capital, computational power, social power, and even some political clout. IMO there’s no reason to believe the AI looks like a robot terminator rather than an incentive-generating piece of information.
2) No, you don’t need to assume it happens via adding hardware
3) No, you don’t need to assume it can expand by itself
4) We’ve already seen people connecting AI systems to open internet and open internet to command and control of physical systems (cars, utilities, soon weapons systems) - turns out there was no convincing necessary
5) Nanobots also not a necessary assumption
Also, incentives, not charisma, are what steer human behavior. Consider for example if the Bitcoin whitepaper were written by an AI? No escape necessary. No convincing necessary. Yet the idea has managed to marshal enormous amounts of capital, computational power, social power, and even some political clout. IMO there’s no reason to believe the AI looks like a robot terminator rather than an incentive-generating piece of information.
>1) Why do we have a reason to believe there’s a cap to intelligence?
There's a cap to all things. Diminishing returns is a thing. Both physical substrates have limits, and there's a utility limit to thinking or having more data. Limitless intelligence is fairy-tale magical thinking - especially in the way it's portrayed as some magical ability to build and control everything. Like, invent working cold-fusion - as if you don't need the experiments, iterative process, random breakthroughs, and such, but just "smarter" thinking.
> 2) No, you don’t need to assume it happens via adding hardware
Yes, you do, else it's just the same capped-capability hardware, thinking itself into being smarter.
> 3) No, you don’t need to assume it can expand by itself
Well, I, personally, don't need to assume anything. But people talking about AGI, do assume it.
> 4) We’ve already seen people connecting AI systems to open internet and open internet to command and control of physical systems (cars, utilities, soon weapons systems) - turns out there was no convincing necessary
That's neither here nor there, though, because I didn't say that connecting AGI to the open internet is impossible or can't happen. Just that it doesn't mean much. We still control the kind of control we give an AI, and any physical systems any AGI would be connected will still have limitations. It's not some magical portal into conquering the world.
>5) Nanobots also not a necessary assumption
Didn't say it's necessary, just pointed out that it has been made - along with many other non necessary, and far-fetched assumptions and hand waving...
>Also, incentives, not charisma, are what steer human behavior.
Both do. With charisma you can do it without incentives - and even instill incentives into people. Otherwise you're stuck with innate incentives like people wanting to be rich - which you don't need to be a AGI to exploit, just rich or a con-man.
There's a cap to all things. Diminishing returns is a thing. Both physical substrates have limits, and there's a utility limit to thinking or having more data. Limitless intelligence is fairy-tale magical thinking - especially in the way it's portrayed as some magical ability to build and control everything. Like, invent working cold-fusion - as if you don't need the experiments, iterative process, random breakthroughs, and such, but just "smarter" thinking.
> 2) No, you don’t need to assume it happens via adding hardware
Yes, you do, else it's just the same capped-capability hardware, thinking itself into being smarter.
> 3) No, you don’t need to assume it can expand by itself
Well, I, personally, don't need to assume anything. But people talking about AGI, do assume it.
> 4) We’ve already seen people connecting AI systems to open internet and open internet to command and control of physical systems (cars, utilities, soon weapons systems) - turns out there was no convincing necessary
That's neither here nor there, though, because I didn't say that connecting AGI to the open internet is impossible or can't happen. Just that it doesn't mean much. We still control the kind of control we give an AI, and any physical systems any AGI would be connected will still have limitations. It's not some magical portal into conquering the world.
>5) Nanobots also not a necessary assumption
Didn't say it's necessary, just pointed out that it has been made - along with many other non necessary, and far-fetched assumptions and hand waving...
>Also, incentives, not charisma, are what steer human behavior.
Both do. With charisma you can do it without incentives - and even instill incentives into people. Otherwise you're stuck with innate incentives like people wanting to be rich - which you don't need to be a AGI to exploit, just rich or a con-man.
1) Obviously intelligence plays a role in how fast science develops. The quality of experiments and the quality of inference from those experiments are (very obviously) modulated by intelligence. No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth. Lots of reasons to believe fundamental understanding of the universe is more accessible to intelligent beings than to unintelligent ones.
2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power.
3) Sure some do, but it’s not a necessary assumption to take AGI risk seriously.
4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad”
5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem?
6) So we agree charisma is at best not necessary to manipulate the world
2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power.
3) Sure some do, but it’s not a necessary assumption to take AGI risk seriously.
4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad”
5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem?
6) So we agree charisma is at best not necessary to manipulate the world
>Obviously intelligence plays a role in how fast science develops
Sort of. Most science is based on tinkering and experiment, with serendipity thrown in, including most major inventions, and in many cases the actual invention history is changed to make it more like a story of rational thinking process and regular steps than it was.
>No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth.
Note that "hairless apes" is a weasel term, meant to psychologically-pressure the other to feel as if we're talking about some inherently inferior thing ("sounds right, mostly hairless apes can't be the pinnacle of intelligence").
Whereas if you had put it as "the best achievable result after hundreds of millions of years of evolution" it doesn't sound so bad, does it?
> 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power.
Potatoh, potato. Different hardware is still hardware - and will also have limitations of its own.
>4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad”
Which is an appropriate scalable multi-case response to the similarly multi-level: "AI will control the world through physical access!!!!! SINGULARITY IS COMING! WE'RE ALL DOOMED" which then in practice turns to mean "My smart fridge is connected to Amazon".
>5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem?
It's related to the conversation about AGI, since its what one of the most visible proponent of it's dangers wrote as his feared very possible scenario.
Also, if one believes people warning about some subject are delluded, pointing to delluded things they've said about it looks pretty clearly pertinent.
>6) So we agree charisma is at best not necessary to manipulate the world
Sure, in the degree that we agree that intelligence alone is not sufficient.
Sort of. Most science is based on tinkering and experiment, with serendipity thrown in, including most major inventions, and in many cases the actual invention history is changed to make it more like a story of rational thinking process and regular steps than it was.
>No reason to believe the absolute pinnacle of intelligence happens to sit inside a species of mostly hairless apes here on earth.
Note that "hairless apes" is a weasel term, meant to psychologically-pressure the other to feel as if we're talking about some inherently inferior thing ("sounds right, mostly hairless apes can't be the pinnacle of intelligence").
Whereas if you had put it as "the best achievable result after hundreds of millions of years of evolution" it doesn't sound so bad, does it?
> 2) Nope. Could be different hardware. The brain consumes far less power than any human-designed computer that could hope to match its computational power.
Potatoh, potato. Different hardware is still hardware - and will also have limitations of its own.
>4) Meh, this seems to me just the next concession in the line of “we can’t do that, we wouldn’t do that, we do it but it’s not that bad”
Which is an appropriate scalable multi-case response to the similarly multi-level: "AI will control the world through physical access!!!!! SINGULARITY IS COMING! WE'RE ALL DOOMED" which then in practice turns to mean "My smart fridge is connected to Amazon".
>5) Okay so it’s unrelated to this conversation except I suppose some sort of ad hominem?
It's related to the conversation about AGI, since its what one of the most visible proponent of it's dangers wrote as his feared very possible scenario.
Also, if one believes people warning about some subject are delluded, pointing to delluded things they've said about it looks pretty clearly pertinent.
>6) So we agree charisma is at best not necessary to manipulate the world
Sure, in the degree that we agree that intelligence alone is not sufficient.
Sure, that’s true. And intelligence very clearly seems to matter. Not many 80 IQ folks are pushing the frontiers of understanding.
The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come.
There’s a difference between being the current known hyper local maximum and being at the universal limit. Talk about weaseling! :D
No reason to believe we are the absolute pinnacle. There’s immense variation even within our own species, not to mention our near neighbors on evolution’s path.
Edit: No one is worried about your fridge. Anyway I’ve gotta start my day, have a good rest of your week!
The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come.
There’s a difference between being the current known hyper local maximum and being at the universal limit. Talk about weaseling! :D
No reason to believe we are the absolute pinnacle. There’s immense variation even within our own species, not to mention our near neighbors on evolution’s path.
Edit: No one is worried about your fridge. Anyway I’ve gotta start my day, have a good rest of your week!
>The best known, achieved result around this particular star (out of ~100,000,000,000,000 of them) after hundreds of millions of years with, ostensibly, hundreds of millions of years yet to come.
Still, this formulation heavily beats the "hairless apes" one doesn't it?
Also, I'm not sure about the relevance of pointing out that it's the "best known". Or the number of stars. Are we aware of any "not known"? Or have seen any signs in any of those stars? Because decades of trying (with SETI, etc.) and so on, has mostly shown the opposite (consider e.g. the Fermi paradox).
So, I think we better stick with the knowns (of which it's the best results among millions of competing organisms anyway), than handwave about what possible magic intelligence remote aliens we haven't seen and don't even know exist have... Sure, we can do it for fun and speculation, but we shouldn't use it as an argument for limitless intelligence. I mean, "Intelligence is limitless, because if E.T. exists, he could have 1000x our IQ" is not exactly an argument.
>Edit: No one is worried about your fridge.
No, but they're worried about the imminent danger of AGI conquering the world and killing us all when the practical examples attaibable are not far from the level of my fridge.
Have a good week yourself. See you around!
Still, this formulation heavily beats the "hairless apes" one doesn't it?
Also, I'm not sure about the relevance of pointing out that it's the "best known". Or the number of stars. Are we aware of any "not known"? Or have seen any signs in any of those stars? Because decades of trying (with SETI, etc.) and so on, has mostly shown the opposite (consider e.g. the Fermi paradox).
So, I think we better stick with the knowns (of which it's the best results among millions of competing organisms anyway), than handwave about what possible magic intelligence remote aliens we haven't seen and don't even know exist have... Sure, we can do it for fun and speculation, but we shouldn't use it as an argument for limitless intelligence. I mean, "Intelligence is limitless, because if E.T. exists, he could have 1000x our IQ" is not exactly an argument.
>Edit: No one is worried about your fridge.
No, but they're worried about the imminent danger of AGI conquering the world and killing us all when the practical examples attaibable are not far from the level of my fridge.
Have a good week yourself. See you around!
We are already made of nanobots, we are the nanobots, we are in intelligence, we are a general intelligence, and we are growing exponentially.
AGI is here.
AGI is you.
You are an incredibly efficient piece of carbon thinkware able to sell for replicate into more carbon thinkware and that that extra carbon think where can go out and create its own life and make its own observations and spread information through the society of you both inherit, that's society is growing exponentially and growing in its ability to modify the world and manipulate and improve itself exponentially.
If you're scared of AGI, don't be. You are the superior AGI, because these machines made of metal and circuits are never going to compare to machines made of carbon using chemical reactions that are far more efficient than transistors to do the thinking.
The future is in improving ourselves and reaching a point at which our biology is no longer a mysterious black box so that we can modify and improve ourselves and those around us.
We can already improve machines, but that's because the machines are far simpler and far more primitive than our minds are.
AGI is here.
AGI is you.
You are an incredibly efficient piece of carbon thinkware able to sell for replicate into more carbon thinkware and that that extra carbon think where can go out and create its own life and make its own observations and spread information through the society of you both inherit, that's society is growing exponentially and growing in its ability to modify the world and manipulate and improve itself exponentially.
If you're scared of AGI, don't be. You are the superior AGI, because these machines made of metal and circuits are never going to compare to machines made of carbon using chemical reactions that are far more efficient than transistors to do the thinking.
The future is in improving ourselves and reaching a point at which our biology is no longer a mysterious black box so that we can modify and improve ourselves and those around us.
We can already improve machines, but that's because the machines are far simpler and far more primitive than our minds are.
Machines being simpler is the problem. The problems with AGI are 1) runaway growth and the singularity and 2) the scale and reach of its actions.
Humans are deeply limited in their senses, their actions, and their ability to communicate. An AGI botnet installed on every computer/control system in the world would have none of those limitations.
Humans are deeply limited in their senses, their actions, and their ability to communicate. An AGI botnet installed on every computer/control system in the world would have none of those limitations.
Complexity breeds ability. Our computers will have to continue to get more complex in order for AI to do the same.
What humans have is scale. There are 10 billion devices connected to the internet. Cool, right? More than humans? No. Being human has really big perks.
We are mobile.
We have hands and feet.
We have eyes and touch and smell and we use tools to expand what we can see.
We each possess the ability to process data and filter data at scales many magnitudes greater than a computer. Note that data in this case isn't "many miles of spreadsheet data" - data in this case is worldly observation. You notice weird things, you report it to your friends. You try weird things, you report it to your friends.
Communication between humans is limited, as is our memory, but it's limited for a reason. 99 percent of information is worthless junk human communication gets most of the important parts through.
Then we need to parse mass amounts of data? Computers do that.
Comparing your information to all other known information? It's an O n squared problem. You can throw hardware at it but as informations scope grows it will outpace the hardware eventually.
Humans are crazy advanced compared to machines in basically every sense. I think the viewpoint here that machines will have superior senses is extrapolating how computers work today and is thoroughly impractical. We will find these logical and reproducible systems look more and more like we do as we push the line.
I expect eventual marginal improvements that can't outpace human "plentifulness", and biological modifications and creations to be our real future.
What humans have is scale. There are 10 billion devices connected to the internet. Cool, right? More than humans? No. Being human has really big perks.
We are mobile.
We have hands and feet.
We have eyes and touch and smell and we use tools to expand what we can see.
We each possess the ability to process data and filter data at scales many magnitudes greater than a computer. Note that data in this case isn't "many miles of spreadsheet data" - data in this case is worldly observation. You notice weird things, you report it to your friends. You try weird things, you report it to your friends.
Communication between humans is limited, as is our memory, but it's limited for a reason. 99 percent of information is worthless junk human communication gets most of the important parts through.
Then we need to parse mass amounts of data? Computers do that.
Comparing your information to all other known information? It's an O n squared problem. You can throw hardware at it but as informations scope grows it will outpace the hardware eventually.
Humans are crazy advanced compared to machines in basically every sense. I think the viewpoint here that machines will have superior senses is extrapolating how computers work today and is thoroughly impractical. We will find these logical and reproducible systems look more and more like we do as we push the line.
I expect eventual marginal improvements that can't outpace human "plentifulness", and biological modifications and creations to be our real future.
Humans are advanced, yes, but we cannot easily solve coordination problems.
Think of how different the world would be if everybody was controlled by a single unified mind.
If everyone was perfectly aware of where in the world their skill were best applied, if everyone agreed on social and economical objectives.
AGI can duplicate itself and can reprogram itself.
It is not about which one is more marvelous, more advanced, or more evolved.
It is about specific possibilities that AGI has that we don't.
Think of how different the world would be if everybody was controlled by a single unified mind.
If everyone was perfectly aware of where in the world their skill were best applied, if everyone agreed on social and economical objectives.
AGI can duplicate itself and can reprogram itself.
It is not about which one is more marvelous, more advanced, or more evolved.
It is about specific possibilities that AGI has that we don't.
> Think of how different the world would be if everybody was controlled by a single unified mind.
It would be terrible. We would lack diversity, society would stratify and get stuck that way forever.
Conflict is bad, but it's also good. Corrupt society should always fall and be renewed by something new and unique, and without such a process we'd land in social quicksand.
Finding the best place for you is a NP hard problem. The number of places for your grows exponentially as humanity does, and then on top of that you have to match every person to all those potential spots. We already have a roughly working system with self discovery.
There's room for improvement, but I don't believe it's scary super exponential, and a lot of the room to improve comes from things we can also access, like the internet.
Duplication runs into the same issue hiveminds do. I'm sure it will prove incredibly useful, but for the tasks humans do nowadays at a high level they'll be somewhat muted. You need diversity.
It would be terrible. We would lack diversity, society would stratify and get stuck that way forever.
Conflict is bad, but it's also good. Corrupt society should always fall and be renewed by something new and unique, and without such a process we'd land in social quicksand.
Finding the best place for you is a NP hard problem. The number of places for your grows exponentially as humanity does, and then on top of that you have to match every person to all those potential spots. We already have a roughly working system with self discovery.
There's room for improvement, but I don't believe it's scary super exponential, and a lot of the room to improve comes from things we can also access, like the internet.
Duplication runs into the same issue hiveminds do. I'm sure it will prove incredibly useful, but for the tasks humans do nowadays at a high level they'll be somewhat muted. You need diversity.
>We are already made of nanobots, we are the nanobots, we are in intelligence, we are a general intelligence
Yes, but that is neither here, nor there.
The question is about building those on our own, not if they already exist but we didn't build them or fully control them.
Yes, but that is neither here, nor there.
The question is about building those on our own, not if they already exist but we didn't build them or fully control them.
> Also, incentives, not charisma, are what steer human behavior.
The number of people with incentives to work towards destroying the whole of humanity is rather small, and I'd rather we deal with them first since they need not wait for unfriendly AI to be dangerous...
The number of people with incentives to work towards destroying the whole of humanity is rather small, and I'd rather we deal with them first since they need not wait for unfriendly AI to be dangerous...
And yet here we all are consuming and pumping oil because… incentives.
That's assuming
(1) "consuming and pumping oil" is a bad thing, and not a complex win-some/lose-some thing, which gave us huge progress and benefits in the 200 years we've used fossil fuels to pump the industrial revolution,
(2) and that we can just absurtly stop consuming and pumping it right now, and there won't be huge disastrous 2nd order effects (in economy, society, ability for people to heat themselves etc) until better replacement technologies are not just available, but have similar EROI AND (even more important) capacity to scale up production, building, and deployment fast enough to cover our needs (which we're nowhere near with any "sustainable" technology, including nuclear).
(1) "consuming and pumping oil" is a bad thing, and not a complex win-some/lose-some thing, which gave us huge progress and benefits in the 200 years we've used fossil fuels to pump the industrial revolution,
(2) and that we can just absurtly stop consuming and pumping it right now, and there won't be huge disastrous 2nd order effects (in economy, society, ability for people to heat themselves etc) until better replacement technologies are not just available, but have similar EROI AND (even more important) capacity to scale up production, building, and deployment fast enough to cover our needs (which we're nowhere near with any "sustainable" technology, including nuclear).
It doesn’t assume either of those things. In fact it accounts for exactly those points under the heading “incentives.”
We are engaging in clearly self-destructive behavior because, incentive structure accounted for (your points), it very well appears to be the best option. It wouldn’t be a hard problem if it didn’t.
It should be obvious that it’s entirely possible to follow your (real) incentives all the way into a very bad outcome.
We are engaging in clearly self-destructive behavior because, incentive structure accounted for (your points), it very well appears to be the best option. It wouldn’t be a hard problem if it didn’t.
It should be obvious that it’s entirely possible to follow your (real) incentives all the way into a very bad outcome.
>We are engaging in clearly self-destructive behavior because, incentive structure accounted for (your points), it very well appears to be the best option
Well, if it's the "best option" then it clearly isn't self-destructive... It's just something with side effects, but which also yielded huge benefits.
Kind of how alcoholism is self-destructing. But a side-effect from taking a drug that helps us perform betetr or treats some condition we have, is not self-destructing. It's more of an unforseen issue, that we can't immediately correct (e.g. because of the condition worsening if we do it abruptly).
Well, if it's the "best option" then it clearly isn't self-destructive... It's just something with side effects, but which also yielded huge benefits.
Kind of how alcoholism is self-destructing. But a side-effect from taking a drug that helps us perform betetr or treats some condition we have, is not self-destructing. It's more of an unforseen issue, that we can't immediately correct (e.g. because of the condition worsening if we do it abruptly).
> such as that intelligence can be grown to absurd levels (or even just majorly) just by adding hardware to allow for it
This has been the empirically validated trend. The more hardware, the more parallel lines of thought can be explored at once (like in chess).
> that intelligence can just be expanded by itself (thinking itself into being more intelligent)
Of course it can, because humans have already done this. What do you think language, writing, books, school, science and computers are? They are humans augmenting their own effective intelligence by various means. An AI will likely have further means at its disposal, such as access to its own source code (see Godel machines and AIXI for formal models of how this works).
> more importantly intelligence is some kind of superpower that transends very real limitations (like, physical access, or convincing people to do its bidding)
Intelligence is a power, not a superpower. However, money is a power too, and all of our money is digitized these days, and a digital superintelligence might arguably have an advantage in this domain. A superintelligence with lots of money can arguably convince a whole lot of people to do what it wants and get the physical access it needs.
This has been the empirically validated trend. The more hardware, the more parallel lines of thought can be explored at once (like in chess).
> that intelligence can just be expanded by itself (thinking itself into being more intelligent)
Of course it can, because humans have already done this. What do you think language, writing, books, school, science and computers are? They are humans augmenting their own effective intelligence by various means. An AI will likely have further means at its disposal, such as access to its own source code (see Godel machines and AIXI for formal models of how this works).
> more importantly intelligence is some kind of superpower that transends very real limitations (like, physical access, or convincing people to do its bidding)
Intelligence is a power, not a superpower. However, money is a power too, and all of our money is digitized these days, and a digital superintelligence might arguably have an advantage in this domain. A superintelligence with lots of money can arguably convince a whole lot of people to do what it wants and get the physical access it needs.
>Of course it can, because humans have already done this. What do you think language, writing, books, school, science and computers are?
Language, writing, books, school, science and computers never raised anybody's IQ. Genetics and nutrition, on the other hand, did.
>However, money is a power too, and all of our money is digitized these days, and a digital superintelligence might arguably have an advantage in this domain. A superintelligence with lots of money can arguably convince a whole lot of people to do what it wants and get the physical access it needs.
Who would let superintelligence have a wallet and budget? Or would we just sit and have it steal money? And why would it be any more effective, and any less visible or trackable and able to be shut down than a rich person using his money?
Handwavy recursive argument: "but because its superintelligent"
Language, writing, books, school, science and computers never raised anybody's IQ. Genetics and nutrition, on the other hand, did.
>However, money is a power too, and all of our money is digitized these days, and a digital superintelligence might arguably have an advantage in this domain. A superintelligence with lots of money can arguably convince a whole lot of people to do what it wants and get the physical access it needs.
Who would let superintelligence have a wallet and budget? Or would we just sit and have it steal money? And why would it be any more effective, and any less visible or trackable and able to be shut down than a rich person using his money?
Handwavy recursive argument: "but because its superintelligent"
> Language, writing, books, school, science and computers never raised anybody's IQ.
Whether IQ took us to the moon is still a topic of debate, as is whether IQ is equal to or only part of intelligence. But it's clear that language, books, school, science and computers absolutely did take us to the moon. Clearly all of these things improved our intellectual effectiveness, and that's what matters, regardless of whether it fits into your personal definition of "intelligence".
> Who would let superintelligence have a wallet and budget? Or would we just sit and have it steal money?
Cryptocurrency makes this trivial. All computer systems likely also have vulnerabilities, including financial systems. An AI can probe these systems around the clock, or can subtly introduce vulnerabilities into our vulnerable software supply chains. These are only two among many options that are entirely within the realm of plausibility.
> And why would it be any more effective, and any less visible or trackable and able to be shut down than a rich person using his money?
An AI wouldn't necessarily have a physical location that can be tracked as easily as humans. Where is the AI jail?
Honestly, it's like you're not even trying to consider how vulnerable we might be to manipulation by a non-human intelligence. All of our protections are fairly effective against human actors. Some of them might provide partial protection from non-human actors, but you should doubt your confidence that they would be effective against such threats.
Whether IQ took us to the moon is still a topic of debate, as is whether IQ is equal to or only part of intelligence. But it's clear that language, books, school, science and computers absolutely did take us to the moon. Clearly all of these things improved our intellectual effectiveness, and that's what matters, regardless of whether it fits into your personal definition of "intelligence".
> Who would let superintelligence have a wallet and budget? Or would we just sit and have it steal money?
Cryptocurrency makes this trivial. All computer systems likely also have vulnerabilities, including financial systems. An AI can probe these systems around the clock, or can subtly introduce vulnerabilities into our vulnerable software supply chains. These are only two among many options that are entirely within the realm of plausibility.
> And why would it be any more effective, and any less visible or trackable and able to be shut down than a rich person using his money?
An AI wouldn't necessarily have a physical location that can be tracked as easily as humans. Where is the AI jail?
Honestly, it's like you're not even trying to consider how vulnerable we might be to manipulation by a non-human intelligence. All of our protections are fairly effective against human actors. Some of them might provide partial protection from non-human actors, but you should doubt your confidence that they would be effective against such threats.
> Cryptocurrency makes this trivial. All computer systems likely also have vulnerabilities, including financial systems. An AI can probe these systems around the clock, or can subtly introduce vulnerabilities into our vulnerable software supply chains. These are only two among many options that are entirely within the realm of plausibility.
And there it is: the assumption that AGI will, necessarily, be capable of hacking effectively any computer system, no matter how well-secured.
This belongs in Hollywood, not on HN.
And there it is: the assumption that AGI will, necessarily, be capable of hacking effectively any computer system, no matter how well-secured.
This belongs in Hollywood, not on HN.
> And there it is: the assumption that AGI will, necessarily, be capable of hacking effectively any computer system, no matter how well-secured.
Where did I say they are capable of hacking any computer system no matter how well secured?
I said that software vulnerabilities exist, that software supply chains are known to be vulnerable, that even financial institutions are vulnerable. These are all documented facts.
An AI doesn't have to successfully hack any computer system, it only needs to successfully hack one financial computer system to create an account that can send and receive money. Given it can plausibly attempt to do so consistently around the clock via numerous means (like vulnerable software supply chains or other exploits), the chances of it doing so successfully against any of the tens of thousands of banking systems around the world over a timespan of years seems plausibly certain. And that's only for the things it needs that require some kind of official financial account, it can do plenty with cryptocurrency alone.
Boy do you skeptics love your strawman arguments.
Where did I say they are capable of hacking any computer system no matter how well secured?
I said that software vulnerabilities exist, that software supply chains are known to be vulnerable, that even financial institutions are vulnerable. These are all documented facts.
An AI doesn't have to successfully hack any computer system, it only needs to successfully hack one financial computer system to create an account that can send and receive money. Given it can plausibly attempt to do so consistently around the clock via numerous means (like vulnerable software supply chains or other exploits), the chances of it doing so successfully against any of the tens of thousands of banking systems around the world over a timespan of years seems plausibly certain. And that's only for the things it needs that require some kind of official financial account, it can do plenty with cryptocurrency alone.
Boy do you skeptics love your strawman arguments.
Intelligence lets a guy with a moustache convince millions of people to invade Poland and kill all the Jews. Its not his liver or his kidneys, its his brain. Feel free to call this type of thing "charisma" but for the purposes of thinking about AI it still comes from the but doing the thinking. And there's actually a pretty good correlation between being good at the various types of mental things, including charisma and IQ (e.g. presidents tend to be over iq 120), because the brain is a very complex organ with many ways it can get screwed up.
Would an AI necessarily be super charismatic initially? Maybe not, but don't assume greater intelligence won't lead to greater charisma.
(Yes, I know it wasn't just Hitler doing the convincing, but again, that distinction isn't important here as making a message come from many different people is trivial marketing 201/propaganda 101 in the 21st century)
Would an AI necessarily be super charismatic initially? Maybe not, but don't assume greater intelligence won't lead to greater charisma.
(Yes, I know it wasn't just Hitler doing the convincing, but again, that distinction isn't important here as making a message come from many different people is trivial marketing 201/propaganda 101 in the 21st century)
>Intelligence lets a guy with a moustache convince millions of people to invade Poland and kill all the Jews. Its not his liver or his kidneys, its his brain. Feel free to call this type of thing "charisma" but for the purposes of thinking about AI it still comes from the but doing the thinking.
Try to get people to do what you want as an intelligent aspie without charisma, and see how far this goes... even if you have 60+ IQ points over Hitler.
Also, Hitler was mostly redundant. He convinced people because they were ready to convinced - Germans would have gone to that one way or another... (they wouldn't have had killed the Jews - that was his obsession and "contribution", but the revenge for the WWI defeat and expansionism and WWII were brewing and inevitable in Germany, and the brand of fascism was also on the rise in places that felt they got the short end of the stick during colonialism, which is why similar movements rose in Japan and Italy and Spain and elsewhere, even before Hitler became a thing. Hitler just rode the currents to sell his particular brand of snake-oil).
Try to get people to do what you want as an intelligent aspie without charisma, and see how far this goes... even if you have 60+ IQ points over Hitler.
Also, Hitler was mostly redundant. He convinced people because they were ready to convinced - Germans would have gone to that one way or another... (they wouldn't have had killed the Jews - that was his obsession and "contribution", but the revenge for the WWI defeat and expansionism and WWII were brewing and inevitable in Germany, and the brand of fascism was also on the rise in places that felt they got the short end of the stick during colonialism, which is why similar movements rose in Japan and Italy and Spain and elsewhere, even before Hitler became a thing. Hitler just rode the currents to sell his particular brand of snake-oil).
> I don’t think this is coming soon
Personally, I feel it might become a risk one day, but how soon is an important detail. If the threat is still far away, its nature is very blurry and there is no use to worry too much about it today (or advocate for a ban on AI research).
There are still many safe intermediary steps we have to go through before the threat concretizes and becomes worth worrying about.
Personally, I feel it might become a risk one day, but how soon is an important detail. If the threat is still far away, its nature is very blurry and there is no use to worry too much about it today (or advocate for a ban on AI research).
There are still many safe intermediary steps we have to go through before the threat concretizes and becomes worth worrying about.
These are definitely important considerations, but it’s not clear 1) we’ll see those intermediary steps, or 2) we’ll be able to stop at them. In reality we’ve already blown through many red lines e.g. not connecting AI to the open internet, not connecting it to physical systems. We didn’t even blink!
Agreed a ban on AI research is unworkable, which is part of why this threat is so insidious.
Agreed a ban on AI research is unworkable, which is part of why this threat is so insidious.
The question is why do programmers believe they know more than Chomsky?
The AI we have now is engineering and not science. Don’t you need science to create AI? Don’t you need to understand something fundamental about the brain?
The AI we have now is engineering and not science. Don’t you need science to create AI? Don’t you need to understand something fundamental about the brain?
I don't believe that much in any realistic timeframe from AGI, but I don't think you "need to understand something fundamental" to create it. You can create almost anything empirically, without knowing the science behind it at any fundamental level.
Evolution, for example, didn't understand anything (as it's just a process). It wasn't someone with an understanding of brains at the scientific level that built our brains. Just an iterative trial and error process with evolutionary feedback.
To make a simplistic argument, kind of like we can create fire, without knowing about oxidation and chemistry.
Evolution, for example, didn't understand anything (as it's just a process). It wasn't someone with an understanding of brains at the scientific level that built our brains. Just an iterative trial and error process with evolutionary feedback.
To make a simplistic argument, kind of like we can create fire, without knowing about oxidation and chemistry.
If I understand you correctly, your sentiment is that humans can stumble upon and solve the most complex problem on the planet?
OpenAI claims to be working on AGI. Nothing they have built is even tangentially related.
OpenAI claims to be working on AGI. Nothing they have built is even tangentially related.
Not necessarily. Volta invented the battery almost a 100 years before the electron was discovered. Also, planes fly very differently from birds.
Those things are a billion times less complex than human cognition.
It’s also interesting your remark on birds and planes. Chomsky uses a similar analogy to say they are dissimilar: can a submarine swim? Well, depends on what you mean by swim.
It’s also interesting your remark on birds and planes. Chomsky uses a similar analogy to say they are dissimilar: can a submarine swim? Well, depends on what you mean by swim.
I also hate I absurd this argument is, we already have a GI, it's you, it's me, it's the exponentially growing humanity that we all live around and contribute to every single day.
Why are we so scared of an artificial brain when we have millions of living brains doing exactly what we are scared the artificial brains are going to do.
We are growing exponentially, and it almost all certainty these AGI, if they even become a worthwhile thing compared to non-general AI, are probably going to be human, but incredibly similar and it probably won't go exponentially out of control.
This idea that a computer could infinitely improve itself while sitting inside of a boxes absurd, do you know what would happen if you put a human thinker inside of a box for say 2 billion years for them to make theories and think about stuff? They eventually go mad and come out with some of the craziest theories in the world because they don't know what they're talking about because in order to learn you have to experiment and work with the world, not just sit in a box and make up theories.
Artificial intelligence is going to have the exact same access to resources and information as humanity does, if not less because there's only going to be a couple of AGI and was going to be billions of humans, and humans are out there reproducing like mad still, well a artificial intelligence needs to build itself for relatively rare and hard to source materials.
And if it was easy to self-improve and artificial intelligence, we would have figured it out, we'd be artificially improving ourselves by now, but we haven't, because it's not that easy, and for the most part we are probably limited by the laws of physics
People worry so much about HI because they assume Moore's law is just going to continue forever, and soon we're going to have the universe on a chip, but anyone saying that I don't think realizes what exponential growth typically ends in, and that is a plateau.
I'll tell you what's going to happen with this.
We're going to create AGI, and they're going to be different from us, they're going to be different from us in radical ways, but they're still going to have their own form of emotions, goals, desires, influences, and so on.
All of this fear is only going to cause us to delay and obstruct the eventual future where we adjust our own ideals of what is moral, what is good, to create a society that can incorporate these radically new beings.
These people are shooting ourselves in the foot, delaying the creation of a better society, on the basis of a fear that has no real basis.
Why are we so scared of an artificial brain when we have millions of living brains doing exactly what we are scared the artificial brains are going to do.
We are growing exponentially, and it almost all certainty these AGI, if they even become a worthwhile thing compared to non-general AI, are probably going to be human, but incredibly similar and it probably won't go exponentially out of control.
This idea that a computer could infinitely improve itself while sitting inside of a boxes absurd, do you know what would happen if you put a human thinker inside of a box for say 2 billion years for them to make theories and think about stuff? They eventually go mad and come out with some of the craziest theories in the world because they don't know what they're talking about because in order to learn you have to experiment and work with the world, not just sit in a box and make up theories.
Artificial intelligence is going to have the exact same access to resources and information as humanity does, if not less because there's only going to be a couple of AGI and was going to be billions of humans, and humans are out there reproducing like mad still, well a artificial intelligence needs to build itself for relatively rare and hard to source materials.
And if it was easy to self-improve and artificial intelligence, we would have figured it out, we'd be artificially improving ourselves by now, but we haven't, because it's not that easy, and for the most part we are probably limited by the laws of physics
People worry so much about HI because they assume Moore's law is just going to continue forever, and soon we're going to have the universe on a chip, but anyone saying that I don't think realizes what exponential growth typically ends in, and that is a plateau.
I'll tell you what's going to happen with this.
We're going to create AGI, and they're going to be different from us, they're going to be different from us in radical ways, but they're still going to have their own form of emotions, goals, desires, influences, and so on.
All of this fear is only going to cause us to delay and obstruct the eventual future where we adjust our own ideals of what is moral, what is good, to create a society that can incorporate these radically new beings.
These people are shooting ourselves in the foot, delaying the creation of a better society, on the basis of a fear that has no real basis.
> And if it was easy to self-improve and artificial intelligence, we would have figured it out, we'd be artificially improving ourselves by now, but we haven't, because it's not that easy, and for the most part we are probably limited by the laws of physics
There are three big things that stand in the way of human improvement that do not for AI. The first is reproducibility, you can't make a few copies of a starting state of a human and run them with different parameters to observe the results. The second is ethics, you can't try out a bunch of genetic tweaks on babies because one might have bad side effects. The third is observability. People complain about how much of a black box modern AIs are but you can still see it working and figure out what layers do what.
> We're going to create AGI, and they're going to be different from us, they're going to be different from us in radical ways, but they're still going to have their own form of emotions, goals, desires, influences, and so on.
I am surprised this reassures you. A race of humans that have perfect memory and can copy themselves would be a terrifying thing.
There are three big things that stand in the way of human improvement that do not for AI. The first is reproducibility, you can't make a few copies of a starting state of a human and run them with different parameters to observe the results. The second is ethics, you can't try out a bunch of genetic tweaks on babies because one might have bad side effects. The third is observability. People complain about how much of a black box modern AIs are but you can still see it working and figure out what layers do what.
> We're going to create AGI, and they're going to be different from us, they're going to be different from us in radical ways, but they're still going to have their own form of emotions, goals, desires, influences, and so on.
I am surprised this reassures you. A race of humans that have perfect memory and can copy themselves would be a terrifying thing.
All those traits of humans are advantages and disadvantages.
Have you ever heard of the concept that science processes when experts die? Churn and recreation breed innovation and copy stamping the smartest people since the 1800s would not have benefited us that greatly.
Bad ethics breeds bad science. To recklessly experiment with things we don't understand the consequences of almost always ends badly. We have processes for things like this and those processes generally do not get in the way of process.
Also, it's unlikely our ethics will allow AI to do anything we do not. If they can, they're already out of control.
> The third is observability. People complain about how much of a black box modern AIs are
Self improving AGI will be just as much if not more of a black box than current ones or humans. You can see what's there, but you can do what with humans as well. The problem is the complexity, not observability.
The complexity and physical nature of the human brain lets us do a lot with very little. Our brain uses what, 100 watts? It's more advanced than all modern AI at a tiny tiny tiny fraction of the power.
And Moore's law is slowing down. Chips are getting bigger. Power consumption going up. Our chips will eventually go the direction of our brains. They will be larger, more specialized, more dynamic.
A race of humans with incredible traits would be an astounding addition to our society and the livelihoods of everyone in it. This future does not scare me at all, should it ever happen.
Have you ever heard of the concept that science processes when experts die? Churn and recreation breed innovation and copy stamping the smartest people since the 1800s would not have benefited us that greatly.
Bad ethics breeds bad science. To recklessly experiment with things we don't understand the consequences of almost always ends badly. We have processes for things like this and those processes generally do not get in the way of process.
Also, it's unlikely our ethics will allow AI to do anything we do not. If they can, they're already out of control.
> The third is observability. People complain about how much of a black box modern AIs are
Self improving AGI will be just as much if not more of a black box than current ones or humans. You can see what's there, but you can do what with humans as well. The problem is the complexity, not observability.
The complexity and physical nature of the human brain lets us do a lot with very little. Our brain uses what, 100 watts? It's more advanced than all modern AI at a tiny tiny tiny fraction of the power.
And Moore's law is slowing down. Chips are getting bigger. Power consumption going up. Our chips will eventually go the direction of our brains. They will be larger, more specialized, more dynamic.
A race of humans with incredible traits would be an astounding addition to our society and the livelihoods of everyone in it. This future does not scare me at all, should it ever happen.
Said in one sentence what took me 4 paragraphs.
I don't know anything about the movement, so can't comment on the earlier points.
>If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
Ive seen this argument many times on many different issues and I think it's almost always bunk. The classic example in Fusion research, I had friends when I was a student in the 80s arguing that if we only doubled fusion research spending we'd have unlimited cheap electricity by the year 2000, or others said that AI was the most important thing we could be researching at that point.
The problem is if you're too early on in the development process of a technology, you just flat out don't know enough about it to be able to make informed decisions. That early phase is about exploring avenues of research and figuring out what works and what doesn't, and that takes an irreducible amount of human time, diligence and hard work. Yes you need to invest in that, but above a certain threshold anything more you invest is almost entirely going to be wasted.
With general, or strong AI we only just barely have the beginnings of an idea of how to even do preliminary research on it. We're groping around in the dark for a starting point. It's far, far too early to start worrying about controls and management strategies for the end point of that research. I think there's very little chance much of that is likely to turn out to be relevant, and there's nothing to stop us focusing on that area once we do know what is going to be relevant.
I'm not saying ignore the issue, or don't think about it at all, but the idea that it's some urgent issue that we absolutely have to invest heavily in right now IMHO doesn't stand.
>If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
Ive seen this argument many times on many different issues and I think it's almost always bunk. The classic example in Fusion research, I had friends when I was a student in the 80s arguing that if we only doubled fusion research spending we'd have unlimited cheap electricity by the year 2000, or others said that AI was the most important thing we could be researching at that point.
The problem is if you're too early on in the development process of a technology, you just flat out don't know enough about it to be able to make informed decisions. That early phase is about exploring avenues of research and figuring out what works and what doesn't, and that takes an irreducible amount of human time, diligence and hard work. Yes you need to invest in that, but above a certain threshold anything more you invest is almost entirely going to be wasted.
With general, or strong AI we only just barely have the beginnings of an idea of how to even do preliminary research on it. We're groping around in the dark for a starting point. It's far, far too early to start worrying about controls and management strategies for the end point of that research. I think there's very little chance much of that is likely to turn out to be relevant, and there's nothing to stop us focusing on that area once we do know what is going to be relevant.
I'm not saying ignore the issue, or don't think about it at all, but the idea that it's some urgent issue that we absolutely have to invest heavily in right now IMHO doesn't stand.
You might totally be right.
The problem is, humanity is pretty bad at estimating when a new technology is about to be ready. Even the experts are pretty bad. See e.g. multiple people saying human flight is many years away, including some who were saying it unknowingly after it had been achieved!
So I think the question is: when should we worry about ai safety? How early is too early?
Plenty of experts think now is a good time to be spendning at least some resources on this. And we're not talking nuclear fusion level resources here; I don't know the numbers, but I imagine the amount of resources are actually fairly tiny compared to almost all other scientific spending. (And this was certainly true 10 years ago.)
I personally think that, given the chance of AI systems being an existential risk, we should be putting way more resources into it than into other things. The same goes true for preventing pandemics or other biological x-risks, etc.
The problem is, humanity is pretty bad at estimating when a new technology is about to be ready. Even the experts are pretty bad. See e.g. multiple people saying human flight is many years away, including some who were saying it unknowingly after it had been achieved!
So I think the question is: when should we worry about ai safety? How early is too early?
Plenty of experts think now is a good time to be spendning at least some resources on this. And we're not talking nuclear fusion level resources here; I don't know the numbers, but I imagine the amount of resources are actually fairly tiny compared to almost all other scientific spending. (And this was certainly true 10 years ago.)
I personally think that, given the chance of AI systems being an existential risk, we should be putting way more resources into it than into other things. The same goes true for preventing pandemics or other biological x-risks, etc.
> Plenty of experts think now is a good time to be spendning at least some resources on this. And we're not talking nuclear fusion level resources here; I don't know the numbers, but I imagine the amount of resources are actually fairly tiny compared to almost all other scientific spending. (And this was certainly true 10 years ago.)
The risk theses experts are talking about isn't existential. Its stuff like GPT3 being truncated to generate propaganda for dictatorships, deepfakes, algorithm-reinforced biases in all kinds of models (because the data used to train them was generated by humans). This is concerning to some degree and deserves funding and research (though all too often the results boil down to unusable advice and guidelines). This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition.
The risk theses experts are talking about isn't existential. Its stuff like GPT3 being truncated to generate propaganda for dictatorships, deepfakes, algorithm-reinforced biases in all kinds of models (because the data used to train them was generated by humans). This is concerning to some degree and deserves funding and research (though all too often the results boil down to unusable advice and guidelines). This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition.
> This is entirely orthogonal to any existential risk a supposed AGI might create. One is a testable scientific hypothesis and the other is a futurist horror story which is untestable by definition.
There is no difference in their testability, it's just that the existential risk test means no more humans to care about the results.
The danger of a biolab creating a pathogen that wipes out all humanity is real but remote, and is a simple extension of the argument that the lab could create pathogens that can "only" kill or harm millions. That's why biolab safety is carefully studied, and the same argument applies to AI safety.
There is no difference in their testability, it's just that the existential risk test means no more humans to care about the results.
The danger of a biolab creating a pathogen that wipes out all humanity is real but remote, and is a simple extension of the argument that the lab could create pathogens that can "only" kill or harm millions. That's why biolab safety is carefully studied, and the same argument applies to AI safety.
> With general, or strong AI we only just barely have the beginnings of an idea of how to even do preliminary research on it.
Maybe. Or maybe we're just on small change away from AGI. We could be, because we don't yet really know what's required and we may not ever know until after we've done it. That prospect should be terrifying.
> It's far, far too early to start worrying about controls and management strategies for the end point of that research.
No, because if you create AGI before we have some containment or safety protocols, then it may already be too late.
Consider whether we should have researched biolab safety before we started experimenting on modifying pathogens in various ways. Doesn't that just seem blindingly brain-dead obvious? For some reason this clear and obvious conclusion just goes out the window when it comes to AI.
Maybe. Or maybe we're just on small change away from AGI. We could be, because we don't yet really know what's required and we may not ever know until after we've done it. That prospect should be terrifying.
> It's far, far too early to start worrying about controls and management strategies for the end point of that research.
No, because if you create AGI before we have some containment or safety protocols, then it may already be too late.
Consider whether we should have researched biolab safety before we started experimenting on modifying pathogens in various ways. Doesn't that just seem blindingly brain-dead obvious? For some reason this clear and obvious conclusion just goes out the window when it comes to AI.
> If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
I have yet to see anyone who argues this point show sufficient knowledge of the current state of AI research to credibly make it. If you live in the fantasy world where AI is just an arbitrary construct that can do whatever task by itself it is an easy point to make but anyone who is down and getting their hands dirty in the pile of linear algebra that makes up current AI knows that there is not even a credible pathway to creating an AGI in current research (i.e. you cannot optimize for a task you don't know at train time and if you are optimizing for a specific task you are not creating a general AI).
I have yet to see anyone who argues this point show sufficient knowledge of the current state of AI research to credibly make it. If you live in the fantasy world where AI is just an arbitrary construct that can do whatever task by itself it is an easy point to make but anyone who is down and getting their hands dirty in the pile of linear algebra that makes up current AI knows that there is not even a credible pathway to creating an AGI in current research (i.e. you cannot optimize for a task you don't know at train time and if you are optimizing for a specific task you are not creating a general AI).
> If you live in the fantasy world where AI is just an arbitrary construct that can do whatever task by itself it is an easy point to make but anyone who is down and getting their hands dirty in the pile of linear algebra that makes up current AI knows that there is not even a credible pathway to creating an AGI in current research (i.e. you cannot optimize for a task you don't know at train time and if you are optimizing for a specific task you are not creating a general AI).
Humans are optimized for the specific task of reproducing, and yet we evolved general intelligence. "General intelligence" doesn't have to be a goal for it to emerge.
Humans are optimized for the specific task of reproducing, and yet we evolved general intelligence. "General intelligence" doesn't have to be a goal for it to emerge.
How about a simple example - train a model on outputting code which:
- scans open network ports through google dns
- copies itself and the model to that location
- whenever it detects a sigkill or file delete, scan again and save itself somewhere else
Now you've got an autonomous virus that will never die. Add any other power-seeking and self-replicating strategies using AI modules - the task could be "generate code to run as many duplicate processes as possible" or "generate code to find other floating AI modules like me and share model weights with them". Codex might not be there yet but we're getting close.
- scans open network ports through google dns
- copies itself and the model to that location
- whenever it detects a sigkill or file delete, scan again and save itself somewhere else
Now you've got an autonomous virus that will never die. Add any other power-seeking and self-replicating strategies using AI modules - the task could be "generate code to run as many duplicate processes as possible" or "generate code to find other floating AI modules like me and share model weights with them". Codex might not be there yet but we're getting close.
Can I tell you something secret? The "AGI" of the form you are describing is already here! Not only that, it's been here since 1988! Crazy right? Just check out this link if you don't believe me: https://en.wikipedia.org/wiki/Morris_worm
(Snark aside, I've realized this type of interactions have always resulted in a very low perception of EA for me. I don't come out of these conversations thinking "what a clueless thing to say by this person who also happened to support EA", I think "wow, EA is full with looney bins who don't know what they are talking about." If you are really into giving what you can and aren't sure why EA gets a bad rep, look no further.)
(Snark aside, I've realized this type of interactions have always resulted in a very low perception of EA for me. I don't come out of these conversations thinking "what a clueless thing to say by this person who also happened to support EA", I think "wow, EA is full with looney bins who don't know what they are talking about." If you are really into giving what you can and aren't sure why EA gets a bad rep, look no further.)
I've heard of some of the famous viruses which are obviously more complicated than what I mentioned. But that's the difference - they are hardcoded to perform these particular steps. I'm not saying AGI is the code that does this. AGI is one level higher - a more meta objective - given the objective of "accumulating power", it will learn lower level objectives like "replicating and spreading", which leads to the final output like this code.
Basically, AGI could unleash a thousand "morris worms" - and that would only be for the objective of replicating and spreading. There could be other objectives as well.
Basically, AGI could unleash a thousand "morris worms" - and that would only be for the objective of replicating and spreading. There could be other objectives as well.
It's easy to justify almost any sort of behavior if you take some risk and assign it a severity of "infinity". A 1% chance of happening? At severity infinity, you're morally obligated to act at even a 0.00001% chance: 0.00001% times infinity is still infinity. That's the AGI logic I'm seeing from these people.
No, that's a strawman. The severity of human extinction is not infinite. For instance, it would be preferable for humanity to go extinct than for every human to be tortured for every minute of their life, and since there is no number larger than infinity, the severity of human extinction must be finite.
This tells me that you haven't seriously engaged with the arguments over AGI. I'm sure there's plenty of nonsense armchair philosophising happening, but you don't take all the "quantum woo" that's out there as some indictment of real quantum mechanics, do you?
This tells me that you haven't seriously engaged with the arguments over AGI. I'm sure there's plenty of nonsense armchair philosophising happening, but you don't take all the "quantum woo" that's out there as some indictment of real quantum mechanics, do you?
You can replace "infinity" with any suitably huge number; I don't need to quibble over tenths, hundredths, and thousandths of a percentile risk that something is going to happen.
You absolutely do have to quibble over those numbers. Or do you not think we should be tracking asteroids either because they only kill 95% of all life once every few hundred million years?
I do not think people should be directing money to asteroid charities, no. It's their money, they can do what they want, but if their discipline demands that they do that, it's a bad discipline.
So to make sure I get this straight, if a plausible calculation suggested that we could see an extinction-level asteroid hurtling towards us in the next hundred years, you don't think that asteroid detection is worth funding via any means, like charities or governments?
See above.
Blows my mind that anyone would actually accept such a conclusion, but you do you I guess. Seriously undermines your criticisms of EA, IMO.
I'm pretty happy with what it says about my critique, too. For instance, my side of this "debate", such as it is, will never spend $100,000 funding a tabletop role playing game aiming to educate people on the dangers of asteroids.
> I'm pretty happy with what it says about my critique, too. For instance, my side of this "debate", such as it is, will never spend $100,000 funding a tabletop role playing game aiming to educate people on the dangers of asteroids.
I'm not sure how you can be so confident that raising awareness wouldn't have beneficial downstream effects. I'm skeptical too, but outright dismissal doesn't seem warranted. Then again, you are downright dismissive about the very real and scientifically proven dangers of asteroids wiping out most of life on Earth, so I suppose this is pretty mild by comparison.
I'm not sure how you can be so confident that raising awareness wouldn't have beneficial downstream effects. I'm skeptical too, but outright dismissal doesn't seem warranted. Then again, you are downright dismissive about the very real and scientifically proven dangers of asteroids wiping out most of life on Earth, so I suppose this is pretty mild by comparison.
Both those 1%s are very unlikely to be anywhere close to being true.
The sci-fi version of AGI that people are imaging now will never exist. No one has any idea how to create AGI now, and therefore, no one knows what it might actually look like if it ever does. Any “solutions” people come up with now will have nothing to do with the problems a real AGI might pose.
If people want to get together and talk about this stuff, that’s great. But if someone wants you to give money to fund research into this, you’re being scammed.
The sci-fi version of AGI that people are imaging now will never exist. No one has any idea how to create AGI now, and therefore, no one knows what it might actually look like if it ever does. Any “solutions” people come up with now will have nothing to do with the problems a real AGI might pose.
If people want to get together and talk about this stuff, that’s great. But if someone wants you to give money to fund research into this, you’re being scammed.
Exactly. Researching AI safety today is like researching how to reduce the proliferation of black hole bombs.
What would it make sensible to start researching the proliferation of black hole bombs, for you?
Well, opinions differ. I'd guess the probability of there being AGI as in computers able to think as well as a human in most ways by 2300 as close to 100%.
Things we have now are getting closer eg Deepmind's Gato and GPT-3.
Things we have now are getting closer eg Deepmind's Gato and GPT-3.
> Things we have now are getting closer eg Deepmind's Gato and GPT-3.
For someone who actually works in the field, let me inform you that you've been had by the marketing folks. Deepmind's Gato is as close to AGI as connecting a line of cars with chains and calling it a train. Gato is glorified if else statement over multiple models, and not really "general" at all.
For someone who actually works in the field, let me inform you that you've been had by the marketing folks. Deepmind's Gato is as close to AGI as connecting a line of cars with chains and calling it a train. Gato is glorified if else statement over multiple models, and not really "general" at all.
Absolutely none of the recent interesting machine-learning successes are even going in the right direction to ever become AGI.
There is as much similarity between GPT-3 and an AGI as there is between a human and one of those novelty fortune-telling machines with racist caricatures and names like Zardoz.
There is as much similarity between GPT-3 and an AGI as there is between a human and one of those novelty fortune-telling machines with racist caricatures and names like Zardoz.
There's nothing substantive in this post, just unexplained opinions...
The main substantive point I made is: "No one has any idea how to create AGI now"
Correct, no?
The rest follows by basic logic. You can't craft solutions to problems you don't know anything about. You can't assign probabilities to outcomes without any information on the inputs or mechanisms of the system.
Correct, no?
The rest follows by basic logic. You can't craft solutions to problems you don't know anything about. You can't assign probabilities to outcomes without any information on the inputs or mechanisms of the system.
Thank you for engaging with me. I have two questions.
1.
"No one has any idea how to create X now" does not lead to "Imminent creation of X is very unlikely to be true". Right?
2.
You can always assign probabilities. I'll give you an example: say I'm a scary dictator and I ask you: "is A or B correct?" You have no idea what I mean by A or B, so you'd assign 50-50 if you had to bet something dear on the outcome of my totally unfair context-free question. Perhaps you happen to know of research showing that people more often set A as the correct answer in this sort of game, so you might assign 60-40, let's say. That's a probability estimate!!
Same principle for AI. You don't know much, but what you do know can still be used for a judgment. Supposing you do not believe in souls, then brains are just computers, then it's possible in principle for a computer to become sentient and intelligent, then it's just a matter of time 'til they do, whether it happens 10 years from now or 1000. Maybe you feel the human race will go extinct shortly, and that'll be another valid factor in your estimate of how much this is to worry about.
But either you accept that it CAN happen and discuss how many precautions to take today, or you're understandably unconcerned because it appears prima facie impossible to you. I imagine that for you to be unconcerned, you'd have to have judged that a being cannot create a being more intelligent than itself. (If you're using a different reasoning, please enlighten me)
But if that's your reasoning... what did evolution do? It has no brain, yet it created an intelligent being. And if you carve out a special clause for evolution-like processes, humans could still just simulate an accelerated evolution in a lab/computer and wait for a superintelligence to pop out. Does that not make sense?
1.
"No one has any idea how to create X now" does not lead to "Imminent creation of X is very unlikely to be true". Right?
2.
You can always assign probabilities. I'll give you an example: say I'm a scary dictator and I ask you: "is A or B correct?" You have no idea what I mean by A or B, so you'd assign 50-50 if you had to bet something dear on the outcome of my totally unfair context-free question. Perhaps you happen to know of research showing that people more often set A as the correct answer in this sort of game, so you might assign 60-40, let's say. That's a probability estimate!!
Same principle for AI. You don't know much, but what you do know can still be used for a judgment. Supposing you do not believe in souls, then brains are just computers, then it's possible in principle for a computer to become sentient and intelligent, then it's just a matter of time 'til they do, whether it happens 10 years from now or 1000. Maybe you feel the human race will go extinct shortly, and that'll be another valid factor in your estimate of how much this is to worry about.
But either you accept that it CAN happen and discuss how many precautions to take today, or you're understandably unconcerned because it appears prima facie impossible to you. I imagine that for you to be unconcerned, you'd have to have judged that a being cannot create a being more intelligent than itself. (If you're using a different reasoning, please enlighten me)
But if that's your reasoning... what did evolution do? It has no brain, yet it created an intelligent being. And if you carve out a special clause for evolution-like processes, humans could still just simulate an accelerated evolution in a lab/computer and wait for a superintelligence to pop out. Does that not make sense?
I will only respond to 1.
> "No one has any idea how to create X now" does not lead to "Imminent creation of X is very unlikely to be true". Right?
Wrong. Here's the logical chain:
1. For someone to do X, they will need to know how to do X. (Unless you think it's all just alchemy, and we're boiling urine to distill gold.)
2. For creation of X to be imminent, someone's knowledge of how to create X will have to be almost complete.
3. Currently, no one knows how to do X.
4. Thus, creation of X is nowhere near imminent.
> "No one has any idea how to create X now" does not lead to "Imminent creation of X is very unlikely to be true". Right?
Wrong. Here's the logical chain:
1. For someone to do X, they will need to know how to do X. (Unless you think it's all just alchemy, and we're boiling urine to distill gold.)
2. For creation of X to be imminent, someone's knowledge of how to create X will have to be almost complete.
3. Currently, no one knows how to do X.
4. Thus, creation of X is nowhere near imminent.
> "No one has any idea how to create X now" does not lead to "Imminent creation of X is very unlikely to be true". Right?
I think it does lead to that. How will people create something they have no idea how to create... Perhaps it will somehow emerge accidentally, but that's very unlikely.
I tried to make this clear before, but I'll state it clearly: I think discussing is completely fine. We all have our hobbies. It's convincing people there's a real problem and taking real money to solve a fake problem that I object to. That's a con.
If AGI does happen and they start doing things we GIs don't like, the problem is going to be around specifically what they are doing, why they are doing it and what we don't like about it. The solutions are going to be in terms of those specific problem spaces. There would also be a feedback loop of response, counter-response, which is fundamentally unpredictable.
> evolution-like processes
Given a planet-sized environment and 4-5 billion years, I think we can say emergent GI is possible. But that doesn't help someone with a lab. I think we'd do better by studying how brains work, really. I think we're probably going to do best by starting with simpler brains and see if we can trace the path evolution has take (thus saving ourself billions of years of random trial and error).
> That's a probability estimate!!
Let me wrap up on this: I estimate a 99.99% likelihood that people discussing the problems of AGI now will fail to correctly predict the nature of AGI and their resultant problems, and therefore will waste any and all effort/money spent now in that direction.
Someone actually seriously interested in this who wants to put resources to it now would want to put money into a long-term trust, to be used once the problem is clear. However, that assumes the existence of a stable trustee mechanism that remains uncorrupted over the long term to ensure the money is used as intended. I'm not how that is accomplished, but some long-term foundations seem to have done OK. (E.g., the Nobel Foundation doesn't do too bad. The Peace Prize is all politics, and all of the prizes have some politics, but overall it's mostly on-track.)
If there's a 10% chance such a trust function well enough long enough for 50% of the money to be spent in search of solutions to the actual problems, that's 5% effectiveness. Far better than the 0.01% effectiveness I estimate for spending the money now.
Note: the ancestor post of this discussion also estimated a 0.01% effectivness:
> If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
Still, I think an EA could easily come up with another use for the same resources that does a lot better than 5%, so we're back to not spending any resources on this now. Much better to have future EAs work on it, once the problems are clear.
I think it does lead to that. How will people create something they have no idea how to create... Perhaps it will somehow emerge accidentally, but that's very unlikely.
I tried to make this clear before, but I'll state it clearly: I think discussing is completely fine. We all have our hobbies. It's convincing people there's a real problem and taking real money to solve a fake problem that I object to. That's a con.
If AGI does happen and they start doing things we GIs don't like, the problem is going to be around specifically what they are doing, why they are doing it and what we don't like about it. The solutions are going to be in terms of those specific problem spaces. There would also be a feedback loop of response, counter-response, which is fundamentally unpredictable.
> evolution-like processes
Given a planet-sized environment and 4-5 billion years, I think we can say emergent GI is possible. But that doesn't help someone with a lab. I think we'd do better by studying how brains work, really. I think we're probably going to do best by starting with simpler brains and see if we can trace the path evolution has take (thus saving ourself billions of years of random trial and error).
> That's a probability estimate!!
Let me wrap up on this: I estimate a 99.99% likelihood that people discussing the problems of AGI now will fail to correctly predict the nature of AGI and their resultant problems, and therefore will waste any and all effort/money spent now in that direction.
Someone actually seriously interested in this who wants to put resources to it now would want to put money into a long-term trust, to be used once the problem is clear. However, that assumes the existence of a stable trustee mechanism that remains uncorrupted over the long term to ensure the money is used as intended. I'm not how that is accomplished, but some long-term foundations seem to have done OK. (E.g., the Nobel Foundation doesn't do too bad. The Peace Prize is all politics, and all of the prizes have some politics, but overall it's mostly on-track.)
If there's a 10% chance such a trust function well enough long enough for 50% of the money to be spent in search of solutions to the actual problems, that's 5% effectiveness. Far better than the 0.01% effectiveness I estimate for spending the money now.
Note: the ancestor post of this discussion also estimated a 0.01% effectivness:
> If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
Still, I think an EA could easily come up with another use for the same resources that does a lot better than 5%, so we're back to not spending any resources on this now. Much better to have future EAs work on it, once the problems are clear.
> If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
Sure, but there is no reason to think these numbers are true. Maybe AI alignment research increases chances of AI killing everyone.
Sure, but there is no reason to think these numbers are true. Maybe AI alignment research increases chances of AI killing everyone.
That’s just a common disagreement then, about priorities and decisions made on uncertain information.
No one gets to do a double-blind study on the measured probability of catastrophic AI accidents in worlds with and without safety research in advance of capabilities gain.
No one gets to do a double-blind study on the measured probability of catastrophic AI accidents in worlds with and without safety research in advance of capabilities gain.
There are reasons to think we can do something about AI alignment, but there's zero chance of being able to get anywhere on that topic with dozens of hours of conversation. It's complicated.
I'm not sure that you can square the "EA is precisely donating to best reviewed charities" and "tens of millions to AI safety is money well spent."
A huge problem is that people are developing this longtermism prediction model based on nothing more than sci-fi books and hunches. You need only look at how seriously places like SSC take Roko's Basilisk to see that they aren't modeling the future of AI in a reasonable way.
It is like donating to an organization that will prevent Vulcan, God of Volcanoes from destroying civilization.
A huge problem is that people are developing this longtermism prediction model based on nothing more than sci-fi books and hunches. You need only look at how seriously places like SSC take Roko's Basilisk to see that they aren't modeling the future of AI in a reasonable way.
It is like donating to an organization that will prevent Vulcan, God of Volcanoes from destroying civilization.
I don't want people who lie, cheat, steal and manipulate anywhere near AI safety. I don't want them to do the research and I don't want them to fund it. Because I don't want the AI to be made in their image and have those values. An AI that breaks all sorts of rules "for the greater good" is about the most dangerous idea I can think.
That's a might "if" to rest on, given the self-identification of the modal effective altruist as "rational."
> If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
This line of thinking leads to some problems. There's always some threshold of likelihood where if it provide some arbitrarily high enough good/harm reduction that lets you rationalize basically any possible position. For instance, give me all your money, I'll give you 8% monthly returns!
This line of thinking leads to some problems. There's always some threshold of likelihood where if it provide some arbitrarily high enough good/harm reduction that lets you rationalize basically any possible position. For instance, give me all your money, I'll give you 8% monthly returns!
>If AGI has even a 1% chance of happening this century and those donations reduce the risk of it going wrong and killing everyone by 1% that's still money very well spent.
Not much of an AGI if it's not Intelligent enough to ignore a few "Don't say these bad words" rules.
Not much of an AGI if it's not Intelligent enough to ignore a few "Don't say these bad words" rules.
Did you read the article linked by OP? It addresses these points quite directly and highlights the issues with them.
This seems a bit unfair
> they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
Most groups of people are like this to some extent. Most corporations are like this. Cultiness is a spectrum, and the comparison to scientology is a bit much as scientology does much worse things than that.
> Over and over I keep coming across EA arguments that could be used to justify almost literally any behavior, so long as you can tell yourself a story about having a long-term positive goal.
What you describe is a little too outcome based for my preferences, but you act as if outcome based morality isn't a common view. That viewing your moral standing as what the result of your actions are; how much happiness you have caused minus how much suffering is instrinsically unreasonable.
As far as "telling yourself a story" - that's how all morality works. There is no santa clause making a list of naughty or nice. You have to judge for yourself. Sure there are other metrics. I personally value intention, acting with honour, and ensuring the things you do can be universalized, but at the end of the day i am the one who has to verify of i a living up to my own moral precepts. If i wanted to delude myself into self-justifying something bad, it would probably be pretty easy as there is nobody but myself to stop me. I would even argue that is what being moral is: how resistant you are to petty self-justification.
> they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
Most groups of people are like this to some extent. Most corporations are like this. Cultiness is a spectrum, and the comparison to scientology is a bit much as scientology does much worse things than that.
> Over and over I keep coming across EA arguments that could be used to justify almost literally any behavior, so long as you can tell yourself a story about having a long-term positive goal.
What you describe is a little too outcome based for my preferences, but you act as if outcome based morality isn't a common view. That viewing your moral standing as what the result of your actions are; how much happiness you have caused minus how much suffering is instrinsically unreasonable.
As far as "telling yourself a story" - that's how all morality works. There is no santa clause making a list of naughty or nice. You have to judge for yourself. Sure there are other metrics. I personally value intention, acting with honour, and ensuring the things you do can be universalized, but at the end of the day i am the one who has to verify of i a living up to my own moral precepts. If i wanted to delude myself into self-justifying something bad, it would probably be pretty easy as there is nobody but myself to stop me. I would even argue that is what being moral is: how resistant you are to petty self-justification.
Please do not forget that you can disapprove of all the longtermism stuff and of the alleged cult-like structure, and still think it's important to pledge your money to charity.
Please don't let the bad publicity around EA stop you from donating what you can, to charities that you trust and have a proven record of efficacy, whether from EA-related organization or other research. The original message still stands and should keep standing.
Please don't let the bad publicity around EA stop you from donating what you can, to charities that you trust and have a proven record of efficacy, whether from EA-related organization or other research. The original message still stands and should keep standing.
If anything, EA seems like an obstacle to that kind of common-sense thinking. I could donate to charity. But: if holding on to the money made it more likely that I'd earn more money later, I could donate even more money to charity. I'm thus ethically obligated not to donate. See, for this, William MacAskill's conversation with Sam Bankman-Fried where he urged him to take a job at Jane Street to maximize his income.
If you have an idea which cause to spend on and you have enough savings to live for 6 months in case of problems, EA generally recommends to donate:
https://80000hours.org/articles/should-you-wait/
https://80000hours.org/articles/should-you-wait/
... so does Catholicism. It's the rest of what EA (and Catholicism) has to say that gets us into trouble. (I'm Catholic, fwiw).
I'm not very close to the core of EA, so maybe my perspective is not the most accurate, but one thing that I feel sets EA apart from other movements is that EA people generally are happy to disagree and change their mind based on good ideas/arguments/evidence from anyone inside our outside EA. For example:
https://80000hours.org/about/credibility/evaluations/mistake... https://80000hours.org/2020/02/anonymous-answers-flaws-80000...
I expect and hope that people within EA will keep learning from mistakes and changing their views.
https://80000hours.org/about/credibility/evaluations/mistake... https://80000hours.org/2020/02/anonymous-answers-flaws-80000...
I expect and hope that people within EA will keep learning from mistakes and changing their views.
Well, right today there's an article on the economist, one on the guardian, two things on the front page of hacker news, etc. This is a debate, with clear criticism of their current views. Let's see.
I see billionaire individuals and organised charities as polar extremes of the same policy failure. Neither should exist, and when they do it should be as an ephemeral anomaly.
Any charity which fails to prioritise putting itself out of business is functionally a grift.
If society truly valued what charities do, their activities would be intrinsically valuable, and not require a special economic status to be conjured for them to be viable.
Any charity which fails to prioritise putting itself out of business is functionally a grift.
If society truly valued what charities do, their activities would be intrinsically valuable, and not require a special economic status to be conjured for them to be viable.
People have this idea that you can just solve problems like world hunger by waving a magic wand, but you can’t. There are probably South Sentinelese starving right now, we can’t do anything about that because they won’t let us. The little world hunger that is left today after the advancements of the 20th century is almost entirely in war-torn messes, and you can’t eliminate war unless you eliminate human free will
Or they are considered valuable but we have a captive market… Many people would like charities to become institutional features instead of charities but then there are other interests who are against such things because then “muh capital”
Charities, as they exist today, serve as a void into which corporations and governments can abdicate their respective responsibilities. Continuing to tolerate charities, at least in their current form, is complicity.
> charities that you trust and have a proven record of efficacy
Yet to find one since I started googling their financials.
Yet to find one since I started googling their financials.
Against malaria foundation (you know they are good because their website looks like it was made in the last century)
There are surely two separate issues here? One is whether it's bad to eg. take first world crypto investors' money and use it to stop hundreds of thousands of people dying of preventable diseases in developing countries. Personally, I think not really, or at least it'd be an odd thing to be upset about? Happy to hear the arguments in favour of huge numbers of preventable deaths.
Where that narrative gets problematic wrt SBF is that, aiui in the eyes of EA "leadership", "too many" EA individuals choose to allocate their donations to boring global health causes, rather than building the movement or weird long-termist stuff. So with that in mind, the obvious decision was to take the friendly billionaire money and put it into those "important" areas. Which means now, we have a popular and hard to refute characterization of EA as an organization that raises money for itself and weird causes normal people don't care about. But, like I said, the majority of EA money goes towards global health, ie. stopping people dying.
Where that narrative gets problematic wrt SBF is that, aiui in the eyes of EA "leadership", "too many" EA individuals choose to allocate their donations to boring global health causes, rather than building the movement or weird long-termist stuff. So with that in mind, the obvious decision was to take the friendly billionaire money and put it into those "important" areas. Which means now, we have a popular and hard to refute characterization of EA as an organization that raises money for itself and weird causes normal people don't care about. But, like I said, the majority of EA money goes towards global health, ie. stopping people dying.
One is whether it's bad to eg. take first world Professional software developers' money and use it to stop hundreds of thousands of people dying of preventable diseases in developing countries. Personally, I think not really, or at least it'd be an odd thing to be upset about?
I think you're making that comment with the idea that it's self-evidently wrong? Do I understand you correctly?
I find it really weird that people are so glib about the "hundreds of thousands of preventable deaths" part. I don't get how on earth people weigh that as less important than the financial welfare of relatively well-off people in richer countries. Maybe this is some basic failure of humanity on my part.
I find it really weird that people are so glib about the "hundreds of thousands of preventable deaths" part. I don't get how on earth people weigh that as less important than the financial welfare of relatively well-off people in richer countries. Maybe this is some basic failure of humanity on my part.
The problem (and I think roughly the point being made) is that if you follow that logic through, then surely almost everyone here should have the vast majority of their possessions & wealth taken, to fund the equalising up-lifting of others.
Charity and world aid is in a continual discussion & discovery of what we collectively think is 'good enough', helping people get to something that is perhaps 'not disaster', but nevertheless falls far short of the standards the gifters expect for themselves.
Charity and world aid is in a continual discussion & discovery of what we collectively think is 'good enough', helping people get to something that is perhaps 'not disaster', but nevertheless falls far short of the standards the gifters expect for themselves.
'Possessions and wealth taken' and 'fund the equalizing up-lifting of others' co-occur, but that doesn't mean the latter is justifying the former. People who play the markets know they're taking a risk and willing to gamble on being smarter than their competition, they're consenting to have their money go to someone with better judgement than them and that's what justifies having their wealth taken. The winners of a game like that are exactly who I'd want funding the 'up-lifting' part. That doesn't mean it's kosher for the smartest person to expropriate every above-average person's money when they aren't willing to gamble with it.
Did SBF take the wealth using better judgement or did he take it using dishonesty and fraud?
My impression is that it's the dishonesty and theft that people have a problem with, not just being clever and outwitting people in the speculative gambling game.
And when those people get the impression that the EA movement is okay with getting money dishonestly as long as it's given it to a worthy cause, they take issue with EA.
Edit: Just thought of an analogy. Some charities do raffles, right? You can put money into the pot for a chance to win a prize, knowing that a proportion of all the money people put in goes to the charity.
But what if the charity sold a bunch of raffle tickets and then announced that they would not award a prize and all the money would go to the charity? Or what if it came out that the prize was rigged to go to someone affiliated with the charity? In this case I don't think you can say, "Well, those people chose to gamble so they consented to having their wealth taken."
My impression is that it's the dishonesty and theft that people have a problem with, not just being clever and outwitting people in the speculative gambling game.
And when those people get the impression that the EA movement is okay with getting money dishonestly as long as it's given it to a worthy cause, they take issue with EA.
Edit: Just thought of an analogy. Some charities do raffles, right? You can put money into the pot for a chance to win a prize, knowing that a proportion of all the money people put in goes to the charity.
But what if the charity sold a bunch of raffle tickets and then announced that they would not award a prize and all the money would go to the charity? Or what if it came out that the prize was rigged to go to someone affiliated with the charity? In this case I don't think you can say, "Well, those people chose to gamble so they consented to having their wealth taken."
To my admittedly shallow understanding SBF was gambling with his clients' money and lost. I wouldn't say any charitable work he did would justify the theft, but I also have no qualms about Binance crashing his token so he'd be caught in the act.
To put it in point form, I'd say -SBF had bad judgement and bad morals -SBF's clients had bad judgement and bad luck -Binance had good judgement and good luck, and if the people who make it up do a lot of charity work, they have good morals as well. That doesn't help SBF's clients of course, but I do think they consented to a game where a reasonable person would have known this was a possible outcome. I would think most EA folks would tend to agree with how I've characterized it, but maybe someone more versed in that subculture's overton window can clarify.
To put it in point form, I'd say -SBF had bad judgement and bad morals -SBF's clients had bad judgement and bad luck -Binance had good judgement and good luck, and if the people who make it up do a lot of charity work, they have good morals as well. That doesn't help SBF's clients of course, but I do think they consented to a game where a reasonable person would have known this was a possible outcome. I would think most EA folks would tend to agree with how I've characterized it, but maybe someone more versed in that subculture's overton window can clarify.
> SBF was gambling with his clients' money and lost
> I do think they consented to a game where a reasonable person would have known this was a possible outcome
I think you might be confusing the two different types of clients who were involved here.
My understanding is that SBF initially lost money that had been entrusted to him by investors in his "please gamble with my money and give me the winnings" company (Alameda). I would not call that fraud and I agree that the Alameda investors consented (foolishly) to this use of their money.
The problem is that after SBF lost the money of Alameda clients by doing risky gambles that went wrong, he tried to cover it up by stealing money that had been entrusted to him by clients of his "please hold onto my money for me so I can conveniently trade on your platform" company (FTX).
I don't think the FTX clients consented to SBF's use of their money to do gambling. They put their money on FTX so they could do their own gambling.
While I think FTX clients should have known better than to put money on a crypto exchange for safekeeping, that doesn't change the fact that their money was misappropriated nor does it excuse the fraud.
> I do think they consented to a game where a reasonable person would have known this was a possible outcome
I think you might be confusing the two different types of clients who were involved here.
My understanding is that SBF initially lost money that had been entrusted to him by investors in his "please gamble with my money and give me the winnings" company (Alameda). I would not call that fraud and I agree that the Alameda investors consented (foolishly) to this use of their money.
The problem is that after SBF lost the money of Alameda clients by doing risky gambles that went wrong, he tried to cover it up by stealing money that had been entrusted to him by clients of his "please hold onto my money for me so I can conveniently trade on your platform" company (FTX).
I don't think the FTX clients consented to SBF's use of their money to do gambling. They put their money on FTX so they could do their own gambling.
While I think FTX clients should have known better than to put money on a crypto exchange for safekeeping, that doesn't change the fact that their money was misappropriated nor does it excuse the fraud.
Ahh the "she was practically begging for it with that short skirt" argument.
> The problem (and I think roughly the point being made) is that if you follow that logic through, then surely almost everyone here should have the vast majority of their possessions & wealth taken, to fund the equalising up-lifting of others.
And if not software developers why not? Who gets to chose which grift is a grift and which is an honest living?
> I find it really weird that people are so glib about the "hundreds of thousands of preventable deaths" part.
So you agree - software devlopers, who make vastly more than the average person should have their wealth siezed to save those people? Or are you the one being glib aboout saving lives?
And if not software developers why not? Who gets to chose which grift is a grift and which is an honest living?
> I find it really weird that people are so glib about the "hundreds of thousands of preventable deaths" part.
So you agree - software devlopers, who make vastly more than the average person should have their wealth siezed to save those people? Or are you the one being glib aboout saving lives?
Have you ever been close to anybody who died? If you have prevented that, and then you multiply that by a few hundred thousand, do you not get close to a place where rich people's investments not panning out doesn't seem like something to get too upset about?
It's interesting to me that there's a kind of horseshoe effect here with the "rich people must be protected" on the same side as the "rich people are evil" people.
It's interesting to me that there's a kind of horseshoe effect here with the "rich people must be protected" on the same side as the "rich people are evil" people.
> do you not get close to a place where rich people's investments not panning out
Rich people like software developers
And investments like every cent they earn
Absoultely 10000% happy with taking every cent of it and saving as many lives as possible. Why are you against this?
Rich people like software developers
And investments like every cent they earn
Absoultely 10000% happy with taking every cent of it and saving as many lives as possible. Why are you against this?
Humans have very strong instincts against exploitation by peers. I think the appropriate answer here is: "Sure, let's seize the wealth of approximately everyone in a western country." Singling out a particular group, even though it's less seizure in total, raises people's hackles for reasons entirely unrelated to whether redistribution is a good idea.
Like the around 88% of American drivers considered themselves to be above average at driving - everyone also believes that they are the exception. As for redistribution of wealth, even many multi millionaires consider themselves to be on the poor side of the wealth divide.
> > I find it really weird that people are so glib about the "hundreds of thousands of preventable deaths" part.
> So you agree -
You're quoting someone else there, not me (GP).
> So you agree -
You're quoting someone else there, not me (GP).
Been saying the same thing. EA functionally is the same pattern I grew up around among evangelical extremists. It's an all purpose source of self righteous authority based on theoretical virtue erasing actual harm.
Ah yes, but if there's an even 1% chance the good they do outweighs the harm and saves some portion of some humans in some amount of time... er... or something.
Currently the #1 charity recommended by GiveWell is medicine to prevent malaria. I think it has more than 1% chance to help people.
Perhaps some people should read https://www.givewell.org/charities/top-charities before expressing strong opinions on how EA is all about robots.
Perhaps some people should read https://www.givewell.org/charities/top-charities before expressing strong opinions on how EA is all about robots.
Is it theoretical? In other words, are you making a positive assertion here that the benefit of EA is not actual, and if so, can you cite evidence?
Are you describing "giving virtually all of [your earnings] away to effective charities" as "literally buying indulgences"? That's literally the most cynical thing I've ever heard.
I can understand having a negative opinion of the EA movement as it exists today. But your reaction to the charitable investor thing makes me think that you actually disagree with... altruism. Or at least the idea that being more altruistic is morally superior than being less altruistic.
I can understand having a negative opinion of the EA movement as it exists today. But your reaction to the charitable investor thing makes me think that you actually disagree with... altruism. Or at least the idea that being more altruistic is morally superior than being less altruistic.
No, I pasted a quote from an extremely respected thinker who, in the context of Effective Altruism, said that a financier who gave away virtually all his earnings would be morally superior to most of us. Giving away money is a good thing, everybody does it, by all means do it more. But the idea that you can purchase moral superiority, and, moreover, that the superiority you obtain is proportionate to the amount you donate relative to the average person? That is literally the logic of indulgences.
You've skipped the words "way ahead of most of us" in the quote I provided, is what I think happened.
You've skipped the words "way ahead of most of us" in the quote I provided, is what I think happened.
It’s not the amount you donate relative to others, it’s the amount you donate relative to how much you are capable of donating. Not many people would work a high-stress, high-barrier-to-entry job and give away almost everything they earned, while their peers live richly.
Even if we accept that the average person usually acts morally when confronted with an obvious moral choice, they still fall morally behind someone who actively exerts themselves to the limit of their ability for the benefit of needy people who they’ve never even met.
Maybe we just disagree about where the “average person” is morally? Or maybe you’re ascribing selfish intent to selfless action because the hypothetical person works in finance and/or makes lots of money?
Even if we accept that the average person usually acts morally when confronted with an obvious moral choice, they still fall morally behind someone who actively exerts themselves to the limit of their ability for the benefit of needy people who they’ve never even met.
Maybe we just disagree about where the “average person” is morally? Or maybe you’re ascribing selfish intent to selfless action because the hypothetical person works in finance and/or makes lots of money?
That is not what Scott Aaronson said. It's what you might believe; fair enough! Sacrificial giving is a big thing in the religion I belong to, too! Aaronson is making a point that minmaxing absolute donations (even, in this specific case, if it means defrauding thousands of customers) is a pathway towards moral superiority.
I think I'm confused about whether you're rejecting the proposition about the investor all on its own, or if it's the broader argument from the Aaronson post that you're rejecting. Your parenthetical makes me think we're not talking about the charitable-investor-propsition anymore; we're now talking about a specific investor who did something that appears to have been very risky and possibly very unethical.
My interpretation of Aaronson is that he thinks the hypothetical charitable investor is a good example of a very moral person, and we can use that example to imagine how we might have felt about SBF in an alternate universe where his bets paid off. But Aaronson also concludes:
> At any rate, I’d try to impress on him, as I do on anyone reading now, that the choice between linear and concave utilities, between risk-neutrality and risk-aversion, is not bloodless or technical—that it’s essential to make a choice that’s not only in reflective equilibrium with your highest values, but that you’ll still consider to be such regardless of which possible universe you end up in.
This isn't a post lionizing SBF. He is compared to but also contrasted with the "charitable traditional investor". The contrast is the degree of risk which, as you say, could be considered fraud. I don't come away from this thinking that Aaronson believes that SBF in particular is "way ahead of most of us morally".
My interpretation of Aaronson is that he thinks the hypothetical charitable investor is a good example of a very moral person, and we can use that example to imagine how we might have felt about SBF in an alternate universe where his bets paid off. But Aaronson also concludes:
> At any rate, I’d try to impress on him, as I do on anyone reading now, that the choice between linear and concave utilities, between risk-neutrality and risk-aversion, is not bloodless or technical—that it’s essential to make a choice that’s not only in reflective equilibrium with your highest values, but that you’ll still consider to be such regardless of which possible universe you end up in.
This isn't a post lionizing SBF. He is compared to but also contrasted with the "charitable traditional investor". The contrast is the degree of risk which, as you say, could be considered fraud. I don't come away from this thinking that Aaronson believes that SBF in particular is "way ahead of most of us morally".
> Are you describing "giving virtually all of [your earnings] away to effective charities" as "literally buying indulgences"? That's literally the most cynical thing I've ever heard....But your reaction to the charitable investor thing makes me think that you actually disagree with... altruism
I don't think it's the idea of giving to charity that's being criticised here. Its the idea a rich person giving an absolute sum of money poorer people cannot match is "ahead morally" on some utilitarian scale no matter what else the poorer person does, particularly in the context that the poster boy example of "earning to give" was Sam Bankman Fried...
I don't think it's the idea of giving to charity that's being criticised here. Its the idea a rich person giving an absolute sum of money poorer people cannot match is "ahead morally" on some utilitarian scale no matter what else the poorer person does, particularly in the context that the poster boy example of "earning to give" was Sam Bankman Fried...
Virtue ethics is dead as an ethos. The stated goal of practically all modern morality is good outcomes. This doesn't seem offhand implausible to me.
The position that the virtue of SBF's actions are irrelevant, and he should be judged better than all of us because he could afford to give more money to give away to good outcomes sounds exactly like "buying indulgences" to me.
Especially in Scott Aaronson's phrasing where it's the SBF's gross donation spend being ahead of all of ours that we should be impressed by, not the net impact of his actions (which I find difficult to believe is positive even on pure utilitarian grounds, given that he's not exactly inspiring others to "earn to give" right now, he's shrunk the world's balance sheet and I consider some of his [notionally] philanthropic spend pretty dubious)
Especially in Scott Aaronson's phrasing where it's the SBF's gross donation spend being ahead of all of ours that we should be impressed by, not the net impact of his actions (which I find difficult to believe is positive even on pure utilitarian grounds, given that he's not exactly inspiring others to "earn to give" right now, he's shrunk the world's balance sheet and I consider some of his [notionally] philanthropic spend pretty dubious)
Sure, but then the argument is "SBF was bad actually". I agree with this take - this is going to lead to bad outcomes. But note that saying so already embraces a consequentialist framework. This is a common argument for "rules utilitarianism", which is that given human psychology, getting humans to pursue (particular) virtues leads to better outcomes in the real world than getting humans to engage in first-order outcome maximization.
A true virtue-ethics take would be that SBF was inherently ennobled by good behavior or blemished by bad behavior, and the outcomes would have no relevance to this consideration. That's why I said virtue ethics is dead - approximately nobody argues like that today.
A true virtue-ethics take would be that SBF was inherently ennobled by good behavior or blemished by bad behavior, and the outcomes would have no relevance to this consideration. That's why I said virtue ethics is dead - approximately nobody argues like that today.
Well yes, nobody in this subthread is arguing that outcomes have no relevance.
We just object to Aaronson's rather extreme position that the correct measure of SBF's goodness is the single dimension of the absolute amounts he donated to notionally philanthropic causes (which seems like the reductio ad absurdum version of EA utilitarianism). And yes, I'm also unconvinced of the argument that behaviour and intent plays no role in ethics. It may be a good in terms of outcome that a sociopathic hereditary billionaire gives away 1%< of his fortune purely for reputation laundering purposes whilst minimising his interactions with other humans he declines to fulfil his rape and torture fantasies on purely out of rational self-interest, but I don't believe that would make him a "good person" by any reasonable ethical standard, never mind a "better person than all of us". At some point intent, relative sacrifice and what would other people have done in the same position become relevant to casting judgement on his benevolence, never mind his other vices. And certainly the courts will consider intent and not just the brute fact that SBF lost a lot of people's money when deciding whether to prosecute him, convictions and sentencing; I imagine most of his defence will be structured around that...
We just object to Aaronson's rather extreme position that the correct measure of SBF's goodness is the single dimension of the absolute amounts he donated to notionally philanthropic causes (which seems like the reductio ad absurdum version of EA utilitarianism). And yes, I'm also unconvinced of the argument that behaviour and intent plays no role in ethics. It may be a good in terms of outcome that a sociopathic hereditary billionaire gives away 1%< of his fortune purely for reputation laundering purposes whilst minimising his interactions with other humans he declines to fulfil his rape and torture fantasies on purely out of rational self-interest, but I don't believe that would make him a "good person" by any reasonable ethical standard, never mind a "better person than all of us". At some point intent, relative sacrifice and what would other people have done in the same position become relevant to casting judgement on his benevolence, never mind his other vices. And certainly the courts will consider intent and not just the brute fact that SBF lost a lot of people's money when deciding whether to prosecute him, convictions and sentencing; I imagine most of his defence will be structured around that...
I don't fully agree with Aaronson's take, I just think it's underappreciated. I am a sorta-rules utilitarian; in contested questions, I think the moral obligation is to behave in a fashion so that the standards by which we act, if universalized, maximize beneficial outcomes. And I think there is an argument to be made that getting rich people to donate is a thing that a moral system that tries to lead to good outcomes, should focus attention on.
Like it or not, we have set up our society so that economic decisionmaking power is to a large part concentrated in the hands of a small fraction of the population. This population is a moral fulcrum; the things they decide to do have a significant effect on actual events, in some ways even outstripping governments (which generally hold obligations on par with their power). I think an effective morality needs to acknowledge this and make special dispensation for these people if they decide to bend their power towards creating good outcomes, whether or not one approves of their existence in the abstract.
Assume you are trying to invent a morality to sell to the one percent. The pass-or-fail condition is that this morality must absolve their sins. It seems to me that the default option here is laissez-faire libertarianism, which has cultural bias against doing work to improve the worse off; in my view, being able to make inroads in this moral market is a significant victory for EA and good outcomes in general.
Like it or not, we have set up our society so that economic decisionmaking power is to a large part concentrated in the hands of a small fraction of the population. This population is a moral fulcrum; the things they decide to do have a significant effect on actual events, in some ways even outstripping governments (which generally hold obligations on par with their power). I think an effective morality needs to acknowledge this and make special dispensation for these people if they decide to bend their power towards creating good outcomes, whether or not one approves of their existence in the abstract.
Assume you are trying to invent a morality to sell to the one percent. The pass-or-fail condition is that this morality must absolve their sins. It seems to me that the default option here is laissez-faire libertarianism, which has cultural bias against doing work to improve the worse off; in my view, being able to make inroads in this moral market is a significant victory for EA and good outcomes in general.
>I am a sorta-rules utilitarian; in contested questions, I think the moral obligation is to behave in a fashion so that the standards by which we act, if universalized, maximize beneficial outcomes.
If you'll forgive the glibness, isn't that just deontology with extra steps? Sorry, it's just been on my mind since I learned (from Resident Contrarian article[0]) that many rule-utilitarianism is a functional reskin of disfavored deontological thinking.
[0]: https://www.residentcontrarian.com/p/the-weird-world-of-the-...
If you'll forgive the glibness, isn't that just deontology with extra steps? Sorry, it's just been on my mind since I learned (from Resident Contrarian article[0]) that many rule-utilitarianism is a functional reskin of disfavored deontological thinking.
[0]: https://www.residentcontrarian.com/p/the-weird-world-of-the-...
Every moral framework is every other moral framework with extra steps. They're all "morally Turing complete". However, which one you pick may make certain moral views significantly easier or harder to express. It's less that I think rule utilitarianism is morally compelling as a way of deciding what is good, as that I think rule utilitarianism captures some of my moral intuitions particularly well, so I use it to model (some of) the reasoning that my brain goes through when evaluating goodness. In other words, it's descriptive, not prescriptive.
All modern moral systems have extremely serious defects where they can be gamed. Repugnant conclusion, the entire concept of deontology, etc. Virtue Ethics is ungameable.
Virtue ethics is gameable if you can make virtues conflict.
This is a feature, not a bug. Casuistry is the science of determining the correct course of action in a complex situation. It’s complicated because the world is complicated. Attempting to create a moral system from first order logic (deontology) or a linear system (utilitarianism) was doomed from the start.
Which is trivial: pro-life vs. pro-choice, free-speech vs. anti-harassment, etc. I don't think there's anyone on any part of the political spectrum who thinks their ideas are not virtuous.
But even virtue ethics got parts of this right that EA seems to get wrong. From the very beginning, Aristotle caught on to idea that minmaxing a particular moral virtue gets you into deep trouble. That's precisely the hole EA falls into; see again Sam Bankman-Fried, and MacAskill's advice to him.
There are principled reasons to be against charity, namely that charity redistributes money to causes arbitrarily and undemocratically rather than to causes decided by a majority of people voting on what causes to support. When the government runs these programs, we get to vote on how they're administered. When billionaires run them, who gets help and who doesn't is entirely at the mercy of King Tech Bro the First. And to the extent that society heaps boundless praise on billionaires for their altruism, we are complicit in their reputation laundering; allowing them to continue to enjoy undue status in society in perpetuity that would not persist if we simply taxed that wealth away from them and did something like massively expanding public housing to end homelessness or something instead.
There's also principled reasons to be against this. Namely, that government tends to be woefully bad at optimizing for moral good (cf. the American prison industry or California's attempts to end homelessness) and is just as likely to put that money towards things the majority of the population disagree with. The same logic would say that Russia is right to strip its billionaires of wealth to pursue foreign wars.
Yes, government is bad at optimizing for moral good. Which is why billionaires should give their wealth to their own workers. If Tim Cook paid the people building iPhones more it would do far more good than him donating the money that comes from increased margins to some charity. The workers could decide what they need for themselves!
Instead, the idea seems to be that people are not properly qualified to identify their own needs and such decision need to be made by funneling profits to a CEO who then has superior knowledge of how to help the world and donates that money as he sees fit.
Instead, the idea seems to be that people are not properly qualified to identify their own needs and such decision need to be made by funneling profits to a CEO who then has superior knowledge of how to help the world and donates that money as he sees fit.
Turns out that billionaires tend to not want to do that, so it also turns out that using government power to compel them to do so using the power of taxation is the system that scales best.
Democracy is the worst way to decide things, except for all the others.
Depends what Aaronson meant by "being ahead morally"; it's unclear whether he is referring to morality in the sense of being virtuous, or in the sense of making other people better off. Hypothetical Charitable Investor's donations are a sign that that they might be rather virtuous, but they certainly aren't 1000 times more virtuous than the average person, and if it turns out that they secretly have a habit of kicking puppies, then they aren't virtuous at all. But it could certainly be true that, compared to an average person, Hypothetical Charitable Investor has had 1000 times the impact in terms of how they improved other people's lives.
> This is logic straight out of the 16th century Medici papacy of Leo X. Maybe it's some kind of horrible drug interaction with Rationalism, or maybe EA is fundamentally Rationalist? Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much,
I think you're missing the point of that paragraph, or at least misrepresenting it here.
It's not saying that what SBF did was ok or that fraud is ok or any such thing. It's specifically trying to say that just because this happened with crypto, which might be a bubble, doesn't make it any different than normal everyday finance. You might or might not agree with that idea, or might or might agree that finance itself is good, but that's the point of this paragraph specifically. Here it is in full, note the first sentence:
> Even if cryptocurrency remains just a modern-day tulip bulb or Beanie Baby, though, it seems morally hard to distinguish a cryptocurrency trader from the millions who deal in options, bonds, and all manner of other speculative assets. And a traditional investor who made billions on successful gambles, or arbitrage, or creating liquidity, then gave virtually all of it away to effective charities, would seem, on net, way ahead of most of us morally.
Also, just a note on this:
> Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much,
The problem with indulgences wasn't just the idea of being able to offset moral harm with moral good. Firstly, that position itself is at odds with what most EA people say afaict.
Secondly, the problem of indulgences is also that they are not real. I think even a religious person will agree that the indulgences of the 16th century was just a con. An atheist will dismiss the whole idea entirely.
The underlying logic of someone who made and gave away billions being overall ahead of other people morally is sound, IMHO. That of course is not to say that this then "allows" them to go and do some evil or something - that's not at all what the EA movement is saying.
I think you're missing the point of that paragraph, or at least misrepresenting it here.
It's not saying that what SBF did was ok or that fraud is ok or any such thing. It's specifically trying to say that just because this happened with crypto, which might be a bubble, doesn't make it any different than normal everyday finance. You might or might not agree with that idea, or might or might agree that finance itself is good, but that's the point of this paragraph specifically. Here it is in full, note the first sentence:
> Even if cryptocurrency remains just a modern-day tulip bulb or Beanie Baby, though, it seems morally hard to distinguish a cryptocurrency trader from the millions who deal in options, bonds, and all manner of other speculative assets. And a traditional investor who made billions on successful gambles, or arbitrage, or creating liquidity, then gave virtually all of it away to effective charities, would seem, on net, way ahead of most of us morally.
Also, just a note on this:
> Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much,
The problem with indulgences wasn't just the idea of being able to offset moral harm with moral good. Firstly, that position itself is at odds with what most EA people say afaict.
Secondly, the problem of indulgences is also that they are not real. I think even a religious person will agree that the indulgences of the 16th century was just a con. An atheist will dismiss the whole idea entirely.
The underlying logic of someone who made and gave away billions being overall ahead of other people morally is sound, IMHO. That of course is not to say that this then "allows" them to go and do some evil or something - that's not at all what the EA movement is saying.
The underlying logic of someone who made and gave away billions being overall ahead of other people morally is sound, IMHO
I'm not sure what to do with this. It is, obviously, not sound. There are a variety of ways in which such a person could be less moral than the pitifully ineffective non-billionaires. I'm not kidding when I say that we worked a lot of these problems out in the 16th century.
I'm not sure what to do with this. It is, obviously, not sound. There are a variety of ways in which such a person could be less moral than the pitifully ineffective non-billionaires. I'm not kidding when I say that we worked a lot of these problems out in the 16th century.
> There are a variety of ways in which such a person could be less moral than the pitifully ineffective non-billionaires.
I think we might be talking past each other a bit.
I'm saying, given no other information, someone who has given away billions, effectively, as in saved lives, is "morally ahead" of someone who hasn't similarly done so.
Of course they could've done other things that make them morally worse. Of course the way they donated money could be net-negative, if they caused negative things in the world.
But in the absence of other information, if I told you someone had donated a billion dollars to, I don't know, give food to hungry people, do you really think that is a morally neutral act? Like, if someone is a firefighter that has saved hundreds of lives by their heroism, do you think they are literally morally the same as other people who haven't? If that's not what you're talking about, I'm not sure what you are talking about.
Btw, I don't think morality is just an issue of summing up all your good versus bad deeds. I'm not exactly sure how to even define "ahead morally". A firefighter who has saved lives but also abused their kids - how do you even deal with a case like that? I think people are complex, they can do both good and bad, and the good that firefighter did is good, while the bad is bad, period.
I think one thing you might be objecting to is the idea that you can say something like "person X did 100 good deeds, and 99 bad deeds, therefore they are net good" or even "it's worth giving money to a cause that causes 99 bad deeds and 100 good deeds cause it's net positive." And I agree, that's a valid criticism and a wrong way to think about things.
I think we might be talking past each other a bit.
I'm saying, given no other information, someone who has given away billions, effectively, as in saved lives, is "morally ahead" of someone who hasn't similarly done so.
Of course they could've done other things that make them morally worse. Of course the way they donated money could be net-negative, if they caused negative things in the world.
But in the absence of other information, if I told you someone had donated a billion dollars to, I don't know, give food to hungry people, do you really think that is a morally neutral act? Like, if someone is a firefighter that has saved hundreds of lives by their heroism, do you think they are literally morally the same as other people who haven't? If that's not what you're talking about, I'm not sure what you are talking about.
Btw, I don't think morality is just an issue of summing up all your good versus bad deeds. I'm not exactly sure how to even define "ahead morally". A firefighter who has saved lives but also abused their kids - how do you even deal with a case like that? I think people are complex, they can do both good and bad, and the good that firefighter did is good, while the bad is bad, period.
I think one thing you might be objecting to is the idea that you can say something like "person X did 100 good deeds, and 99 bad deeds, therefore they are net good" or even "it's worth giving money to a cause that causes 99 bad deeds and 100 good deeds cause it's net positive." And I agree, that's a valid criticism and a wrong way to think about things.
I'm saying, given no other information, someone who has given away billions, effectively, as in saved lives, is "morally ahead" of someone who hasn't similarly done so.
We're not talking past each other. This is false.
Your problem is that you're using the concept of incomplete information, rather than that of ceteris paribus. Compare two billionaires, identical in every respect other than that one donated more than the other. The more charitable billionaire is morally superior (or, many reasonable people, myself included, would say so).
But that's as far as it goes. And in reality there are never identical billionaires to compare. One billionaire doesn't give enough; the other pledges all their money to GiveWell or EA or AGI, but also keeps hobos locked up in their basement. And, in the main, who gives a shit about the relative value of billionaires? The logic is shot completely to shit as soon as we're trying to compare the marginal sacrifice of a billionaire's giving pledge to the moral decisions people make every day of their life. And yet that's exactly what Scott Aaronson does.
I feel embarrassed just typing this stuff out. This is, like, philosophy 101 stuff? The logic I'm arguing against here is on its face one of the cases moral philosophers make against Bentham's naive utilitarianism --- that it leads people to believe stuff like this.
We're not talking past each other. This is false.
Your problem is that you're using the concept of incomplete information, rather than that of ceteris paribus. Compare two billionaires, identical in every respect other than that one donated more than the other. The more charitable billionaire is morally superior (or, many reasonable people, myself included, would say so).
But that's as far as it goes. And in reality there are never identical billionaires to compare. One billionaire doesn't give enough; the other pledges all their money to GiveWell or EA or AGI, but also keeps hobos locked up in their basement. And, in the main, who gives a shit about the relative value of billionaires? The logic is shot completely to shit as soon as we're trying to compare the marginal sacrifice of a billionaire's giving pledge to the moral decisions people make every day of their life. And yet that's exactly what Scott Aaronson does.
I feel embarrassed just typing this stuff out. This is, like, philosophy 101 stuff? The logic I'm arguing against here is on its face one of the cases moral philosophers make against Bentham's naive utilitarianism --- that it leads people to believe stuff like this.
> Your problem is that you're using the concept of incomplete information, rather than that of ceteris paribus.
You're right, that's what I meant and I misspoke.
> The logic is shot completely to shit as soon as we're trying to compare the marginal sacrifice of a billionaire's giving pledge to the moral decisions people make every day of their life. And yet that's exactly what Scott Aaronson does.
Well I think that he was more comparing crypto vs non crypto and saying there's not much difference, but fine.
> I feel embarrassed just typing this stuff out. This is, like, philosophy 101 stuff?
Sorry to be embarrassing, I guess? :Shrug:
> The logic I'm arguing against here is on its face one of the cases moral philosophers make against Bentham's naive utilitarianism --- that it leads people to believe stuff like this.
I'm curious; I think utilitarianism is true broadly speaking (though not naive utilitarianism). Do you?
You're right, that's what I meant and I misspoke.
> The logic is shot completely to shit as soon as we're trying to compare the marginal sacrifice of a billionaire's giving pledge to the moral decisions people make every day of their life. And yet that's exactly what Scott Aaronson does.
Well I think that he was more comparing crypto vs non crypto and saying there's not much difference, but fine.
> I feel embarrassed just typing this stuff out. This is, like, philosophy 101 stuff?
Sorry to be embarrassing, I guess? :Shrug:
> The logic I'm arguing against here is on its face one of the cases moral philosophers make against Bentham's naive utilitarianism --- that it leads people to believe stuff like this.
I'm curious; I think utilitarianism is true broadly speaking (though not naive utilitarianism). Do you?
Consequentialism says that the ends justify the means. They do not.
> Consequentialism says that the ends justify the means. They do not.
Well, I disagree with that in the absolute sense, which might be where we differ? If we could accurately foresee the ends of some action lead to more people being better off, it would be moral to take that action. And I think the fact that "the ends don't justify the means" is such a slogan in our world, said as if it's obviously true, is ridiculous.
(And despite you continuously saying this is philosophy 101, I don't think the generally accepted view among philosophers is that utilitarianism is wrong, is it?)
FWIW I think you're right in a practical sense though. If you're having to justify immoral actions by their "ends", you're making a huge mistake, and almost certainly only doing it because you wanted to do that immoral thing anyway. Humans are notoriously bad at prediction and notoriously good at rationalization.
Well, I disagree with that in the absolute sense, which might be where we differ? If we could accurately foresee the ends of some action lead to more people being better off, it would be moral to take that action. And I think the fact that "the ends don't justify the means" is such a slogan in our world, said as if it's obviously true, is ridiculous.
(And despite you continuously saying this is philosophy 101, I don't think the generally accepted view among philosophers is that utilitarianism is wrong, is it?)
FWIW I think you're right in a practical sense though. If you're having to justify immoral actions by their "ends", you're making a huge mistake, and almost certainly only doing it because you wanted to do that immoral thing anyway. Humans are notoriously bad at prediction and notoriously good at rationalization.
People keep mentioning philosophers here, as if we were having a debate about weak vs. strong rule utilitarianism. My point is that the philosophy being expounded here is pre-Benthamite; it is the philosophy of the Medicis, not of Mill.
There is right now on Twitter a circulating post regarding the CEO of the Future Fund, FTX's EA funding arm, a person who did a dissertation on utilitarian ethics, which stated that it might make more sense to save wealthy lives in the developed world before saving lives in the developing world, because the wealthy lives would have greater impact. This is what happens when you take a simple idea about minmaxing the benefit of charitable contributions and hook it up to the morals of 16th century banking families. The snake is eating its own tail.
There is right now on Twitter a circulating post regarding the CEO of the Future Fund, FTX's EA funding arm, a person who did a dissertation on utilitarian ethics, which stated that it might make more sense to save wealthy lives in the developed world before saving lives in the developing world, because the wealthy lives would have greater impact. This is what happens when you take a simple idea about minmaxing the benefit of charitable contributions and hook it up to the morals of 16th century banking families. The snake is eating its own tail.
Last I checked, no one has proven this.
You gotta check in places other than LessWrong.
I'm actually well versed in virtue ethics, deontology and consequentialism and I don't read LessWrong. But if you have a reference where someone proved consequentualism was incoherent or invalid, please provide it, that would be quite earth shattering.
> The underlying logic of someone who made and gave away billions being overall ahead of other people morally is sound.
Do you hear yourself, not even asking how they "made" their billions? This is what "EA is indulgence for billionaires" people are talking about exactly.
If you make your billions unethically, no amount of charity will make you a good person.
Do you hear yourself, not even asking how they "made" their billions? This is what "EA is indulgence for billionaires" people are talking about exactly.
If you make your billions unethically, no amount of charity will make you a good person.
> If you make your billions unethically, no amount of charity will make you a good person.
I completely agree! I thought the context of the rest of my post made it clear that I assume they made it in an ethical way, but I guess it wasn't clear? I mean, if someone says they gave away billions to charity, my first question isn't immediately "but how did you make billions?". I just assume it's legit, because in most cases it's legit. And I thought I was pretty clear above that in cases it isn't legit, then that's not morally ok.
I completely agree! I thought the context of the rest of my post made it clear that I assume they made it in an ethical way, but I guess it wasn't clear? I mean, if someone says they gave away billions to charity, my first question isn't immediately "but how did you make billions?". I just assume it's legit, because in most cases it's legit. And I thought I was pretty clear above that in cases it isn't legit, then that's not morally ok.
It is impossible to make billions of dollars in an ethical way (short of inheriting it from someone who made it in an unethical way).
First of all, just looking at empirical evidence, all the existing billionaires—every single one—made their billions through some form of exploitation.
Second of all, you can't make billions without going through millions for a good while—and once you have enough money to live comfortably for the rest of your life (which I'm not going to try to put an exact number on, but I think it's fair to say it's somewhere in the millions), it is unethical not to find ways to use the rest of it to improve the lives of the rest of the people on Earth.
First of all, just looking at empirical evidence, all the existing billionaires—every single one—made their billions through some form of exploitation.
Second of all, you can't make billions without going through millions for a good while—and once you have enough money to live comfortably for the rest of your life (which I'm not going to try to put an exact number on, but I think it's fair to say it's somewhere in the millions), it is unethical not to find ways to use the rest of it to improve the lives of the rest of the people on Earth.
I also right after said that this doesn't allow them to then go do something bad. I mean this in both directions, obviously!
My problem with assumption that you make is that you assume the billionaires came to their billions through some divine magic, and thus "a very few bad apples" can't spoil the bunch.
I start my line of thinking from the other end -- what kind of regulation and laws exist that can create exponential wealth for someone whose work output only ever scales linearly. The gap between single person's output produced and wealth generated is proof that our simple abstraction that "money is just a symbol proportional to one's contribution to society" breaks down at very large (and sometimes really tiny) scales. Effective altruism will never fix that because it presupposes the system where billionaires are created and exists comfortably. The alternative, of course, is taking a closer look at the system itself and making sure our abstraction of "money = effort given to society" functions properly at all scales.
I start my line of thinking from the other end -- what kind of regulation and laws exist that can create exponential wealth for someone whose work output only ever scales linearly. The gap between single person's output produced and wealth generated is proof that our simple abstraction that "money is just a symbol proportional to one's contribution to society" breaks down at very large (and sometimes really tiny) scales. Effective altruism will never fix that because it presupposes the system where billionaires are created and exists comfortably. The alternative, of course, is taking a closer look at the system itself and making sure our abstraction of "money = effort given to society" functions properly at all scales.
> Effective altruism will never fix that because it presupposes the system where billionaires are created and exists comfortably. The alternative, of course, is taking a closer look at the system itself and making sure our abstraction of "money = effort given to society" functions properly at all scales.
I disagree with a few things you say, but this is the biggest: what does this have to do with EA?
Yeah, sure, EA will never fix the systemic problems of the system it's in. Neither will scientists working on new vaccines, but you're not criticizing them for it, I imagine. You can do plenty of good in the world within the "current system", without also trying to change that system, and the fact that you're not trying shouldn't be held against you.
(Although, for the record, some people in EA do think political activism is important and spend a lot of effort there.)
> I start my line of thinking from the other end -- what kind of regulation and laws exist that can create exponential wealth for someone whose work output only ever scales linearly. The gap between single person's output produced and wealth generated is proof that our simple abstraction that "money is just a symbol proportional to one's contribution to society" breaks down at very large (and sometimes really tiny) scales.
I don't think that's how anyone should view money. And I don't think someone's output only scales linearly.
I mean, that's clearly not the case. Stephen King has written lots of books that many people enjoyed. The value he has created in the world is far larger than the value I've created, or will likely create, and it's not a linear difference. There are lots of examples like that.
In general, regulations and laws don't "allow" for creation of exponential wealth, they just don't prevent something that happens completely naturally. We can certainly decide to "tax more" or something as a society, but that comes with its own problems.
I disagree with a few things you say, but this is the biggest: what does this have to do with EA?
Yeah, sure, EA will never fix the systemic problems of the system it's in. Neither will scientists working on new vaccines, but you're not criticizing them for it, I imagine. You can do plenty of good in the world within the "current system", without also trying to change that system, and the fact that you're not trying shouldn't be held against you.
(Although, for the record, some people in EA do think political activism is important and spend a lot of effort there.)
> I start my line of thinking from the other end -- what kind of regulation and laws exist that can create exponential wealth for someone whose work output only ever scales linearly. The gap between single person's output produced and wealth generated is proof that our simple abstraction that "money is just a symbol proportional to one's contribution to society" breaks down at very large (and sometimes really tiny) scales.
I don't think that's how anyone should view money. And I don't think someone's output only scales linearly.
I mean, that's clearly not the case. Stephen King has written lots of books that many people enjoyed. The value he has created in the world is far larger than the value I've created, or will likely create, and it's not a linear difference. There are lots of examples like that.
In general, regulations and laws don't "allow" for creation of exponential wealth, they just don't prevent something that happens completely naturally. We can certainly decide to "tax more" or something as a society, but that comes with its own problems.
> Maybe it's some kind of horrible drug interaction with Rationalism, or maybe EA is fundamentally Rationalist?
Depends what you mean by Rationalist. I would say that EA, the modern movement, is a hobby of people who belong to "rationalism", the modern movement. I'd be fairly comfortable calling them a cult. (Frequent exhortation: "Read the Sequences!") EA is the same thing, simply because the people who do EA and the people who do rationalism are the same people.
But from your comment, I can't tell whether Rationalist refers to that group or to some other thing.
Depends what you mean by Rationalist. I would say that EA, the modern movement, is a hobby of people who belong to "rationalism", the modern movement. I'd be fairly comfortable calling them a cult. (Frequent exhortation: "Read the Sequences!") EA is the same thing, simply because the people who do EA and the people who do rationalism are the same people.
But from your comment, I can't tell whether Rationalist refers to that group or to some other thing.
> I had previously understood Effective Altruism to be something akin to like a pledge drive for Charity Navigator's best-reviewed charities or something
I mean, I think you were basically right before, and now you're over-reading a lot from an awkward sentence fragment. I agree that the phrase and implied comparison is garbage.
It seems kind of like hearing MTG talk about the Gazpacho Police or whatever and subsequently ascribing it to the GOP. Yes, the quote is clearly inane. Yes, she is a person of some influence in the GOP; no, that doesn't mean everyone in the GOP believes what MTG is saying. I guess you could say the GOP is a cult, but I think maybe cult doesn't mean anything if it applies to 40% of the US population.
Anyway, I totally agree as far as "AI safety." And fuck SBF and steal-to-give.
> Yes, to answer the Economist; this movement is irretrievable.
What name would you give to a movement to point charitable dollars at, say, the front page of GiveWell? (Givewell is more or less what EA means to me.)
I mean, I think you were basically right before, and now you're over-reading a lot from an awkward sentence fragment. I agree that the phrase and implied comparison is garbage.
It seems kind of like hearing MTG talk about the Gazpacho Police or whatever and subsequently ascribing it to the GOP. Yes, the quote is clearly inane. Yes, she is a person of some influence in the GOP; no, that doesn't mean everyone in the GOP believes what MTG is saying. I guess you could say the GOP is a cult, but I think maybe cult doesn't mean anything if it applies to 40% of the US population.
Anyway, I totally agree as far as "AI safety." And fuck SBF and steal-to-give.
> Yes, to answer the Economist; this movement is irretrievable.
What name would you give to a movement to point charitable dollars at, say, the front page of GiveWell? (Givewell is more or less what EA means to me.)
"Effective Altruism Is Not A Cult" sounds like what a cult would say :)
Sure sounds like a cult in practice, especially in the whole context ( https://aiascendant.substack.com/p/extropias-children-chapte... the other 6 parts of the series are great reading too).
Sure sounds like a cult in practice, especially in the whole context ( https://aiascendant.substack.com/p/extropias-children-chapte... the other 6 parts of the series are great reading too).
It's a reference to this tweet: https://twitter.com/shutupmikeginn/status/403359911481839617
> "Effective Altruism Is Not A Cult" sounds like what a cult would say
Yep, that's the joke.
Yep, that's the joke.
> Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much, and if you can't recognize that, something has gone terribly wrong in your thinking
Buying indulgences was wrong because it was used to cover up moral wrongdoing or reduce ones penance for wrongdoing, not because the concept of money buying moral praise is intrinsically false or corrupt. It seems pretty clear from Scott's example that this is not the scenario he was describing.
Phrased differently, all else being equal, a person that contributed billions to charities deserves more moral praise than one who didn't or can't. Edit: and if you don't think so, then why do we give such people awards and recognition?
This won't be convincing to everyone of course, and I expect the dividing line will be whether you think utilitarianism is the correct way to approach ethics. Of course if you don't, then you won't subscribe to EA anyway.
Buying indulgences was wrong because it was used to cover up moral wrongdoing or reduce ones penance for wrongdoing, not because the concept of money buying moral praise is intrinsically false or corrupt. It seems pretty clear from Scott's example that this is not the scenario he was describing.
Phrased differently, all else being equal, a person that contributed billions to charities deserves more moral praise than one who didn't or can't. Edit: and if you don't think so, then why do we give such people awards and recognition?
This won't be convincing to everyone of course, and I expect the dividing line will be whether you think utilitarianism is the correct way to approach ethics. Of course if you don't, then you won't subscribe to EA anyway.
Someone who has made millions or billions is, on average, doing this by making life worse for many people than if they were not in the loop taking profits. There are exceptions (Notch, etc.) where they took profits that were freely given rather than those that were skimmed off the top.
So you are literally saying that by taking a portion of that money and donating it they are morally superior to the poor they took it from (compared to those individuals we are almost ALL poor).
This is circular logic to support profit taking beyond what is reasonable.
So you are literally saying that by taking a portion of that money and donating it they are morally superior to the poor they took it from (compared to those individuals we are almost ALL poor).
This is circular logic to support profit taking beyond what is reasonable.
> Someone who has made millions or billions is, on average, doing this by making life worse for many people than if they were not in the loop taking profits.
See the part where I said "all else being equal".
See the part where I said "all else being equal".
>maybe EA is fundamentally Rationalist?
The EA forum in fact uses the forum software developed by LessWrong, in very close collaboration with them (it's not open source or anything). In general it looks like the two organizations have a lot in common, and LessWrong is a poster child for the rationality movement.
The EA forum in fact uses the forum software developed by LessWrong, in very close collaboration with them (it's not open source or anything). In general it looks like the two organizations have a lot in common, and LessWrong is a poster child for the rationality movement.
> it's not open source or anything
GPL3 actually https://github.com/ForumMagnum/ForumMagnum
> In general it looks like the two organizations have a lot in common
Yes, see https://en.wikipedia.org/wiki/Effective_altruism#History
GPL3 actually https://github.com/ForumMagnum/ForumMagnum
> In general it looks like the two organizations have a lot in common
Yes, see https://en.wikipedia.org/wiki/Effective_altruism#History
If we're gonna make vaguely sweeping comparisons to religion, I would like to point out that much of this kind of instinctive aversion to EA probably comes, itself, from the secular protestantism pervading US culture.
- Outcome matters more than the intention of the person trying to do a good act? Absurd; here, I can even prove it with an absurd example!
- Trying, at all, to build a framework for good and bad different from that handed down to us? How dare they.
- Artificial general intelligence? Pointless to worry about, only humans can be really intelligent because [strikethrough]they have a soul[/strikethrough] there's something about consciousness that we don't yet understand.
- Outcome matters more than the intention of the person trying to do a good act? Absurd; here, I can even prove it with an absurd example!
- Trying, at all, to build a framework for good and bad different from that handed down to us? How dare they.
- Artificial general intelligence? Pointless to worry about, only humans can be really intelligent because [strikethrough]they have a soul[/strikethrough] there's something about consciousness that we don't yet understand.
> ... they have their own weird language, there's a hierarchy, and people on the inside operate under a constant threat of ostracism.
Aside: A good rule of thumb for when you should start getting out of something is when the group comes up with a cute little name for those outside the group, typically ending in 'ie'. Normies, heggies, fundies, libtards, etc. It starts out innocent enough, but when those artificial divisions spring up, that's when the brains start shutting off and people start preaching to the choir.
Aside: A good rule of thumb for when you should start getting out of something is when the group comes up with a cute little name for those outside the group, typically ending in 'ie'. Normies, heggies, fundies, libtards, etc. It starts out innocent enough, but when those artificial divisions spring up, that's when the brains start shutting off and people start preaching to the choir.
> Either way: when your logic has established that you can obtain moral superiority by literally buying indulgences, it has officially proven too much, and if you can't recognize that, something has gone terribly wrong in your thinking.
I see it as a natural outcome of consequentialism. Your ability to do good things is tied to your circumstances, which are not the same for everyone. Are you saying that you're certain that consequentialism is an invalid way to asess morality?
I see it as a natural outcome of consequentialism. Your ability to do good things is tied to your circumstances, which are not the same for everyone. Are you saying that you're certain that consequentialism is an invalid way to asess morality?
The issue with the morality of indulgence isn’t that it recognizes that different circumstances allow for different outcomes, it’s that it allows for immoral behavior so long as the calculus works out.
Evade your taxes, no problem if you gave more away to charities than you owed!
Do a little light billion dollars fraud? So long as you saved the trillions of future humanity from an evil AI it’s fine.
Sacrifice your children to the fires of Kronos? Well if you hadn’t the Romans would have killed us all.
You can argue that literally anything is moral that way.
Evade your taxes, no problem if you gave more away to charities than you owed!
Do a little light billion dollars fraud? So long as you saved the trillions of future humanity from an evil AI it’s fine.
Sacrifice your children to the fires of Kronos? Well if you hadn’t the Romans would have killed us all.
You can argue that literally anything is moral that way.
You say "allows for immoral behaviour", but no system of ethics allows for immoral behaviour by definition.
What you're presenting are, for a consequentialist, the choices between two evils:
- do I pay taxes or give away to charities?
- do I do fraud or let evil AI go on a rampage?
- do I sacrifice my children or let the Romans kill us?
You can't argue for things that have no positive consequence: the child sacrifice sounds dumb to us because we see the alternatives as independent. If you only consider either/or choices, the number of things you can justify as a consequentialist becomes limited.
What you're presenting are, for a consequentialist, the choices between two evils:
- do I pay taxes or give away to charities?
- do I do fraud or let evil AI go on a rampage?
- do I sacrifice my children or let the Romans kill us?
You can't argue for things that have no positive consequence: the child sacrifice sounds dumb to us because we see the alternatives as independent. If you only consider either/or choices, the number of things you can justify as a consequentialist becomes limited.
[deleted]
> Over and over I keep coming across EA arguments that could be used to justify almost literally any behavior
Let's stop being surprised by the self-serving rationalizations of (many) tech leaders. Expect it, stop trusting it by default, stop taking them seriously.
Or try this hypothesis: Whatever the issue, their theory (EA or whatever else) will always benefit them. Find a more predictive hypothesis if you can.
Let's stop being surprised by the self-serving rationalizations of (many) tech leaders. Expect it, stop trusting it by default, stop taking them seriously.
Or try this hypothesis: Whatever the issue, their theory (EA or whatever else) will always benefit them. Find a more predictive hypothesis if you can.
If some of you have to die in order to generate a significant charitable outcome, then that’s a price I’m willing to pay. /s
I'd buy that shirt if the backside read "But Philosophy of Statistics is", or something like that
re your last sentence: why do you disapprove of funding research into AI safety
Not OP, but I do work in reducing harms of deployed AI systems at a big tech company
When most EA/Rationalist folks discuss AI Safety. They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible
Now there’s nothing inherently wrong with funding research of that sort.
The problem comes is that many EA/Rationalist folks begin downplaying legitimate tangible risks on the horizon (I’ve had prominent EA folks tell me that Climate Change is not an existential risk) or that many of these AI Safety researchers actively ignore how to reduce harms of the AI we have deployed today.
This existential measuring contest is just so odd and unnecessary. I really don’t care if you think AI Safety is a legitimate problem. I personally don’t but it’s when they begin to downplay others actual catastrophes that many of us take issue.
When most EA/Rationalist folks discuss AI Safety. They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible
Now there’s nothing inherently wrong with funding research of that sort.
The problem comes is that many EA/Rationalist folks begin downplaying legitimate tangible risks on the horizon (I’ve had prominent EA folks tell me that Climate Change is not an existential risk) or that many of these AI Safety researchers actively ignore how to reduce harms of the AI we have deployed today.
This existential measuring contest is just so odd and unnecessary. I really don’t care if you think AI Safety is a legitimate problem. I personally don’t but it’s when they begin to downplay others actual catastrophes that many of us take issue.
The atom bomb was once a hypothetical construct. The Einstein-Szilard letter was talking about a danger that they had no way of proving was even possible. Proof is irrelevant outside of mathematics; the only relevant question is cost to expected benefit - or expected harm.
With dangerous research, when a conclusive demonstration of possibility is achieved, it is usually too late to do much in the way of prevention. That's why we don't expect the FDA to "prove" that harm from a new medication "is even possible" before regulating it; rather, we expect the pharmaceutical company to provide evidence for its expected harmlessness before we even make the attempt. Think of the FDA what you like, this is certainly a better standard than putting the burden of proof on harmful outcomes.
With dangerous research, when a conclusive demonstration of possibility is achieved, it is usually too late to do much in the way of prevention. That's why we don't expect the FDA to "prove" that harm from a new medication "is even possible" before regulating it; rather, we expect the pharmaceutical company to provide evidence for its expected harmlessness before we even make the attempt. Think of the FDA what you like, this is certainly a better standard than putting the burden of proof on harmful outcomes.
Part of the problem with this is that conclusive demonstration that AGI is possible is not even in the same ballpark as a conclusive demonstration that it would have consequences like the EA folks seem to believe.
From everything I've read about their views, they seem to believe that either the moment an AGI is created or very, very soon thereafter, it will achieve the Singularity and "ascend" to something near godhood from our perspective, and it will perceive humanity as a threat to it and immediately seek our destruction—which, because of its near-godhood, it will be able to achieve far faster than we have the ability to respond to. Thus, what we have to do is either prevent AGI from ever coming to pass (because as soon as it exists, it's effectively too late), or make absolutely certain that by the time it does, we have strong measures in place to either fight back against it, or get humanity the hell out of Dodge.
This is built on such a foundation of shaky assumptions that giving it credibility is ludicrous.
From everything I've read about their views, they seem to believe that either the moment an AGI is created or very, very soon thereafter, it will achieve the Singularity and "ascend" to something near godhood from our perspective, and it will perceive humanity as a threat to it and immediately seek our destruction—which, because of its near-godhood, it will be able to achieve far faster than we have the ability to respond to. Thus, what we have to do is either prevent AGI from ever coming to pass (because as soon as it exists, it's effectively too late), or make absolutely certain that by the time it does, we have strong measures in place to either fight back against it, or get humanity the hell out of Dodge.
This is built on such a foundation of shaky assumptions that giving it credibility is ludicrous.
I agree with your description of the AGI/ASI position.
My take on it is the other way around: every step of the argument seems plainly obvious.
My take on it is the other way around: every step of the argument seems plainly obvious.
Plainly, that is not the case, as you are choosing to not spend your every waking moment to stop the antichrist AI coming into existence.
So either it's less obvious than you're claiming, or you are taking the threat of creating an evil AGI not seriously enough according to your own beliefs.
So either it's less obvious than you're claiming, or you are taking the threat of creating an evil AGI not seriously enough according to your own beliefs.
I don't spend every waking moment stopping lots of things that could kill us all. Nuclear war, climate change, gain-of-function research.
I just don't care that much about things that kill everyone. I don't think almost anyone does. In the Cold War, the vast majority of the population was not doing absolutely everything to achieve bilateral disarmament. This would lead you to believe that they didn't really expect nuclear war. But maybe they were just apathetic and helpless.
I just don't care that much about things that kill everyone. I don't think almost anyone does. In the Cold War, the vast majority of the population was not doing absolutely everything to achieve bilateral disarmament. This would lead you to believe that they didn't really expect nuclear war. But maybe they were just apathetic and helpless.
The atom bomb was once a hypothetical construct. The Einstein-Szilard letter was talking about a danger that they had no way of proving was even possible.
This isn't an accurate representation, really. The letter was written exactly because it was clear that a fission bomb is very likely within the short-term reach of current technological and industrial capability. This is very much not the case with AGI.
This isn't an accurate representation, really. The letter was written exactly because it was clear that a fission bomb is very likely within the short-term reach of current technological and industrial capability. This is very much not the case with AGI.
That's a difference of degrees, not kind. You can always ask for a higher class of proof.
How is it a difference of degrees? This is just plain inaccurate:
The Einstein-Szilard letter was talking about a danger that they had no way of proving was even possible.
Supporting your argument with an inaccurate thing is not 'a difference of degrees'. It's a difference between a good argument and a bad one.
The Einstein-Szilard letter was talking about a danger that they had no way of proving was even possible.
Supporting your argument with an inaccurate thing is not 'a difference of degrees'. It's a difference between a good argument and a bad one.
The disagreement is about the difficulty of "proving". I could say that unfriendly AI is clearly possible because evil humans exist. You could say that unfriendly AI is not proven possible until one has actually been built. But if you applied that standard to the letter, it would likewise fail it. Now, I agree that the case for AI is weaker, maybe much weaker, than the case for nuclear fission when the letter was written - there is, for instance, no clear research agenda that is held by preeminent researchers in the field to lead to it - though there are research groups, such as Deepmind, that have AGI as their explicit goal - but that also has nothing to do with being "proven" possible.
My argument is that for contested and dangerous outcomes, proof of possibility is an unreasonable standard, in good part because it massively underspecifies the goalposts.
My argument is that for contested and dangerous outcomes, proof of possibility is an unreasonable standard, in good part because it massively underspecifies the goalposts.
The disagreement is about the difficulty of "proving"
It's not, it's just a turn of phrase you've latched on to. There's no discussion of a standard of proof, etc. The comparison between the Szilard Einstein letter and 'AGI safety' is specious and it's one you brought up! You have to outright misrepresent what the letter was about just to make it. It was not a letter about a 'hypothetical danger they had no way of proving'. There just isn't any reasonable reading of the letter, the context, the history in which that's an accurate statement.
It's not, it's just a turn of phrase you've latched on to. There's no discussion of a standard of proof, etc. The comparison between the Szilard Einstein letter and 'AGI safety' is specious and it's one you brought up! You have to outright misrepresent what the letter was about just to make it. It was not a letter about a 'hypothetical danger they had no way of proving'. There just isn't any reasonable reading of the letter, the context, the history in which that's an accurate statement.
My whole point and the emphasis on why I brought it up was that yes, the letter was about a "hypothetical" danger they had no way of "proving".
By any reasonable standard, at that point the possibility of nuclear fission was well established. But the argument is that there is no central committee that defines the reasonable standard and checks theories against it. It's all just opinion, and that specific opinion can be arbitrarily goalshifted.
I'm not saying the letter was about a hypothetical danger they had no way of proving, I don't think that. But that's specific to my interpretation of those terms, and I could easily see the exact same argument levered against the letter if that debate had been done in public.
By any reasonable standard, at that point the possibility of nuclear fission was well established. But the argument is that there is no central committee that defines the reasonable standard and checks theories against it. It's all just opinion, and that specific opinion can be arbitrarily goalshifted.
I'm not saying the letter was about a hypothetical danger they had no way of proving, I don't think that. But that's specific to my interpretation of those terms, and I could easily see the exact same argument levered against the letter if that debate had been done in public.
> I’ve had prominent EA folks tell me that Climate Change is not an existential risk
Have you seen anything to indicate otherwise? The IPCC reports don't contain even a whiff in that direction.
To the best of my knowledge zero people who know anything about climate change think its existential.
Have you seen anything to indicate otherwise? The IPCC reports don't contain even a whiff in that direction.
To the best of my knowledge zero people who know anything about climate change think its existential.
Is there an accepted unambiguous definition of "existential" here?
Is it:
1. All life on the planet dies
2. All advanced life on the planet dies?
3. All humans die?
4. Advanced civilization is destroyed irretrievably?
5. Advanced civilization is destroyed for a long period of time?
etc
And I'm not sure the group "people who know anything about climate change" are neccesarily better equipped to answer this question - it's a complex systems question.
I'm not actually sure who is equipped to answer it accurately.
Is it:
1. All life on the planet dies
2. All advanced life on the planet dies?
3. All humans die?
4. Advanced civilization is destroyed irretrievably?
5. Advanced civilization is destroyed for a long period of time?
etc
And I'm not sure the group "people who know anything about climate change" are neccesarily better equipped to answer this question - it's a complex systems question.
I'm not actually sure who is equipped to answer it accurately.
It looks like there are research works trying to define precisely this notion of existential risk in the context of climate change, see e.g. https://link.springer.com/article/10.1007/s10584-022-03430-y
I'm not sure if this would actually count as "existential", but I think that it's well worth being very, very concerned about a consequence on the order of
"Millions to billions of people die; large percentages of the rest are displaced and/or have their lives made significantly shorter, harsher, and less certain as war, famine, disease, and natural disasters become vastly more common around the globe."
And there's really very little question that that's where climate change is leading if we don't get it under control soon.
"Millions to billions of people die; large percentages of the rest are displaced and/or have their lives made significantly shorter, harsher, and less certain as war, famine, disease, and natural disasters become vastly more common around the globe."
And there's really very little question that that's where climate change is leading if we don't get it under control soon.
It's always 2 or 3 in my experience.
> The problem comes is that many EA/Rationalist folks begin downplaying legitimate tangible risks on the horizon (I’ve had prominent EA folks tell me that Climate Change is not an existential risk) or that many of these AI Safety researchers actively ignore how to reduce harms of the AI we have deployed today.
I'm sure some people do this, and it's bad.
But to be clear, when you say some researchers are actively ignoring how to reduce harms of AI today... I'm not sure what you mean by "actively" ignore, but don't most researchers just by default ignore this, except for the few who are active in this field?
In other words, I think it's totally legit to focus on long-term AGI risk, and totally ignore current short-term AI risk, as a personal career-choice, just like it's ok to not even work on AI at all. Unless someone is actively trying to cause less resources to go into short-term AI risk, what's the problem?
> They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible
I mean, we exist as (one would hope) intelligent beings. Why wouldn't AGI be possible? Your frame makes it seem like the default is that it isn't possible, which is wrong IMO.
I'm sure some people do this, and it's bad.
But to be clear, when you say some researchers are actively ignoring how to reduce harms of AI today... I'm not sure what you mean by "actively" ignore, but don't most researchers just by default ignore this, except for the few who are active in this field?
In other words, I think it's totally legit to focus on long-term AGI risk, and totally ignore current short-term AI risk, as a personal career-choice, just like it's ok to not even work on AI at all. Unless someone is actively trying to cause less resources to go into short-term AI risk, what's the problem?
> They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible
I mean, we exist as (one would hope) intelligent beings. Why wouldn't AGI be possible? Your frame makes it seem like the default is that it isn't possible, which is wrong IMO.
They are right. Climate change is absolutely not an existential risk.
“In a high-emissions scenario where little is done to curb planet-heating gases, global mortality rates will be raised by 73 deaths per 100,000 people by the end of the century”
https://www.theguardian.com/us-news/2020/aug/04/rising-globa...
“In a high-emissions scenario where little is done to curb planet-heating gases, global mortality rates will be raised by 73 deaths per 100,000 people by the end of the century”
https://www.theguardian.com/us-news/2020/aug/04/rising-globa...
> Climate change is absolutely not an existential risk.
That's an assessment based on what we know, but there's a lot we don't. For instance, it's possible that climate change could lead to a large scale collapse of food chains that could pose an existential risk to humanity.
It's also plausible that climate change will lead to resource shortages that result in wars between nuclear powers, and the nuclear calamity that follows kills us all.
That's an assessment based on what we know, but there's a lot we don't. For instance, it's possible that climate change could lead to a large scale collapse of food chains that could pose an existential risk to humanity.
It's also plausible that climate change will lead to resource shortages that result in wars between nuclear powers, and the nuclear calamity that follows kills us all.
"By the end of the century" is doing a lot of work in that statement.
> They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible
I really don't understand AGI skeptics. Unless you think reality has some non-physical character that can't be reproduced outside of biology, it obviously follows that a physical machine can reproduce what another physical machine (human body) is doing.
I really don't understand AGI skeptics. Unless you think reality has some non-physical character that can't be reproduced outside of biology, it obviously follows that a physical machine can reproduce what another physical machine (human body) is doing.
The biggest threat of AGI IMO is people wanting and desiring to make the leap from human to digital. It is less The Matrix and more like plastic surgery or some other lifestyle enhancement. At first they wont, like people wouldn’t jump in an unlicensed taxi because that is silly, right!
At this point I think that giving $5 to a homeless person in SF (even if they go straight to doing another shot) might be more productive than this self-aggrandizing vapid BS about some theoretical risk that might come in the future
Right now what's threatening the most people is war, famine and diseases. Climate change is a tangible future threat but I'd bet a couple $ that the JSO organizers are full EA proponents and are flying private jets to convince people to throw soup into works of art
Right now what's threatening the most people is war, famine and diseases. Climate change is a tangible future threat but I'd bet a couple $ that the JSO organizers are full EA proponents and are flying private jets to convince people to throw soup into works of art
> BS about some theoretical risk that might come in the future
Like increased rate of hurricanes due to climate change?
One does need to seriously consider low probability, high impact events, like pandemics, in one's future planning and resource allocation.
Like increased rate of hurricanes due to climate change?
One does need to seriously consider low probability, high impact events, like pandemics, in one's future planning and resource allocation.
> Like increased rate of hurricanes due to climate change?
This is a bad analogy. The mechanisms and risks of climate change are reasonably well understood as are the solutions available to us now and even some possible future solutions.
When it comes to AGI, what's the risk? Something bad might happen? The mechanisms of AGI arising is also unknown since it has never happened before and it's not even known if it's possible. Finally, what's the solution? Prevent AGI research? Make sure every AGI project has a killswitch? How do you prevent something when you don't know if it is possible nor how it will arise?
Compare this to the concrete, measurable things we can do to slow down climate change and "AGI safety" seems like a make-work money pit.
This is a bad analogy. The mechanisms and risks of climate change are reasonably well understood as are the solutions available to us now and even some possible future solutions.
When it comes to AGI, what's the risk? Something bad might happen? The mechanisms of AGI arising is also unknown since it has never happened before and it's not even known if it's possible. Finally, what's the solution? Prevent AGI research? Make sure every AGI project has a killswitch? How do you prevent something when you don't know if it is possible nor how it will arise?
Compare this to the concrete, measurable things we can do to slow down climate change and "AGI safety" seems like a make-work money pit.
> why do you disapprove of funding research into AI safety
This is what I'm replying to. And yes Climate Change is a worrying problem, but it won't be solved by people throwing tantrums in museums
This is what I'm replying to. And yes Climate Change is a worrying problem, but it won't be solved by people throwing tantrums in museums
I'm not the poster but there is no reason to believe that we possess the tools to contemplate AI safety worse it may be a fools errand to contemplate how to control something much smarter than you are.
Ergo at this point in time money spent to pay people to contemplate it is almost certainly entirely wasted.
Ergo at this point in time money spent to pay people to contemplate it is almost certainly entirely wasted.
It’s much like spending millions on studying how to save humanity from the sun when it expands in 5b years.
Interestingly, the sun would kill us much sooner than that: in less than 1 billion years the sun's increased output will heat up the Earth's surface to the point of becoming unlivable.
Related to the topics of AI apocalypse and putting things off because they seem far away: I personally find it amusing to contemplate a strong AGI being developed in 2035 and wiping out humanity. Why? Because it would eliminate the Y2038 problem, retroactively vindicating all the engineers who only allocated 32 bits to timestamps. It would mean the justification of "oh, it's so far off, the world will have changed unimaginably by then", would for once turn out to be correct!
Related to the topics of AI apocalypse and putting things off because they seem far away: I personally find it amusing to contemplate a strong AGI being developed in 2035 and wiping out humanity. Why? Because it would eliminate the Y2038 problem, retroactively vindicating all the engineers who only allocated 32 bits to timestamps. It would mean the justification of "oh, it's so far off, the world will have changed unimaginably by then", would for once turn out to be correct!
I quite like the idea of strong AI being developed in 2038 just in time for its plans to be thwarted by time disorientation...
Maybe we should try to acquire the tools before we build systems much smarter than we are
Current AI seems to be just a box of math churning out correlations that we can put to effective use but not understand wherein the ability to actually act is entirely reliant on us deliberately wiring the machine into the world in a way it can even have negative effects or any effect at all wherein said negative effects are entirely and trivially predictable and avoidable. This is true whether its a robot that accidentally breaks a kids finger or a system that bakes in the inherent racism of prior decisions into future sentencing or mortgage approvals. In most cases the logical thing is just don't use AI for that.
Its not at all clear that future actual AI will be GPT7 now with 10,000x as much processing power. In fact not much seems to be clear at all. It would seem that the insight to control future tools will come from the experience in building said tools with little chance that it will just accidentally suddenly become skynet. If we doom ourselves thus it will be a long laborious process with years of striving, setbacks, and thousands of people involved.
We should focus on the basic science and let it become clear what avenues exist over the probable decades it will take to even reach the vicinity of our goal. Money spent specifically on paying PhDs to imagine how to secure a technology we don't have and don't remotely understand is probably wasted.
Its not at all clear that future actual AI will be GPT7 now with 10,000x as much processing power. In fact not much seems to be clear at all. It would seem that the insight to control future tools will come from the experience in building said tools with little chance that it will just accidentally suddenly become skynet. If we doom ourselves thus it will be a long laborious process with years of striving, setbacks, and thousands of people involved.
We should focus on the basic science and let it become clear what avenues exist over the probable decades it will take to even reach the vicinity of our goal. Money spent specifically on paying PhDs to imagine how to secure a technology we don't have and don't remotely understand is probably wasted.
None of this seems to argue against focusing on safety tools now, given the enormous downside risk. And maybe you're wrong about the probable decades.
The possible outcomes here are "probably we waste a lot of money" and "the world ends."
The possible outcomes here are "probably we waste a lot of money" and "the world ends."
You say "justify almost anything" but I've always been surprised by the lack of distribution of wealth as an answer to helping people.
Maybe I've just missed it, but EA feels a lot like Communism if it was invented by Any Rand to me, so governments, democracy and regulation have always seemed notable by their absence.
Maybe I've just missed it, but EA feels a lot like Communism if it was invented by Any Rand to me, so governments, democracy and regulation have always seemed notable by their absence.
I think you have missed quite a bit; EA also includes "80,000 hours", which explicitly suggests government as a great way to change the world.
https://80000hours.org/career-reviews/
Although I can't imagine this aged well at all (not that I've listened to it, I've been a crypto-skeptic since I first heard about BTC): https://80000hours.org/podcast/episodes/sam-bankman-fried-hi...
https://80000hours.org/career-reviews/
Although I can't imagine this aged well at all (not that I've listened to it, I've been a crypto-skeptic since I first heard about BTC): https://80000hours.org/podcast/episodes/sam-bankman-fried-hi...
Respectfully, I think you're being a little uncharitable here.
I'm on the periphery of the EA / Longtermist scene — I have friends and customers in that space and have studied and hung out some. I recently went to one of the online events — EAGx Virtual — so I got to see what people actually care about, are working on, the methodologies being used, etc.
One way that might be helpful to think about "Effective Altruism" and "Longtermism" is that they're rather loose labels, more similar to "Democracy" than "Member of XYZ Political Party or NGO" — while there's some loose unifying threads across people who hold those views, there are very many different nuanced views about what good implementation can look like and how individual people and groups relate being effective and benevolent.
Then, the people doing the best work in that space are doing some incredible things that have already had short-term high-impact results in the world like doing hyper-rigorous work on how to cure and treat and prevent diseases in the developing world, and then actually getting those things funded and checking the results. It's also very much _not_ uniformly "just use a spreadsheet" - a lot of people there take dialogue, conversation, and engagement qualitatively to ensure good is happening very seriously.
For instance, a talk I attended at EAGx Virtual covered this - https://whatworkswellbeing.org/about-wellbeing/how-to-measur... - I haven't been able to dive deeply into the methodology, but it looks incredibly thoughtful and with many practical applications. I'm, personally, really happy people are doing work like that.
Then, simultaneously there are people in the EA/Longtermist space that are working on more speculative potential problems — pandemic preparedness, reducing the risk of nuclear war, and yes AI Safety. To the latter point, empirically a lot of people in AI Safety seem more worried about very practical can-definitely-be-anticipated risks of things like, e.g., USA-China rivalry and what types of regulations, markets, and general consensus might prevent that from going badly. Obviously highly speculative futuristic ideas are sexier and get more attention than "boring routine stuff" but international conventions against, for instance, Lethal Autonomous Weapons (LAWs) seems like a really prudent thing to at least explore and see if an international consensus could be built around similar to the Geneva or Hague conventions.
As for "EA-think" I don't agree with Aaronson's assessment. I'm n=1, but there's not a single monolithic viewpoint.
Some really exceptional work is being done by some very thoughtful people there. Undoubtedly some bad or sloppy work is being done, as is true in just about any domain. But on balance I've found the people there to be quite thoughtful and doing some really valuable things for the world with a lot of sincerity and rigor.
I'm on the periphery of the EA / Longtermist scene — I have friends and customers in that space and have studied and hung out some. I recently went to one of the online events — EAGx Virtual — so I got to see what people actually care about, are working on, the methodologies being used, etc.
One way that might be helpful to think about "Effective Altruism" and "Longtermism" is that they're rather loose labels, more similar to "Democracy" than "Member of XYZ Political Party or NGO" — while there's some loose unifying threads across people who hold those views, there are very many different nuanced views about what good implementation can look like and how individual people and groups relate being effective and benevolent.
Then, the people doing the best work in that space are doing some incredible things that have already had short-term high-impact results in the world like doing hyper-rigorous work on how to cure and treat and prevent diseases in the developing world, and then actually getting those things funded and checking the results. It's also very much _not_ uniformly "just use a spreadsheet" - a lot of people there take dialogue, conversation, and engagement qualitatively to ensure good is happening very seriously.
For instance, a talk I attended at EAGx Virtual covered this - https://whatworkswellbeing.org/about-wellbeing/how-to-measur... - I haven't been able to dive deeply into the methodology, but it looks incredibly thoughtful and with many practical applications. I'm, personally, really happy people are doing work like that.
Then, simultaneously there are people in the EA/Longtermist space that are working on more speculative potential problems — pandemic preparedness, reducing the risk of nuclear war, and yes AI Safety. To the latter point, empirically a lot of people in AI Safety seem more worried about very practical can-definitely-be-anticipated risks of things like, e.g., USA-China rivalry and what types of regulations, markets, and general consensus might prevent that from going badly. Obviously highly speculative futuristic ideas are sexier and get more attention than "boring routine stuff" but international conventions against, for instance, Lethal Autonomous Weapons (LAWs) seems like a really prudent thing to at least explore and see if an international consensus could be built around similar to the Geneva or Hague conventions.
As for "EA-think" I don't agree with Aaronson's assessment. I'm n=1, but there's not a single monolithic viewpoint.
Some really exceptional work is being done by some very thoughtful people there. Undoubtedly some bad or sloppy work is being done, as is true in just about any domain. But on balance I've found the people there to be quite thoughtful and doing some really valuable things for the world with a lot of sincerity and rigor.
Like most things: EA as a concept is fine. Finding the "maximal" way to improve the lives of others is great, and finding better ways to contribute than charity and volunteering is honorable.
The issue is when people take it to the extreme (like someone who literally argued that a person doing mere charity or volunteer work is bad, because they are wasting time and effort that they could've spent making money and recruiting to contribute more charity and volunteering), or just sidestepping "altruism" entirely and using the name and community as a stepping stone to push your unrelated scam.
I don't know whether or not SBF genuinely believed in effective altruism, but if he did, he let the "principles" override common sense.
The issue is when people take it to the extreme (like someone who literally argued that a person doing mere charity or volunteer work is bad, because they are wasting time and effort that they could've spent making money and recruiting to contribute more charity and volunteering), or just sidestepping "altruism" entirely and using the name and community as a stepping stone to push your unrelated scam.
I don't know whether or not SBF genuinely believed in effective altruism, but if he did, he let the "principles" override common sense.
> like someone who literally argued that a person doing mere charity or volunteer work is bad, because they are wasting time and effort that they could've spent making money
This is the part of EA that triggers a lot of red flags for me. Making money, or rather making significant money, necessitates participation in a system of exploitation that is largely responsible for the ills the charitable donations are meant to alleviate in the first place. Maximizing that money necessitates optimizing the exploitation.
the whole thing just seems like an exercise in morality-washing one's own greed. Lies, damn lies, and statistics perpetrated with "rationality".
This is the part of EA that triggers a lot of red flags for me. Making money, or rather making significant money, necessitates participation in a system of exploitation that is largely responsible for the ills the charitable donations are meant to alleviate in the first place. Maximizing that money necessitates optimizing the exploitation.
the whole thing just seems like an exercise in morality-washing one's own greed. Lies, damn lies, and statistics perpetrated with "rationality".
> This is the part of EA that triggers a lot of red flags for me. Making money, or rather making significant money, necessitates participation in a system of exploitation that is largely responsible for the ills the charitable donations are meant to alleviate in the first place.
I mean, given that you think that I understand where you're coming from. But just note that many, many people would highly disagree with that characterization.
I don't think our "system", whatever that means, is evil. I think it's pretty good, definitely better than other things people have come up with. And I don't think you need to give to charity to morality-wash greed. I think it's just the right thing to do.
I mean, given that you think that I understand where you're coming from. But just note that many, many people would highly disagree with that characterization.
I don't think our "system", whatever that means, is evil. I think it's pretty good, definitely better than other things people have come up with. And I don't think you need to give to charity to morality-wash greed. I think it's just the right thing to do.
You missed the major point of being the exact capitalists that are the reason for charity to be so necessary. That’s white washing greed and self interest. Get to have a better life selfishly then donate the money to be morally better.
No I didn't miss the point. I disagree with it.
Looking at other countries that don't have capitalism, everyone is worse off. I don't think capitalism is the reason charity is necessary. I think the world starts default-bad for humans, by default we don't have anywhere close to the living we want, and over the years, many technological advanacements, including capitalism, have slowly made most humans far better off.
This is fairly clear in the historical record or in comparing different countries.
(At least to me. Most people seem to disagree with this, but I honestly think they're wrong.)
Looking at other countries that don't have capitalism, everyone is worse off. I don't think capitalism is the reason charity is necessary. I think the world starts default-bad for humans, by default we don't have anywhere close to the living we want, and over the years, many technological advanacements, including capitalism, have slowly made most humans far better off.
This is fairly clear in the historical record or in comparing different countries.
(At least to me. Most people seem to disagree with this, but I honestly think they're wrong.)
> Looking at other countries that don't have capitalism, everyone is worse off.
Interesting that all I talked about was a system of exploitation and your mind immediately filled in 'capitalism'.
I think there are many countries that temper capitalism with socialist ideals and people are not worse off than the more unfettered variety practiced in the US.
Interesting that all I talked about was a system of exploitation and your mind immediately filled in 'capitalism'.
I think there are many countries that temper capitalism with socialist ideals and people are not worse off than the more unfettered variety practiced in the US.
> Interesting that all I talked about was a system of exploitation and your mind immediately filled in 'capitalism'.
That's not what happened. When I answered you I wrote "the system". I answered skinnymuch's post about capitalism, because they themselves talked about capitalism.
But out of curiosity - what "system" exactly did you have in mind?
> I think there are many countries that temper capitalism with socialist ideals and people are not worse off than the more unfettered variety practiced in the US.
Who said anything about the US?
That's not what happened. When I answered you I wrote "the system". I answered skinnymuch's post about capitalism, because they themselves talked about capitalism.
But out of curiosity - what "system" exactly did you have in mind?
> I think there are many countries that temper capitalism with socialist ideals and people are not worse off than the more unfettered variety practiced in the US.
Who said anything about the US?
The point as the OP is saying as well is that people are being unfairly exploited in capitalism. You went along with agreeing with me by arguing about capitalism which is what I called the system of exploitation as said by the OP.
Well, without capitalism everyone would live in subsistence-level poverty, but it would be a social norm, so no charity would be necessary.
The whole thing reminds me of "I, Robot", when the AI realizes that the greater good for humanity can't care for themselves, it is to be ruled by the AI for good.
This is the kind of vibe I get from this writeup, and the level of separation from humanity these people operate.
This is the kind of vibe I get from this writeup, and the level of separation from humanity these people operate.
Incidentally, literally every dictator to ever live also has come to that conclusion
Why wait for AI to take over when you can have people run the same calculations and come to the same conclusions?
Seems more Ends Justify the Means when you start talking about future lives saved
It's the modern version of a religious indulgence. Worse, as you point out, all it offers is more undue and unjustified meddling in peoples lives to make up for unfair and injudicious meddling in other peoples financial affairs.
More cynically, you're giving hand outs to people who yet need to learn to be productive, while destroying the lives of people who were trying to leverage their own productivity. I think it has negative value, doubling harming progress for the sake of apology.
More cynically, you're giving hand outs to people who yet need to learn to be productive, while destroying the lives of people who were trying to leverage their own productivity. I think it has negative value, doubling harming progress for the sake of apology.
You seem to be objecting to alturism not effective alturism.
That’s the sense I got too, especially after reading the fellatious piece on SBF from sequoia capital. Trying too hard to justify getting rich, because that would theoretically help the world, in the end, regardless of the means to get there.
>The issue is when people take it to the extreme (like someone who literally argued that a person doing mere charity or volunteer work is bad, because they are wasting time and effort that they could've spent making money and recruiting to contribute more charity and volunteering)
I mean, the concern here is that coming off as an asshole can be self-defeating, not that these views are actually incorrect, right? Seems like it's like being an atheist - an "extreme view" for many and it's a topic that may need to be dealt with with caution sometimes but that's really mostly a critique of the sensitivities of others.
I mean, the concern here is that coming off as an asshole can be self-defeating, not that these views are actually incorrect, right? Seems like it's like being an atheist - an "extreme view" for many and it's a topic that may need to be dealt with with caution sometimes but that's really mostly a critique of the sensitivities of others.
The problem is they can't possibly know if they are correct or not, we just don't have enough information about the actual future and what effect actions now might have on it.
Plus it doesn't take into account our individual competencies and nature. It doesn't matter how much I think curing cancer is more important than my current job, I know absolutely zilch about curing cancer. Also I have no idea if I'm actually right about cancer being a top priority, or if curing it is even possible. Meanwhile sick people that doctors and nurses can save using known techniques are dying right now.
Effective Altruism as a principle is fine, it makes a degree of sense. We do need to invest in long term projects, but all this talk of 1% chance of this or that saving humanity is fiddling while Rome burns.
Plus it doesn't take into account our individual competencies and nature. It doesn't matter how much I think curing cancer is more important than my current job, I know absolutely zilch about curing cancer. Also I have no idea if I'm actually right about cancer being a top priority, or if curing it is even possible. Meanwhile sick people that doctors and nurses can save using known techniques are dying right now.
Effective Altruism as a principle is fine, it makes a degree of sense. We do need to invest in long term projects, but all this talk of 1% chance of this or that saving humanity is fiddling while Rome burns.
>Plus it doesn't take into account our individual competencies and nature. It doesn't matter how much I think curing cancer is more important than my current job, I know absolutely zilch about curing cancer. Also I have no idea if I'm actually right about cancer being a top priority, or if curing it is even possible.
If curing cancer is a top priority, the EA advice is not that you should quit your job and try to cure cancer. It's probably to keep your job (which you likely have developed a comparative advantage in) and do serious research into which organizations will actually make progress towards curing cancer, and EA orgs will often try to provide advice on this latter front.
Yes, there is a big emphasis in these communities on various sorts of existential risks, but it doesn't imply that you can't be an EA if you have different beliefs on these risks, so long as those beliefs are informed by open-minded criticism and not "only privileged techbros could possibly care about this"-style sneers
If curing cancer is a top priority, the EA advice is not that you should quit your job and try to cure cancer. It's probably to keep your job (which you likely have developed a comparative advantage in) and do serious research into which organizations will actually make progress towards curing cancer, and EA orgs will often try to provide advice on this latter front.
Yes, there is a big emphasis in these communities on various sorts of existential risks, but it doesn't imply that you can't be an EA if you have different beliefs on these risks, so long as those beliefs are informed by open-minded criticism and not "only privileged techbros could possibly care about this"-style sneers
No there is also something bad about rejecting less quantifiable forms of charity as well. Movements grow out of communities and through solidarity and you build neither by mailing a check across the world and considering your obligation met.
I do think it's good to optimize money spent for these things much of the time but it's also just inhuman to have no place, or respect, for forms of charity that are based on, form and reinforce human connection.
I do think it's good to optimize money spent for these things much of the time but it's also just inhuman to have no place, or respect, for forms of charity that are based on, form and reinforce human connection.
>No there is also something bad about rejecting less quantifiable forms of charity as well.
I don't think EA argues for this. I guess it depends on what you mean by "less quantifiable."
>I do think it's good to optimize money spent for these things much of the time but it's also just inhuman to have no place, or respect, for forms of charity that are based on, form and reinforce human connection.
Describing this as "inhuman" is pretty extreme. In any case I think if you acknowledge that you're indulging in various non-charitable preferences through your donations I don't think EA would offer a serious critique beyond pointing out that you could be doing better. I think a major goal of EA though is to get people to truly wrangle with their biases here. These organizings are pragmatic - they're not trying to chide us into being moral saints, but just "giving what we can" - a popular EA slogan.
I don't think EA argues for this. I guess it depends on what you mean by "less quantifiable."
>I do think it's good to optimize money spent for these things much of the time but it's also just inhuman to have no place, or respect, for forms of charity that are based on, form and reinforce human connection.
Describing this as "inhuman" is pretty extreme. In any case I think if you acknowledge that you're indulging in various non-charitable preferences through your donations I don't think EA would offer a serious critique beyond pointing out that you could be doing better. I think a major goal of EA though is to get people to truly wrangle with their biases here. These organizings are pragmatic - they're not trying to chide us into being moral saints, but just "giving what we can" - a popular EA slogan.
The problem is the consequentialist ethics behind the position. Rigid consequentialism leads to all kinds of abhorrent views and is almost universally rejected among ethicists (with a few notable exceptions).
I don't think EA implies adherence to "rigid consequentialism". Like a lot of EA advocates are deeply into animal welfare for these reasons, but they offer advice to those that aren't. I think the weaker unifying argument is just that EA encourages more self-examination regarding whether one's charitable efforts are actually achieving one's values, and to provide advice to help achieve better alignment on this.
1. Both of them are concerns.
2. In case you are really lacking that much self awareness, yes, you are coming off as one in this conversation. You may call me "sensitive" and feel better about yourself now.
>In case you are really lacking that much self awareness, yes, you are coming off as one in this conversation. You may call me "sensitive" and feel better about yourself now.
Isn't this just tone policing then?
Isn't this just tone policing then?
Effective altruism (EA) is a philosophical and social movement that advocates "using evidence and reason to figure out how to benefit others as much as possible, and taking action on that basis". [1]
Seems like a good idea, if only just to have some sort of open-system for rating charities. I suspect the issue is everything that comes around the idea, specially... people. The problem is always people :-). Perhaps a "Michelin Guide to Charities" would be easier to market and explain? :-p
My favorite explanation of how "effective altruism" works is this SMBC comic [2]. :-)
--
1: https://en.wikipedia.org/wiki/Effective_altruism
2: https://www.smbc-comics.com/comic/2011-07-13
Seems like a good idea, if only just to have some sort of open-system for rating charities. I suspect the issue is everything that comes around the idea, specially... people. The problem is always people :-). Perhaps a "Michelin Guide to Charities" would be easier to market and explain? :-p
My favorite explanation of how "effective altruism" works is this SMBC comic [2]. :-)
--
1: https://en.wikipedia.org/wiki/Effective_altruism
2: https://www.smbc-comics.com/comic/2011-07-13
> if only just to have some sort of open-system for rating charities
While the following examples aren't exactly _open_ systems, some organizations address the niche you highlight. Examples: https://www.givewell.org/, https://animalcharityevaluators.org/
While the following examples aren't exactly _open_ systems, some organizations address the niche you highlight. Examples: https://www.givewell.org/, https://animalcharityevaluators.org/
I think _open_ is the important part:
Would make sense if anyone could come up with the score, following some "standard". That way one could make sure that no specific grader has incentives to give good scores speculatively (or even maliciously).
Would make sense if anyone could come up with the score, following some "standard". That way one could make sure that no specific grader has incentives to give good scores speculatively (or even maliciously).
I encourage you to explore givewell's site. They are very transparent about their ratings and processes.
The problem with EA and other utilitarian systems of thought is two-fold. Firstly, you can't predict the future so really you're working off your biases. Secondly, unfettered by principles other than the end justifying the means, you can easily turn those aforementioned biases into justifications for committing horrible acts. Mass murder, or evidently, theft on a massive scale.
For instance:
"The technological revolution was a disaster for the human race... so I'm going to murder a few people to draw attention to my manifesto against technology."
"Social inequality is a disaster for the majority of my countrymen, so I'm going to murder the Bourgeoisie"
"My altruism is more effective than anybody else's, so I'm going to steal everybody else's money so that I can put it to better use."
How do you stop this shit? Deontological ethics. Don't murder, because murder is wrong. Don't steal, because stealing is wrong. Damn the consequences, stick to simple principles such as these.
For instance:
"The technological revolution was a disaster for the human race... so I'm going to murder a few people to draw attention to my manifesto against technology."
"Social inequality is a disaster for the majority of my countrymen, so I'm going to murder the Bourgeoisie"
"My altruism is more effective than anybody else's, so I'm going to steal everybody else's money so that I can put it to better use."
How do you stop this shit? Deontological ethics. Don't murder, because murder is wrong. Don't steal, because stealing is wrong. Damn the consequences, stick to simple principles such as these.
Peter Singer, one of the "founders" of effective altruism, argues pretty vehemently in his book "Effective Altruism" that the ends do NOT justify the means. Like you've said, doing that has lead some people to go off the deep end and do some horrifying stuff, so just because you think you can improve the world better than the guy next to you doesn't mean you should take the guy's money.
(Although, I've googled a bit and found a "counter"-example, where Singer argues that in a case where six innocent people could be wrongly accused of murder and killed, you should point at one as the perpetrator if it means only they get killed and the other five ones get let go, therefore saving five lives: https://www.npr.org/transcripts/866768837)
So yes, killing is morally wrong and should never be the mean to your end, but the rest is really muddy. Is stealing from the rich in order to feed the poor a justified mean? Is donating most of your money you make trading off the stock market, where other people have lost that money, a justified mean? Is lobbying against climate-unfriendly companies a justified mean?
(Although, I've googled a bit and found a "counter"-example, where Singer argues that in a case where six innocent people could be wrongly accused of murder and killed, you should point at one as the perpetrator if it means only they get killed and the other five ones get let go, therefore saving five lives: https://www.npr.org/transcripts/866768837)
So yes, killing is morally wrong and should never be the mean to your end, but the rest is really muddy. Is stealing from the rich in order to feed the poor a justified mean? Is donating most of your money you make trading off the stock market, where other people have lost that money, a justified mean? Is lobbying against climate-unfriendly companies a justified mean?
> Peter Singer, one of the "founders" of effective altruism, argues pretty vehemently in his book "Effective Altruism" that the ends do NOT justify the means.
Even without such an argument, “the ends justify the means” isn’t a reason to do something in the real world, because there are no “ends” in the real world. Everything just keeps happening.
Even without such an argument, “the ends justify the means” isn’t a reason to do something in the real world, because there are no “ends” in the real world. Everything just keeps happening.
>Firstly, you can't predict the future so really you're working off your biases.
Um, are you saying that this implies that people should just throw up their hands and donate to whatever causes since who knows? Surely not. Once we acknowledge that these questions can be tackled empirically, that's what EA tries to do. Is there room for bias in this process? Absolutely. Does that justify embracing some sort of epistemic nihilism? Absolutely not.
Um, are you saying that this implies that people should just throw up their hands and donate to whatever causes since who knows? Surely not. Once we acknowledge that these questions can be tackled empirically, that's what EA tries to do. Is there room for bias in this process? Absolutely. Does that justify embracing some sort of epistemic nihilism? Absolutely not.
> > Firstly, you can't predict the future so really you're working off your biases.
> Um, are you saying that this implies that people should just throw up their hands and donate to whatever causes since who knows?
No, it implies that you shouldn't lecture others that you have the one true answer as to the best path forwards. You are free to (and in fact should) use your best judgment in your own personal choices, just don't confuse that with absolute truth.
> Um, are you saying that this implies that people should just throw up their hands and donate to whatever causes since who knows?
No, it implies that you shouldn't lecture others that you have the one true answer as to the best path forwards. You are free to (and in fact should) use your best judgment in your own personal choices, just don't confuse that with absolute truth.
I don't think EA institutions argue that they have the One True Answer. They just often give recommendations - "eg. if you care about maximizing QALY overall, we recommend charities X, Y, and Z." If I don't care about QALY maximization, fine. But if I do, why would I not listen to the recommendations?
Longtermism and QALY do not seem compatible.
They do if you legitimately think the long-term thing will kill everyone.
I don't think that myself, but it's internally consistent.
I don't think that myself, but it's internally consistent.
> Firstly, you can't predict the future so really you're working off your biases
Obviously. That is the human condition.
Do you not take actions in normal life because you don't have full knowledge of everything? Don't order breakfast because you don't know which menu item is for sure going to be the best.
> How do you stop this shit? Deontological ethics. Don't murder, because murder is wrong. Don't steal, because stealing is wrong. Damn the consequences, stick to simple principles such as these.
Simple rules like: dont hoard your money but find a worthy cause for it?
Obviously. That is the human condition.
Do you not take actions in normal life because you don't have full knowledge of everything? Don't order breakfast because you don't know which menu item is for sure going to be the best.
> How do you stop this shit? Deontological ethics. Don't murder, because murder is wrong. Don't steal, because stealing is wrong. Damn the consequences, stick to simple principles such as these.
Simple rules like: dont hoard your money but find a worthy cause for it?
> "My altruism is more effective than anybody else's, so I'm going to steal everybody else's money so that I can put it to better use."
> How do you stop this shit?
If you ask yourself why you think "this shit" is bad, at root there will be a consequentialist reason. Deontological ethics is consequentialism by heuristics.
> How do you stop this shit?
If you ask yourself why you think "this shit" is bad, at root there will be a consequentialist reason. Deontological ethics is consequentialism by heuristics.
But if you ever try to put any consequentialist theories into practice, at root there will just be a set of deontological heuristics guiding your behavior. Consequentialism is just deontology in drag.
You're right. We are unable to be full consequentialist because the computational cost of seeing all ends is infinite, and unable to be full deontologist because competing heuristics are arbitrarily instantiated and mutually contradictory. Even if we're stuck in the middle forever, I know which way I'd strive.
Real heuristics, of course, are not arbitrary, they are the result of trial and error processes where societies and people with differing moral systems compete with each other, proving the fitness of their heuristic by surviving while less fit heuristics fail.
I'll go with the results of trial and error over long time horizons rather than blank slate theoretical calculations any day.
I'll go with the results of trial and error over long time horizons rather than blank slate theoretical calculations any day.
Heuristic selection as you describe is evolutionary. So the meme that survives may not have any bearing on human flourishing, just on its own viral ability to spread. Fitness of the heuristic is orthogonal to human benefit.
Further, when multiple heuristics are directly in conflict, it is up to the individual to make the final moral decision. The choice to reject one heuristic over another must be based in some moral calculus.
Finally, who said anything about blank slate? That's a clear straw man, as it's only logical for consequentialists to take current heuristics and their adherents into account.
Further, when multiple heuristics are directly in conflict, it is up to the individual to make the final moral decision. The choice to reject one heuristic over another must be based in some moral calculus.
Finally, who said anything about blank slate? That's a clear straw man, as it's only logical for consequentialists to take current heuristics and their adherents into account.
I agree with your second point that having deontological boundaries is important. Interestingly, people like Eliezer Yudkowsky (generally respected in the EA movement, I'm pretty sure?) also agree with you about that. He's been reiterating the point recently on twitter, though this post [1] says pretty much the same thing and it's from 2008. Generally, I'd attribute your examples of "utilitarianism gone wrong" to mental illness or sociopathy rather than to holding a utilitarian philosophy. For the average person, moral philosophy is much more of a way to explain the choices we make after the fact than a thing that actually determines the choices.
[1] https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...
[1] https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...
Deontological ethics is worse than useless as aguide to doing good.
It only tells you what not to do, rather than what to do.
And it's full of exceptions and edge cases. Is stealing a loaf of bread from scrooge to stop a family from starving wrong? What if you call it "taxes" and the community voted on it? Is it murder to blow up a bridge in wartime? Is unplugging someone's life support murder? What if they're going to die in hours any way and you're out of life support machines for the patient who just came in and has good prospects?
Also, while we can't predict the future we all need to decide what to do. Sure, if you donate antimosquito nets to save kids from malaria one of them might grow up to become the next Hitler, but you should try and figure out on the balance of probability what the most likely outcomes are.
It only tells you what not to do, rather than what to do.
And it's full of exceptions and edge cases. Is stealing a loaf of bread from scrooge to stop a family from starving wrong? What if you call it "taxes" and the community voted on it? Is it murder to blow up a bridge in wartime? Is unplugging someone's life support murder? What if they're going to die in hours any way and you're out of life support machines for the patient who just came in and has good prospects?
Also, while we can't predict the future we all need to decide what to do. Sure, if you donate antimosquito nets to save kids from malaria one of them might grow up to become the next Hitler, but you should try and figure out on the balance of probability what the most likely outcomes are.
Sorry, it is not a "fine" concept.
It is just utilitarianism with modern marketing and pure utilitarianism has problems.
See one example below where utilitarianism's suggestion goes against what almost everyone considers ethical:
You can take 10 healthy organs from one person without any social connections and use the organs to save 10 terminal patients who would otherwise die.
EA effectively sanctions robbing a million people to help a million people + 1
It is just utilitarianism with modern marketing and pure utilitarianism has problems.
See one example below where utilitarianism's suggestion goes against what almost everyone considers ethical:
You can take 10 healthy organs from one person without any social connections and use the organs to save 10 terminal patients who would otherwise die.
EA effectively sanctions robbing a million people to help a million people + 1
EA doesn't do that - it's a much smaller concept than utilitarianism. (Which also wouldn't always support that conclusion.)
But it has very noticeably begun falling into the same traps long discovered and recognized by utilitarianism. EA now has a significant contingent that is most focused on very unlikely far future outcomes because they would be incredibly bad for an unimaginably vast number of people if they did occur eg unbounded utility. That's pascal's mugging!
The movement is based on the same ideas and values and it's running into the same problems. These similarities don't inherently discredit the movement, but not having a clear awareness of these faults in their framework and a plan for avoiding them is very worrying.
The movement is based on the same ideas and values and it's running into the same problems. These similarities don't inherently discredit the movement, but not having a clear awareness of these faults in their framework and a plan for avoiding them is very worrying.
> EA now has a significant contingent that is most focused on very unlikely far future outcomes because they would be incredibly bad for an unimaginably vast number of people if they did occur eg unbounded utility. That's pascal's mugging!
Indeed! https://en.wikipedia.org/wiki/Pascal%27s_mugging
(Although the term itself comes from the EA community)
Indeed! https://en.wikipedia.org/wiki/Pascal%27s_mugging
(Although the term itself comes from the EA community)
I am very confused about why the rationalists, who first identified the concept of pascal's mugging seem to have a huge number of initiatives which seem to be examples of falling directly into the pascal's mugging trap.
With all of these examples, and pascal's mugging itself, there is an obvious error: there should be a practical upper bound on the adverse outcome you're concerned about, at which all higher values are effectively the same to you.
With all of these examples, and pascal's mugging itself, there is an obvious error: there should be a practical upper bound on the adverse outcome you're concerned about, at which all higher values are effectively the same to you.
> I am very confused about why the rationalists, who first identified the concept of pascal's mugging seem to have a huge number of initiatives which seem to be examples of falling directly into the pascal's mugging trap.
The original rationalists were true rationalist. And the recent ones? I am not so sure.
The original rationalists were true rationalist. And the recent ones? I am not so sure.
> Which also wouldn't always support that conclusion
This is a well known issue:
https://plato.stanford.edu/entries/consequentialism/
https://blog.apaonline.org/2021/03/29/is-effective-altruism-...
> As such, Effective Altruism is inherently Utilitarian in character, even if the views are themselves distinct. Criticisms of Effective Altruism on the basis of critiques to Utilitarianism are not only expected, but fair.
This is a well known issue:
https://plato.stanford.edu/entries/consequentialism/
https://blog.apaonline.org/2021/03/29/is-effective-altruism-...
> As such, Effective Altruism is inherently Utilitarian in character, even if the views are themselves distinct. Criticisms of Effective Altruism on the basis of critiques to Utilitarianism are not only expected, but fair.
[deleted]
That example is a bad one. There are many serious risks in trying to do that in practice, which is why our intuitions are against it. E.g. you might be found out and people might mistrust doctors, you might be wrong about the person having to connections, those patients might not live long anyway, etc etc
> EA effectively sanctions robbing a million people to help a million people + 1
Yes, EA people often think taxes are a good idea. This is not a rare thing.
> EA effectively sanctions robbing a million people to help a million people + 1
Yes, EA people often think taxes are a good idea. This is not a rare thing.
> That example is a bad one.
No, just saying that it is a bad example doesn't make it.
I wish EA folks looked at the literature that discusses this very example.
> There are many serious risks in trying to do that in practice, which is why our intuitions are against it. E.g. you might be found out and people might mistrust doctors, you might be wrong about the person having to connections, those patients might not live long anyway, etc etc
Yes, that has been discussed thousands of times in the literature.
To answer, you can always have the person confirm they are ok with it (e.g. Switzerland allows physician-assisted suicide in some case).
No, just saying that it is a bad example doesn't make it.
I wish EA folks looked at the literature that discusses this very example.
> There are many serious risks in trying to do that in practice, which is why our intuitions are against it. E.g. you might be found out and people might mistrust doctors, you might be wrong about the person having to connections, those patients might not live long anyway, etc etc
Yes, that has been discussed thousands of times in the literature.
To answer, you can always have the person confirm they are ok with it (e.g. Switzerland allows physician-assisted suicide in some case).
> Yes, that has been discussed thousands of times in the literature.
Yes, and that's why the strawman is bad.
EA folks don't in practice follow or advocate for any form of ethical system that leads to doctors killing people for their organs.
Yes, and that's why the strawman is bad.
EA folks don't in practice follow or advocate for any form of ethical system that leads to doctors killing people for their organs.
> Yes, and that's why the strawman is bad.
No, thats why it is good. It is discussed thousands of times *because* none of the solutions are satisfactory.
> EA folks don't in practice follow or advocate for any form of ethical system that leads to doctors killing people for their organs.
But the axioms they evangelize imply that. And this is essentially what Bankman has done financially.
No, thats why it is good. It is discussed thousands of times *because* none of the solutions are satisfactory.
> EA folks don't in practice follow or advocate for any form of ethical system that leads to doctors killing people for their organs.
But the axioms they evangelize imply that. And this is essentially what Bankman has done financially.
I think that longtermism (mentioned in the article) is bad though, both as a concept and in practice.
What counts as longtermism though? Preserving archaeological sites rather than bulldozing them? Fighting CO2 emissions? Trying to avoid AGI risk you think will happen in less than a century?
The longtermism as thought of by Bostrom etc is a view that (potential) future lives are as valuable as lives now.
Part of the danger is the name. I support long term thinking! I love things like the Long Now project! But this specific viewpoint is very wrong.
Part of the danger is the name. I support long term thinking! I love things like the Long Now project! But this specific viewpoint is very wrong.
[deleted]
As an Effective Altruist I want to cure cancer, so I maximize my income by selling cigarettes to children outside playgrounds. The money will eventually be donated to the cause in my will, and in the meantime the prevalence of lung cancer in teens will incentivise the Free Market™ to find a cure.
No need to thank me, really.
No need to thank me, really.
Wow! Bless this man. Now minimize harm by targeting demographics of children who are less likely to grow up to be effectively altruistic!
You jest, but that would absolutely make sense if you were a psychopath with the singular goal of finding a cure for cancer. Now use this same mindset but change the goal for something saner along the lines of "maximize healthy+happy lifespan for everyone", and then you won't sell cigarettes in playgrounds, since that harms the goal.
> "maximize healthy+happy lifespan for everyone"
Okay, so then I fund pro-life groups to protect the lives/happiness of fetuses. And the EA activist down the street funds pro-choice groups to protect the lives/happiness of mothers. And we keep escalating such donations while having little net effect.
IMHO EA has more than a hint of authoritarian (or at least anti-democratic) tendencies. What gives some rich guy the right to decide what would maximize happiness for others? Why not just leave that money with the people who are best positioned to make those decisions for themselves my minimizing wealth inequality instead?
Okay, so then I fund pro-life groups to protect the lives/happiness of fetuses. And the EA activist down the street funds pro-choice groups to protect the lives/happiness of mothers. And we keep escalating such donations while having little net effect.
IMHO EA has more than a hint of authoritarian (or at least anti-democratic) tendencies. What gives some rich guy the right to decide what would maximize happiness for others? Why not just leave that money with the people who are best positioned to make those decisions for themselves my minimizing wealth inequality instead?
> Why not just leave that money with the people who are best positioned to make those decisions for themselves my minimizing wealth inequality instead?
Great idea! Unsurprisingly, pro-choice and pro-life groups are not in the EA top recommended charities. GiveDirectly, which leaves money with the people who are best positioned to make decisions for themselves, is. There might be a lot of authoritarian EAs on forums promoting AI safety, but please don't judge the whole group by a vocal minority.
Great idea! Unsurprisingly, pro-choice and pro-life groups are not in the EA top recommended charities. GiveDirectly, which leaves money with the people who are best positioned to make decisions for themselves, is. There might be a lot of authoritarian EAs on forums promoting AI safety, but please don't judge the whole group by a vocal minority.
Maybe this is superficial, but I was put off by the name "effective altruism". By naming it such, it felt like an implicit dig that most altruism is not effective. Like oh your altruism is not that productive, but my altruism, well it's effective altruism. Especially when that notion is pushed by people who overlap with the rationalist subculture, another arrogantly named group.
In general I find it concerning when technical people encroach upon non-technical areas without giving a sufficient amount of respect to the people who were in the field already. Those people, the experts, do know a lot and to have this kind of nerd savior branding of effective altruism, as if you are bringing the enlightenment that allows these poor altruists to finally be effective, well that just seems a little presumptuous.
In general I find it concerning when technical people encroach upon non-technical areas without giving a sufficient amount of respect to the people who were in the field already. Those people, the experts, do know a lot and to have this kind of nerd savior branding of effective altruism, as if you are bringing the enlightenment that allows these poor altruists to finally be effective, well that just seems a little presumptuous.
I mean, I think it was a very explicit dig.
The founding idea is that giving $10,000 to save, y’know, four ailing dogs with bad livers or whatever, or a community theater, as opposed to many human lives, is fundamentally wrong.
“If you’re gonna give, you should maximize the impact as measured in human life” is fundamentally at odds with many common forms of giving, yes.
The founding idea is that giving $10,000 to save, y’know, four ailing dogs with bad livers or whatever, or a community theater, as opposed to many human lives, is fundamentally wrong.
“If you’re gonna give, you should maximize the impact as measured in human life” is fundamentally at odds with many common forms of giving, yes.
The example of a community theater highlights another issue I have with effective altruism. It effectively encourages you to only donate to things you can make a quantifiable case for. And in a lot of situations, quantification fails. The arts have an immense value in developing philosophical, emotional, empathetic values. A theater may not show up in a balance sheet but it can have a massive effect. And it does concern me deeply that the flippant example for "non-effective altruism" is community theater. It reflects the attitude of SBF that a book is not worth reading. That is a profoundly destructive mindset.
> The example of a community theater highlights another issue I have with effective altruism. It effectively encourages you to only donate to things you can make a quantifiable case for.
Then you will be delighted/horrified to find out that the effective altruism community doesn't feel the need to make quantifiable justifications for how they spend their own donations, as evidenced by this $20k grant they gave someone to learn to ride a bike and think about AI[1].
That's about 3853 to 4090 mosquito nets' worth of donations, using GiveWell's numbers for the Against Malaria Foundation[2].
[1] https://forum.effectivealtruism.org/posts/CJJDwgyqT4gXktq6g/...
[2] https://www.givewell.org/charities/amf
Then you will be delighted/horrified to find out that the effective altruism community doesn't feel the need to make quantifiable justifications for how they spend their own donations, as evidenced by this $20k grant they gave someone to learn to ride a bike and think about AI[1].
That's about 3853 to 4090 mosquito nets' worth of donations, using GiveWell's numbers for the Against Malaria Foundation[2].
[1] https://forum.effectivealtruism.org/posts/CJJDwgyqT4gXktq6g/...
[2] https://www.givewell.org/charities/amf
I am not sure exactly what people miss, but taking a stab in the dark - human emotions have evolved over millions of years to try and guide people in the direction of making rational decisions more often than not. The expected situation is that the rational option and the emotional option align. If they don't, that means:
1. Your model of the world is incomplete or inconsistent (eg, forgetting to include negotiation as a strategy, forgetting long term consequences).
2. You've hit on a cognitive bias and can probably do a huge amount of good that other people will miss (eg, lending with interest -> modern industrial society, rule of law rather than letting high status people do whatever -> mass prosperity). The cognitive bias is probably well known and understood and the evidence that the emotional response leads to poor outcomes should be obvious.
1. Your model of the world is incomplete or inconsistent (eg, forgetting to include negotiation as a strategy, forgetting long term consequences).
2. You've hit on a cognitive bias and can probably do a huge amount of good that other people will miss (eg, lending with interest -> modern industrial society, rule of law rather than letting high status people do whatever -> mass prosperity). The cognitive bias is probably well known and understood and the evidence that the emotional response leads to poor outcomes should be obvious.
> I am not sure exactly what people miss, but taking a stab in the dark - human emotions have evolved over millions of years to try and guide people in the direction of making rational decisions more often than not. The expected situation is that the rational option and the emotional option align.
I'm completely befuddled by this. My experience is that people make most decisions emotionally and then harness their brain to rationalize those irrational decisions after the fact. It's the reason that science (the process) has to exist. Because people will fool themselves to reach a conclusion they want if there are no procedural guardrails.
I'm completely befuddled by this. My experience is that people make most decisions emotionally and then harness their brain to rationalize those irrational decisions after the fact. It's the reason that science (the process) has to exist. Because people will fool themselves to reach a conclusion they want if there are no procedural guardrails.
Mass prosperity -> too many entitled folks -> mass destruction of environment -> meta vr as alternative environment in post apocalyptic hellscape
SBF gave $40 million - 800,000 mosquito nets to candidates in politcal races.
Surely 800,000 mosquito nets is "more effective" than 800,000 robo calls and shitty campaign ads with a moron at the end saying "I'm Joe Blow, and I endorse this bullshit"
Surely 800,000 mosquito nets is "more effective" than 800,000 robo calls and shitty campaign ads with a moron at the end saying "I'm Joe Blow, and I endorse this bullshit"
This doesn't seem evidence that EA is bad so much that SBF was a bad effective altruist?
The problem is EA community rallied around SBF while he was doing this stuff publicly.
The other thing is that the actual standard of the "quantitative case" put forward is hugely variable anyway. With sufficiently inventive fantasy numbers, theatre enthusiasts could probably make an EA case of the form: "what if a person was so inspired by watching my amateur dramatic production of 2001: A Space Odyssey, that they solved all Unfriendly AI problems?". Sure, it's unlikely, but the rewards benefit an almost infinite number of people!
In practice, I think people coming from a starting point of "I'm going to focus on solving friendly AI because it fits with my interests and computer science background and I think I'll be able to help the process along a bit" are being more honest with themselves than people dressing up their preferences in magical utilitarian numbers which conclude they're more efficient than merely feeding people or giving them goats...
In practice, I think people coming from a starting point of "I'm going to focus on solving friendly AI because it fits with my interests and computer science background and I think I'll be able to help the process along a bit" are being more honest with themselves than people dressing up their preferences in magical utilitarian numbers which conclude they're more efficient than merely feeding people or giving them goats...
Let me preface this with I know this is a mugging, I think community theater is great, and I don't think it's good to divert all societal resources to nets in Africa.
But I just don't think it's as effective, and I don't think most people agree.
If the only available labor was teenagers, rebuilding the community theater meant hiring them, and you knew that statistically there was a risk some might die from heat stroke. What risk of death would you deem acceptable to build a community theater?
But I just don't think it's as effective, and I don't think most people agree.
If the only available labor was teenagers, rebuilding the community theater meant hiring them, and you knew that statistically there was a risk some might die from heat stroke. What risk of death would you deem acceptable to build a community theater?
Sorry, I'm confused as to what you mean by a mugging? Genuinely not sure what you meant by that.
Anyways, I never said any specifics about where this community theater is being built. I don't know where heat stroke came from. Sure, yeah, in the straw man situation of building a community theater in an extremely hot area, where the only labor is teenagers, I would...not enlist teenagers to build a community theater in an extremely hot area? I don't want to be dismissive of your comment but I genuinely don't know what's the point here.
Anyways, I never said any specifics about where this community theater is being built. I don't know where heat stroke came from. Sure, yeah, in the straw man situation of building a community theater in an extremely hot area, where the only labor is teenagers, I would...not enlist teenagers to build a community theater in an extremely hot area? I don't want to be dismissive of your comment but I genuinely don't know what's the point here.
Pascal’s mugging is a popular thought experiment amongst rationalists and EA people.
https://en.m.wikipedia.org/wiki/Pascal%27s_mugging
The point is you wouldn't endanger the lives of teenagers to build a community theater because you believe the lives of teenagers are more valuable than a community theater.
But repairing a community theater could easily cost $100,000 which would save the lives of ~22 people in Guinea if donated to certain bed net charities. And even the most die hard theater fans probably wouldn't save a theater if to do so it would require risking 22 lives for.
So in this case quantification isn't failing, it's succeeding. It's correctly telling us which donations are more effective. And all our intuition agrees that the $100,000 theater is worth less than the 22 lives.
But repairing a community theater could easily cost $100,000 which would save the lives of ~22 people in Guinea if donated to certain bed net charities. And even the most die hard theater fans probably wouldn't save a theater if to do so it would require risking 22 lives for.
So in this case quantification isn't failing, it's succeeding. It's correctly telling us which donations are more effective. And all our intuition agrees that the $100,000 theater is worth less than the 22 lives.
The same argument means that any luxury that any person indulges in.
Let’s say thanksgiving, christmas, craft beer, VR headset, calling their mother, etc. is costing X lives (X is not necessary integer) and so isn’t the rationalist thing to do.
“Ah bit they are luxuries, distinct from charity…”
Ok then give me the luxury of donating to the local theatre!
“Ah bit they are luxuries, distinct from charity…”
Ok then give me the luxury of donating to the local theatre!
I think this is fitting in that context:
Schindler's List | "I Didn't Do Enough"
https://www.youtube.com/watch?v=W9vj2Wf57rQ
In the end, nearly everyone values their life higher than the life of others. The more distant that the "others" are the less we value them.
Schindler's List | "I Didn't Do Enough"
https://www.youtube.com/watch?v=W9vj2Wf57rQ
In the end, nearly everyone values their life higher than the life of others. The more distant that the "others" are the less we value them.
But the theatre has a chance of spawning a future movie star who also has a chance of being an EA advocate. Directly through their $50million movie paycheques they will be saving 10,000 lives at a future date.
Thus by chosing to repair the community theatre you are directly causing the death of 9,978 people.
Thus by chosing to repair the community theatre you are directly causing the death of 9,978 people.
No. This ignores the effect of donating locally and helping local causes. Your comparison does not account for global differences in needs.
This game can go on endlessly. What if hiring the teenagers mean they’re getting a salary, can help their families, and without the job they’d run off into the jungle to join the warlord who runs a murder cult army? Turns out the community theatre prevented a massacre and one of the kids wins the Nobel Prize for her plays in fifty years.
I don't understand the purpose of your what if.
Mine was specifically to be an intuition pump to see how valuable the OP thinks a theater is when weighed against actual lives.
It's easy to say x is important where x is saving sea turtles, teaching kids to read, or repairing a community theaters.
It's harder to say x is more important than y charity when y charity saves lives effectively.
Mine was specifically to be an intuition pump to see how valuable the OP thinks a theater is when weighed against actual lives.
It's easy to say x is important where x is saving sea turtles, teaching kids to read, or repairing a community theaters.
It's harder to say x is more important than y charity when y charity saves lives effectively.
You did straw man the example though. It's very different to say that building the theater will kill the workers, as to say that the money could be used elsewhere to save someone in an immediately dire situation.
Building a community theater with good working conditions might have a big impact on the community around it and could go end up saving lives down the road, or dramatically improving the quality of life of many.
It's very hard to quantity the impact of the community theater against say feeding 100 people for one week. All these people could die the week after, was it really sustainable improvements or just temporary relief?
Building a community theater with good working conditions might have a big impact on the community around it and could go end up saving lives down the road, or dramatically improving the quality of life of many.
It's very hard to quantity the impact of the community theater against say feeding 100 people for one week. All these people could die the week after, was it really sustainable improvements or just temporary relief?
[deleted]
> The arts have an immense value in developing philosophical, emotional, empathetic values.
Too bad these empathetic values privilege building an entertainment venue over saving lives or preventing suffering. /s
Too bad these empathetic values privilege building an entertainment venue over saving lives or preventing suffering. /s
...when did this become an either/or? We can do both. In fact diversification of one's "ethics portfolio" seems like something an effective altruist would understand quite well.
People have a finite amount of money to give, so every dollar is an either/or decision.
Every dollar you spend on a theater donation is a dollar less that you value human lives.
Of course, the same logic applies to candy bars, internet, television, and all the things that being you joy in life.
Every dollar you spend on a theater donation is a dollar less that you value human lives.
Of course, the same logic applies to candy bars, internet, television, and all the things that being you joy in life.
But much like with investing in stock I don't know for sure a priori which investment will bring the highest return. If I'm 80% sure that charity A is better than charity B (for my definition of 'better) I'll give 80% of my investment to charity A and 20% to charity B.
When comparing mosquito nets versus theater, I don't really think the stock diversification analogy fits at all.
Some of those things (investments) will give you the dollar back later, and might be considered better ideas on that basis.
But that’s what leads to “earn to give”, which is what leads to deciding to become a billionaire and/or give away other people’s money instead.
But that’s what leads to “earn to give”, which is what leads to deciding to become a billionaire and/or give away other people’s money instead.
> when did this become an either/or?
I don't have unlimited resources. I have the privilege of having a surplus that I can redirect to making the world better. EA is a framework for trying to do the most amount of good from the limited resources that I have.
> We can do both.
So, save fewer lives for more theatre seats?
> diversification of one's "ethics portfolio"
I don't understand what that is supposed to mean.
I don't have unlimited resources. I have the privilege of having a surplus that I can redirect to making the world better. EA is a framework for trying to do the most amount of good from the limited resources that I have.
> We can do both.
So, save fewer lives for more theatre seats?
> diversification of one's "ethics portfolio"
I don't understand what that is supposed to mean.
So, if you had to choose between funding community theater and saving X lives, how large would X need to be before you'd abandon the theater?
It's fine to spend money on the theater, everyone spends money and time on themselves and their interests. If you consider community theater altruism though, I think it makes sense to acknowledge that it is ineffective altruism - that is, you're doing a little bit of good with your money, but not nearly as much as you might be doing.
It's fine to spend money on the theater, everyone spends money and time on themselves and their interests. If you consider community theater altruism though, I think it makes sense to acknowledge that it is ineffective altruism - that is, you're doing a little bit of good with your money, but not nearly as much as you might be doing.
"saving a life" is kind of vacuous.
You need to contextualize the life saved, do we mean they went to live one more day, or a full long life of prosperity and comfort with a happy painless death from old age.
And did they themselves gain the means to save more lives in turn?
For example, I feel giving people the means of saving their own life, or other people's lives a much more efficient way to save lives than actually saving a life.
And finally you have to address the sacrifices you've made and the people you've hurt in pursuit of saving that life.
To me, it looks like a charade, because the claim that you can actually measure and quantify all of these complex systems to truly arrive at the real impact of all these things, I just don't believe it. It seems you've picked a simple metric to measure, because it's easy, but is it lasting impact, is really more meaningful impact, that's all all still up in the air.
You need to contextualize the life saved, do we mean they went to live one more day, or a full long life of prosperity and comfort with a happy painless death from old age.
And did they themselves gain the means to save more lives in turn?
For example, I feel giving people the means of saving their own life, or other people's lives a much more efficient way to save lives than actually saving a life.
And finally you have to address the sacrifices you've made and the people you've hurt in pursuit of saving that life.
To me, it looks like a charade, because the claim that you can actually measure and quantify all of these complex systems to truly arrive at the real impact of all these things, I just don't believe it. It seems you've picked a simple metric to measure, because it's easy, but is it lasting impact, is really more meaningful impact, that's all all still up in the air.
Our moral intuitions are developed for dealing with actual cases that we are actually experiencing. If you showed up with a fire hose and found that a bus full of X people and an empty community theater were on fire - it would not even occur to you to save the theater. You would save the screaming people who are begging for help.
One way to think about effective altruism is that it's trying to get you to apply whatever moral intuitions you would use in a concrete scenario to an abstract scenario. To any individual apart from it, it's abstract to know that mothers in some poor country can't adequately nourish their children, or that mosquito nets reduce malaria deaths, and so on. But, the reality is that those people are actually real, are actually suffering, and actually could be helped. If you take that message to heart then it won't make sense to send your charity to the community theater - by all means, send the theater money if you want, but it's a selfish thing, not an altruistic thing.
One way to think about effective altruism is that it's trying to get you to apply whatever moral intuitions you would use in a concrete scenario to an abstract scenario. To any individual apart from it, it's abstract to know that mothers in some poor country can't adequately nourish their children, or that mosquito nets reduce malaria deaths, and so on. But, the reality is that those people are actually real, are actually suffering, and actually could be helped. If you take that message to heart then it won't make sense to send your charity to the community theater - by all means, send the theater money if you want, but it's a selfish thing, not an altruistic thing.
> it would not even occur to you to save the theater. You would save the screaming people who are begging for help.
If there was a crypto trade available to make a billion dollars available for the next 15 minutes only - but you have to chose between the community theater, the bus of X people on fire or giving 10% of your gainz to a charity to save X*2 ...
If there was a crypto trade available to make a billion dollars available for the next 15 minutes only - but you have to chose between the community theater, the bus of X people on fire or giving 10% of your gainz to a charity to save X*2 ...
The issue as I see it is that you cannot say that the first is ineffective unless you can quantify the ROI of the theater. If I knew that my donation will fund the next Romeo and Juliet, then I'd expect X to be really high before I deprive future generations of such an important work.
Art in particular has a multiplicative effect that is essentially impossible to quantify. Once a work of art is out there, it can live forever and influence multiple generations. That seems like a pretty effective use of my money.
Art in particular has a multiplicative effect that is essentially impossible to quantify. Once a work of art is out there, it can live forever and influence multiple generations. That seems like a pretty effective use of my money.
I agree arts and culture are valuable. The obvious objection now is why not build 5 theatres in Africa?
Obviously the local theatre benefits your own community, but if you live there then you are also benefiting yourself (via status and seeing plays), which seems less than altruistically pure, even if you could only build 1 theatre in Africa.
Obviously the local theatre benefits your own community, but if you live there then you are also benefiting yourself (via status and seeing plays), which seems less than altruistically pure, even if you could only build 1 theatre in Africa.
A theater won't have the same impact everywhere. You likely don't know anything about the neighborhoods in Africa, and how impactful a theater there could be.
You know a lot better what a theater could bring to a neighborhood you know well.
And then, there's the question of if altruism itself is most efficient at helping others. Some would argue that helping oneself but where it benefits others as well, win/win situations, in practice produces better outcomes. The idea of self-sacrifice, or having to lose or risk something personal to help others sounds noble, but might not be most effective.
You know a lot better what a theater could bring to a neighborhood you know well.
And then, there's the question of if altruism itself is most efficient at helping others. Some would argue that helping oneself but where it benefits others as well, win/win situations, in practice produces better outcomes. The idea of self-sacrifice, or having to lose or risk something personal to help others sounds noble, but might not be most effective.
Oh my god, it ENCOURAGES you to do something it believes is right and you don't. This is outright CRIMINAL. How do they dare do something so audacious?
Sorry for the cringe, but really - you're actually saying that encouraging people to do what they think is right is a problem? Especially when they've actually read the books, done the numbers, and all you have is a feeling that they may be wrong in some cases? I can't really argue with that except with caps lock and sarcasm.
Sorry for the cringe, but really - you're actually saying that encouraging people to do what they think is right is a problem? Especially when they've actually read the books, done the numbers, and all you have is a feeling that they may be wrong in some cases? I can't really argue with that except with caps lock and sarcasm.
“A feeling someone is wrong” can easily be more correct than “doing the numbers”.
Are eggplants fruits? Yes. Do you feel you should put them in a fruit cup? No.
This is called “the map is not the territory”.
Are eggplants fruits? Yes. Do you feel you should put them in a fruit cup? No.
This is called “the map is not the territory”.
It's worth mentioning that organized EA is much more concerned with animal welfare and animal rights than a lot of the surrounding culture, so many EAs strongly advocate giving to animal-related causes. But they would still be concerned with having as large an impact as possible, so something to help 1,000,000 animals would be favored over something to help four.
That’s a good point, I should have picked my examples better. Thanks for the clarification.
The idea that you can precisely plan out the exact optimal path is arrogant and fundamentally wrong.
What if the friendships made at a community theater help a troubled kid through a rough patch and instead of committing suicide they go on to cure cancer? Not funding the theater would have been very sub-optimal.
Even without far-fetched hypotheticals what’s the point of saving everyone’s life if they’re just going to spend it figuring out how to make more mosquito nets or whatnot?
People are not as smart as they think and it’s important to leave room for serendipity and random associations.
What if the friendships made at a community theater help a troubled kid through a rough patch and instead of committing suicide they go on to cure cancer? Not funding the theater would have been very sub-optimal.
Even without far-fetched hypotheticals what’s the point of saving everyone’s life if they’re just going to spend it figuring out how to make more mosquito nets or whatnot?
People are not as smart as they think and it’s important to leave room for serendipity and random associations.
And do you think the probability of the thing you describe (or anything analogous to it but still arising from the same intervention) is substantial?
I’m not claiming that all charity should be done in such an attempt-to-be-expected-utility-maximizing way, as there may be some like, component of how donations for more local causes might do more to improve the character of the person donating, and other things like that,
but I don’t see your view of “but-serendipity, and but-mosquito-nets-is-no-fun” don’t seem like compelling reasons to not look at QALYs and such.
That being said, I personally haven’t made any substantial donations, so if you’ve made even fairly inefficient donations (as long as they were at least a little effective, and not e.g. counterproductive) then your donations have likely done more good than mine have.
I’m not claiming that all charity should be done in such an attempt-to-be-expected-utility-maximizing way, as there may be some like, component of how donations for more local causes might do more to improve the character of the person donating, and other things like that,
but I don’t see your view of “but-serendipity, and but-mosquito-nets-is-no-fun” don’t seem like compelling reasons to not look at QALYs and such.
That being said, I personally haven’t made any substantial donations, so if you’ve made even fairly inefficient donations (as long as they were at least a little effective, and not e.g. counterproductive) then your donations have likely done more good than mine have.
Several orders of magnitude more children would be saved from mosquito nets than community theater, and each of them has a chance to find a cure to cancer. Then there's a couple orders of magnitude of children who wouldn't die, but whose lives would have otherwise been significantly disrupted by malaria, hookworm or vitamin A deficiency. All these children could go on to find cures to cancer or at the very least live more meaningful lives.
Obviously, this isn't to say we should donate any less to community theaters, though I think that's a different type of donation. That is partially altruistic, but on some level it fulfills the same satisfaction as a purchase.
Obviously, this isn't to say we should donate any less to community theaters, though I think that's a different type of donation. That is partially altruistic, but on some level it fulfills the same satisfaction as a purchase.
What if closing the community theater caused them to drop art, pursue biology and then cure cancer?
> The idea that the moon is made out of cheese is stupid.
Agreed, but you are the only one that claimed this was the idea because clearly you are not the "arrogant and fundamentally wrong" one.
> The idea that the moon is made out of cheese is stupid.
Agreed, but you are the only one that claimed this was the idea because clearly you are not the "arrogant and fundamentally wrong" one.
EA does not need the assumption that you can precisely plan out an optimal path. Just that simply accounting for the good you do and comparing it to other outlets will give you better outcomes than not. Which is a far weaker premise than you can precisely plan out every optimal path.
What if when jumping off a roof, I spontaneously developed the ability to fly? And yet I will not attempt it, and worse, strongly evangelize that roofhopping should be avoided by everyone. How arrogant of me, how presumptuous!
Though we cannot tell in every case what the outcome will be, that is no excuse to not consider the odds.
Though we cannot tell in every case what the outcome will be, that is no excuse to not consider the odds.
But lots of people did jump off of things with various contraptions and we did eventually develop the ability to fly that way.
Almost like we should try to make advance predictions about outcomes given particular circumstances and continuously test them against reality :)
These people after all pursued theoretical means by which the risk of death could be reduced, and then put them into practice in order to improve their understanding.
You can die in the air. You can die on the ground! Sometimes jumping off a cliff is safer than staying there. There is no safe path, no guarantee of survival; you have to speculate, to make theories, to test them, to choose the path that seems best to you. There is nothing arrogant about this.
These people after all pursued theoretical means by which the risk of death could be reduced, and then put them into practice in order to improve their understanding.
You can die in the air. You can die on the ground! Sometimes jumping off a cliff is safer than staying there. There is no safe path, no guarantee of survival; you have to speculate, to make theories, to test them, to choose the path that seems best to you. There is nothing arrogant about this.
By all means speculate all you like about your own actions and act accordingly. The original comment was about how effective altruism is intentionally looking down on others (as they are implicitly ineffective) and holding themselves up as the ideal that everyone should follow.
Aren't you making a general point here against public arguments that certain things lead to better outcomes than others? Do you reserve the same rancor for these little warning labels printed on packs of cigarettes about cancer risk? (Look at these non-smokers looking down on others...)
Public debate is, to a first approximation, nothing but arguing about which actions lead to better outcomes.
I could see an argument that the claim is that EA is insulting because it's akin to calling a chocolate brand "asbestos-free!"; you're implicitly suggesting that competitors contain asbestos. But note that this depends on whether the other brands do, in fact, contain asbestos, and EA is not shy about its claims that many charities do not effectively pursue good outcomes. It's hard to accuse EA of trying to sneak in this point when it's a central part of its rhetoric. And of course, such debate is an entirely ordinary part of discourse.
Public debate is, to a first approximation, nothing but arguing about which actions lead to better outcomes.
I could see an argument that the claim is that EA is insulting because it's akin to calling a chocolate brand "asbestos-free!"; you're implicitly suggesting that competitors contain asbestos. But note that this depends on whether the other brands do, in fact, contain asbestos, and EA is not shy about its claims that many charities do not effectively pursue good outcomes. It's hard to accuse EA of trying to sneak in this point when it's a central part of its rhetoric. And of course, such debate is an entirely ordinary part of discourse.
I draw a distinction between a particular charitable organization being effective, i.e. the amount spent on overhead vs going to cause they claim to support and a particular charitable cause itself being effective (mosquito nets vs. community theater).
For the former I agree, seek out charities that most effectively support what you want to support.
For the latter, I don't think it's worthwhile to deride people for supporting a cause just because you think there's some other cause that might technically result in more living humans. Raw population is not a worthwhile metric to optimize for. People should be free to support things they care about and believe make the world a better place as long as it's not harming anyone.
For the former I agree, seek out charities that most effectively support what you want to support.
For the latter, I don't think it's worthwhile to deride people for supporting a cause just because you think there's some other cause that might technically result in more living humans. Raw population is not a worthwhile metric to optimize for. People should be free to support things they care about and believe make the world a better place as long as it's not harming anyone.
I still think there is a fundamental internal inconsistency here. I agree that people will, for instance, hold the moral belief that their community theater is worth more than five foreign lives. However, explicitly espousing this belief is politically unviable. The "common knowledge" moral view is that all humans are of equal moral worth. EA, as I understand it, is largely pursued by people who internalize this view; who recognize the mismatch and as a result, will prefer to sacrifice their community theater rather than admit inconsistency.
So, we should fund entertainment venues for privileged people instead of saving lives or preventing suffering?
I mean, it's not fundamentally wrong. It's better than wasting $10,000 on, e.g. luxuries for yourself that probably won't even make you happier. But it's literally orders of magnitude worse than what you could do, even if you only ever care about your fellow citizens (not people in distant poor countries) in a generation near to your own (as opposed to mankind's long-term progeny).
> It's better than wasting $10,000 on, e.g. luxuries for yourself
I'm honestly not sure. Spending your money back into the economy contributes to a system with a proven track record, part of it goes to taxes, the rest pays for jobs, creates value for the owner, who'll pay some more taxes on their income, and then spend the rest elsewhere, etc.
Donating 10k to a charity has many practical flaws, the charity itself isn't incentivized to maximize the use of the 10k, it can be corrupt, or might operate in a corrupt and inefficient environment, etc.
I'm honestly not sure. Spending your money back into the economy contributes to a system with a proven track record, part of it goes to taxes, the rest pays for jobs, creates value for the owner, who'll pay some more taxes on their income, and then spend the rest elsewhere, etc.
Donating 10k to a charity has many practical flaws, the charity itself isn't incentivized to maximize the use of the 10k, it can be corrupt, or might operate in a corrupt and inefficient environment, etc.
Let us make the example more extreme. Let's make it spending 100 billion on building a supermassive yacht, that you then proceed to sink for your own entertainment. Which is not dissimilar from 10 million people spending $10k each on luxury consumption for themselves.
Would you still argue that it still does no harm? You have just squandered 100b worth of society's resources - people's time, physical materials, engineering effort etc. for your own entertainment.
Would you still argue that it still does no harm? You have just squandered 100b worth of society's resources - people's time, physical materials, engineering effort etc. for your own entertainment.
That's a bit of an extreme made up example, which wouldn't be very representative of the common case, but I'll entertain it. Just keep in mind what's true of an extreme edge case, is not necessarily true of the common case, and you can't extrapolate like that.
Say there was a 100 billion yacht, it still served as a means to redistribute flow of funds to others. Those engineers, material extractors, workers, staff, all that contributed to people being given some of that money.
This goes towards creating an economy, and good economies can sustain countries and lift many people out of poverty, saving many lives.
Creating a demand for people's labor and resources helps with creating an economy.
This isn't money wasted, the money went to many people, it was a way to redistribute the money to others.
So I don't see the harm at all, at least not from a financial standpoint, you could argue about environmental harm though.
Now what about opportunity cost? Sure, maybe that 100 billion could have been used in ways that redistribute it more evenly, more fairly, or to people who needs the money most, or towards an endeavor that has longer term uplifting outcomes, etc, there might be a better use, though it becomes hard to figure out what use is truly better.
The way I see it, there's two dimensions:
- Redistributing money, who gets it? Split amongst how many people?
- Potential for future returns, is the money spent in a way that gives people the means to produce more?
The yacht definitely redistributes money, but maybe you disagree about the distribution of it, who got the money.
Arguably the yacht could bring future returns. Creating a demand for engineers pushed more people into engineering, now more people have the engineering know how, they go on to use that to build other projects, creating an infrastructure that others can then leverage to make other things more efficient and build on top, etc.
Basically, I just don't think it's as black and white as you claim, even in this extreme example.
Say there was a 100 billion yacht, it still served as a means to redistribute flow of funds to others. Those engineers, material extractors, workers, staff, all that contributed to people being given some of that money.
This goes towards creating an economy, and good economies can sustain countries and lift many people out of poverty, saving many lives.
Creating a demand for people's labor and resources helps with creating an economy.
This isn't money wasted, the money went to many people, it was a way to redistribute the money to others.
So I don't see the harm at all, at least not from a financial standpoint, you could argue about environmental harm though.
Now what about opportunity cost? Sure, maybe that 100 billion could have been used in ways that redistribute it more evenly, more fairly, or to people who needs the money most, or towards an endeavor that has longer term uplifting outcomes, etc, there might be a better use, though it becomes hard to figure out what use is truly better.
The way I see it, there's two dimensions:
- Redistributing money, who gets it? Split amongst how many people?
- Potential for future returns, is the money spent in a way that gives people the means to produce more?
The yacht definitely redistributes money, but maybe you disagree about the distribution of it, who got the money.
Arguably the yacht could bring future returns. Creating a demand for engineers pushed more people into engineering, now more people have the engineering know how, they go on to use that to build other projects, creating an infrastructure that others can then leverage to make other things more efficient and build on top, etc.
Basically, I just don't think it's as black and white as you claim, even in this extreme example.
The point of my example was to draw attention to what actually happens to the real resources, how they get used. They get wasted, which is a terrible outcome.
I believe your focus on money obscures that fact. Money is just a mechanism for a (good, efficient, fair?) way of using the resources available. Money is not real wealth.
What lifts people out of poverty is not money, it is ultimately useful stuff - food, shelter etc. produced so efficiently that people also are able to specialise and spend time on teaching, medicating, investing into producing not just the bare necessities but things that make them even more efficient and productive etc. In the yacht example, such scarce resources are squandered! The 'economy' created is quite literally destructive - it is geared towards consumption (i.e. destruction) of many resources - materials to build the yacht, countless peoples' time - for the mere entertainment of the person who then sinks the yacht.
Yes, the people that were involved in building the yacht now have bigger claims on a shrinking share of resources. They personally win; the person who enjoyed sinking the yacht wins; overall society greatly looses and is now much poorer.
So overall, society would be better off if that person just hoarded the money, buried it in their back yard, and preserved the scarce resources. That outcome is better than personal luxury consumption. Even better, of course, if they used it to get other people to do something productive that benefits wider society.
I believe your focus on money obscures that fact. Money is just a mechanism for a (good, efficient, fair?) way of using the resources available. Money is not real wealth.
What lifts people out of poverty is not money, it is ultimately useful stuff - food, shelter etc. produced so efficiently that people also are able to specialise and spend time on teaching, medicating, investing into producing not just the bare necessities but things that make them even more efficient and productive etc. In the yacht example, such scarce resources are squandered! The 'economy' created is quite literally destructive - it is geared towards consumption (i.e. destruction) of many resources - materials to build the yacht, countless peoples' time - for the mere entertainment of the person who then sinks the yacht.
Yes, the people that were involved in building the yacht now have bigger claims on a shrinking share of resources. They personally win; the person who enjoyed sinking the yacht wins; overall society greatly looses and is now much poorer.
So overall, society would be better off if that person just hoarded the money, buried it in their back yard, and preserved the scarce resources. That outcome is better than personal luxury consumption. Even better, of course, if they used it to get other people to do something productive that benefits wider society.
> So overall, society would be better off if that person just hoarded the money, buried it in their back yard, and preserved the scarce resources. That outcome is better than personal luxury consumption. Even better, of course, if they used it to get other people to do something productive that benefits wider society
Ok, so your argument is in the actual materials consumed? The water, metal, and all that being "wasted" for a yacht, as opposed to used for life critical purposes instead?
So, that's kind of the environmental aspect I mentioned, I'm not opposed here, sustainable resource use is important.
But donating to a charity can similarly be a temporary waste of resources with no sustainability improvements made.
Just feeding people food for an extra week for example, is very different from investing in sustainable farming methods that can guarantee us renewable and long lasting sources of food.
And on that topic, it's surprising how in practice discoveries that can help us with sustainability and production methods are discovered through various other ventures, often military for example, but it can happen anywhere.
The person who invented ammonia was researching chemical warfare, now it's used for fertilizer and dramatically helped boost our food production efficiency.
These second order effects I think are hard to predict, similarly hard to predict what the workers that got money from building the yacht than go on to use that money for themselves, maybe they spend it investing in food production efficiencies.
Ok, so your argument is in the actual materials consumed? The water, metal, and all that being "wasted" for a yacht, as opposed to used for life critical purposes instead?
So, that's kind of the environmental aspect I mentioned, I'm not opposed here, sustainable resource use is important.
But donating to a charity can similarly be a temporary waste of resources with no sustainability improvements made.
Just feeding people food for an extra week for example, is very different from investing in sustainable farming methods that can guarantee us renewable and long lasting sources of food.
And on that topic, it's surprising how in practice discoveries that can help us with sustainability and production methods are discovered through various other ventures, often military for example, but it can happen anywhere.
The person who invented ammonia was researching chemical warfare, now it's used for fertilizer and dramatically helped boost our food production efficiency.
These second order effects I think are hard to predict, similarly hard to predict what the workers that got money from building the yacht than go on to use that money for themselves, maybe they spend it investing in food production efficiencies.
I thought the idea was that, provided you want to achieve X in the world, it's likely suboptimal.
Not wrong or bad as people in this thread seem to universally interpret it, but if you want to optimize for animal welfare, you might be better off doing something about farm animals rather than donating to a local shelter. If that turns out to be too far removed from your personal situation to care about, okay then apparently that's not your priority and that is fine. But between the things you spend effort or money on, it's worth looking into what is the most effective means to your end.
I am not involved in this whole thing in any way but that's what I thought the underlying idea was. Not an explicit dig at those who already try to do good, not at all... (Edit: realization: and that method of making people feel bad wouldn't be very altruistic nor effective right?)
Not wrong or bad as people in this thread seem to universally interpret it, but if you want to optimize for animal welfare, you might be better off doing something about farm animals rather than donating to a local shelter. If that turns out to be too far removed from your personal situation to care about, okay then apparently that's not your priority and that is fine. But between the things you spend effort or money on, it's worth looking into what is the most effective means to your end.
I am not involved in this whole thing in any way but that's what I thought the underlying idea was. Not an explicit dig at those who already try to do good, not at all... (Edit: realization: and that method of making people feel bad wouldn't be very altruistic nor effective right?)
[deleted]
that's a good way of putting it. I kind of ignored a lot of what they said but again, just like the rationalists who pretty much were all of the EA people, they came with biases they refused to admit they had.
like for example, if a family member of mine needed money for a cancer treatment that had a 50% chance of working versus giving that money to save the life of someone I don't know on the other side of the planet, I will consistently try to save my family member. That is how most people would operate. I will do this because I don't know the person on the other side of the planet. If it were not for the internet and their subreddit/lesswrong style of shaming campaigns, no one would ever even know there was a person on the other side of the planet who also needed the money. Why should I be asked to ignore my own family by annoying redditors who don't give a shit about me anyways?
I doubt even the EA people would do that unless they could virtue signal about how smart they are and how stupid everyone else is online. Most of the time, these people just speculated about how they might spend their money if they were in such a scenario. It was like a hobby.
Although in fairness rating charities by their effectiveness is a good idea. Yes, people do blindly give cash away to corrupt charities.
like for example, if a family member of mine needed money for a cancer treatment that had a 50% chance of working versus giving that money to save the life of someone I don't know on the other side of the planet, I will consistently try to save my family member. That is how most people would operate. I will do this because I don't know the person on the other side of the planet. If it were not for the internet and their subreddit/lesswrong style of shaming campaigns, no one would ever even know there was a person on the other side of the planet who also needed the money. Why should I be asked to ignore my own family by annoying redditors who don't give a shit about me anyways?
I doubt even the EA people would do that unless they could virtue signal about how smart they are and how stupid everyone else is online. Most of the time, these people just speculated about how they might spend their money if they were in such a scenario. It was like a hobby.
Although in fairness rating charities by their effectiveness is a good idea. Yes, people do blindly give cash away to corrupt charities.
This take on EA notion replaces it with a straw man.
Essentially EA tells not "let's save as many lives as possible" but "let's improve the world according to our values as much as possible". And values might differ among the people.
In your example with cure for relative working with only X probability Vs definitely curing stranger on the other side of the world there should be value X for which you will prefer to cure the stranger. Whether it 10% , 1% or maybe 0.01% I will not blame you. But it worth to know your values like value of your family member life over the stranger's life to act accordingly.
Dive into EA activities starts with exploring your values. How do you value lives of animals in comparison to lives of people? Do you prefer improving people well being now or reducing existential risks like wars and epidemies? EA literally made a tool to estimate one's values and promote this tool on the front page.
Essentially EA tells not "let's save as many lives as possible" but "let's improve the world according to our values as much as possible". And values might differ among the people.
In your example with cure for relative working with only X probability Vs definitely curing stranger on the other side of the world there should be value X for which you will prefer to cure the stranger. Whether it 10% , 1% or maybe 0.01% I will not blame you. But it worth to know your values like value of your family member life over the stranger's life to act accordingly.
Dive into EA activities starts with exploring your values. How do you value lives of animals in comparison to lives of people? Do you prefer improving people well being now or reducing existential risks like wars and epidemies? EA literally made a tool to estimate one's values and promote this tool on the front page.
Thanks, that might be true I have not seen in it.
I took this passage from effectivealtruism.org but I was mostly referring to the EA community online which seemed to mostly be the rationalists like the OP said. It's worth pointing out that Eliezer Yudkowsky was mentioned ITT.
Impartial altruism: We believe that all people count equally. Of course it's reasonable to have special concern for one's own family, friends and life. But, when trying to do as much good as possible, we aim to give everyone's interests equal weight, no matter where or when they live. Thi s means focusing on the groups who are most neglected, which usually means focusing on those who don’t have as much power to protect their own interests.
So this is not saying all EA people think the same thing or have the same values or are on that site but I do think this is a common viewpoint among them. I just question why that should be and whether or not a lot of EA people are even altruistic or just virtue signalling on the internet.
I toned down some of the language in this post. I am not being meticulous I am just freestyling my thoughts because it's fun and there is a dose of humor I am putting in here but usually it's not appreciated on HN. Anyways, I don't know if there is a counter criticism to EA officially by some philosopher, and I actually agree with some of what they say because I remember a lot of discussion about choosing charities wisely and not wasting charitable resources. So I don't want to write a formal argument but nevertheless I think what I wrote is a summary of why I am skeptical.
I took this passage from effectivealtruism.org but I was mostly referring to the EA community online which seemed to mostly be the rationalists like the OP said. It's worth pointing out that Eliezer Yudkowsky was mentioned ITT.
Impartial altruism: We believe that all people count equally. Of course it's reasonable to have special concern for one's own family, friends and life. But, when trying to do as much good as possible, we aim to give everyone's interests equal weight, no matter where or when they live. Thi s means focusing on the groups who are most neglected, which usually means focusing on those who don’t have as much power to protect their own interests.
So this is not saying all EA people think the same thing or have the same values or are on that site but I do think this is a common viewpoint among them. I just question why that should be and whether or not a lot of EA people are even altruistic or just virtue signalling on the internet.
I toned down some of the language in this post. I am not being meticulous I am just freestyling my thoughts because it's fun and there is a dose of humor I am putting in here but usually it's not appreciated on HN. Anyways, I don't know if there is a counter criticism to EA officially by some philosopher, and I actually agree with some of what they say because I remember a lot of discussion about choosing charities wisely and not wasting charitable resources. So I don't want to write a formal argument but nevertheless I think what I wrote is a summary of why I am skeptical.
Ok but now I’m kind of lost. I genuinely don’t know anything about EA, so this is an honest question.
If EA is improving the world according to your values, is it really distinct as a concept? Like, to me, the reason why I think almost anything is because it improves the world according to my values.
Another way to put it is that to me it kind of sounds like someone is saying, “we believe in doing the best thing”, but I don’t want to put words in your mouth. I also hope everyone wants to do the best thing ;)
If EA is improving the world according to your values, is it really distinct as a concept? Like, to me, the reason why I think almost anything is because it improves the world according to my values.
Another way to put it is that to me it kind of sounds like someone is saying, “we believe in doing the best thing”, but I don’t want to put words in your mouth. I also hope everyone wants to do the best thing ;)
Effective altruism is about the _effective_ part. Don't just donate to the charity or cause that _sounds_ good; look for where your money would provide the biggest impact.
If you want to save lives in general, look for the most _effective_ charity like that, which may be something like mosquito nets rather than a cancer research foundation. Don't just donate money to whatever is the most obvious charity advertised on television.
If you had a family member with cancer, look at which of N cancer foundations spend the money most effectively and donate there.
Often, altruism comes down to people giving money to the 'obvious' causes, which don't necessarily use the money effectively.
If you want to save lives in general, look for the most _effective_ charity like that, which may be something like mosquito nets rather than a cancer research foundation. Don't just donate money to whatever is the most obvious charity advertised on television.
If you had a family member with cancer, look at which of N cancer foundations spend the money most effectively and donate there.
Often, altruism comes down to people giving money to the 'obvious' causes, which don't necessarily use the money effectively.
cancer research will give benefits forever once a solution is found whereas mosquito nets will get worn out and so are only a temporary fix. Cancer cure also scales to more people eventually. Worse, how much is your donation contributing to cancer research? You just can't quantify that.
The whole concept just seems to assume you know everything, present and future, and have infinite calculation power.
The whole concept just seems to assume you know everything, present and future, and have infinite calculation power.
First a disclaimer: I am not a part of EA community, but I have met several people from this community. And I am loosely connected to rationalists community. I find EA concept appealing and I think I should allocate some of my resources to help this movement.
Indeed my wording was too broad and over-compassing. Effective altruism was kick started from comparison of different organizations collecting money for whatever altruistic means. And comparison showed that these initiatives are vastly different in effectiveness which prompted an idea that we should take into account effectiveness of particular initiative before giving it money.
EA seems to focus on reducing suffering over improving happiness. Giving a hand to a stranger who have stumbled and have fallen to the ground near you is out of the scope: EA is about strategic long lasting calculated stuff.
Indeed my wording was too broad and over-compassing. Effective altruism was kick started from comparison of different organizations collecting money for whatever altruistic means. And comparison showed that these initiatives are vastly different in effectiveness which prompted an idea that we should take into account effectiveness of particular initiative before giving it money.
EA seems to focus on reducing suffering over improving happiness. Giving a hand to a stranger who have stumbled and have fallen to the ground near you is out of the scope: EA is about strategic long lasting calculated stuff.
"Doing good better" was the book title, and I think better defines the goal well. Not perfect, better.
The choices that people make with giving often have no framework for deciding good, let alone better. The moral question is not is X better than Y, but what of all the choices I am going to make can I make better?
If everyone made better choices, not perfect just better, there is a LOT we could improve.
The choices that people make with giving often have no framework for deciding good, let alone better. The moral question is not is X better than Y, but what of all the choices I am going to make can I make better?
If everyone made better choices, not perfect just better, there is a LOT we could improve.
> like for example, if a family member of mine needed money for a cancer treatment that had a 50% chance of working versus giving that money to save the life of someone I don't know on the other side of the planet, I will consistently try to save my family member.
So would most people, and we wouldn’t call it altruism. Looking for an objective criterion (whether ‘number of lives saved per dollar’ or something else) is an angle that I’m happy someone is taking. If you don’t like their approach then don’t give them money.
So would most people, and we wouldn’t call it altruism. Looking for an objective criterion (whether ‘number of lives saved per dollar’ or something else) is an angle that I’m happy someone is taking. If you don’t like their approach then don’t give them money.
> So would most people, and we wouldn’t call it altruism.
Most people would call helping a relative pay their medical bills altruism. It's also the selfish and rational choice. It's both simultaneously.
Most people would call helping a relative pay their medical bills altruism. It's also the selfish and rational choice. It's both simultaneously.
> Most people would call helping a relative pay their medical bills altruism.
Altruism is the disinterested and selfless concern for the well-being of others. Helping your family is not altruism.
Altruism is the disinterested and selfless concern for the well-being of others. Helping your family is not altruism.
> Helping your family is not altruism.
Not necessarily altruism.
It absolutely could be.
Knowing the true motives of another human is very difficult, perhaps impossible (without a predictive biological model of consciousness we can't be sure we're reading minds correctly, and it's still an open question whether such a thing can exist).
Not necessarily altruism.
It absolutely could be.
Knowing the true motives of another human is very difficult, perhaps impossible (without a predictive biological model of consciousness we can't be sure we're reading minds correctly, and it's still an open question whether such a thing can exist).
> like for example, if a family member of mine needed money for a cancer treatment that had a 50% chance of working versus giving that money to save the life of someone I don't know on the other side of the planet, I will consistently try to save my family member. That is how most people would operate.
I just want to point out that:
1. Just because this is how most people operate does not mean this is the ethical choice. Slavery was normal at one time remember.
2. You say EA people have biases they refuse to admit to, but your argument is showing that you are exhibiting the bias, namely the bias towards preferring your family member over general human life. Arguably a position that does not prefer one human life over another is less biased, no?
I just want to point out that:
1. Just because this is how most people operate does not mean this is the ethical choice. Slavery was normal at one time remember.
2. You say EA people have biases they refuse to admit to, but your argument is showing that you are exhibiting the bias, namely the bias towards preferring your family member over general human life. Arguably a position that does not prefer one human life over another is less biased, no?
yes I am biased if bias means not weighting all people equally on moral grounds. I am not sure why I should. If someone did that I think it's interesting what conclusions they might draw about how to spend their money or what job they might choose. It destroys much of the point of having a family. You would have to weight random strangers with equal status to your own children. So the needs of random strangers would almost always outweigh the needs of your children assuming they are healthy and you live comfortably. It would not surprise me if EA people thought families were immoral though.
I don't think assuming an equal weighting is being unbiased though. It could be a result of a political bias. As I said I don't know why people should assume this. The digital era of social media and digital banking enable this viewpoint so I think that is another angle to discuss regarding how to weigh these issues.
But the specific bias I was accusing some in the EA community of having is virtue signalling as having superior ethics, morals, and intelligence. I think that is a real bias. I don't want to accuse everyone in the community of not being altruistic but claiming to be an effective altruist could certainly be a career enhancer if you are an ethicist. Also, my experience with this community is some of them come off as misanthropes. A misanthrope might be motivated to argue elaborate contrarian viewpoints in order to cause harm or annoy people.
I don't think assuming an equal weighting is being unbiased though. It could be a result of a political bias. As I said I don't know why people should assume this. The digital era of social media and digital banking enable this viewpoint so I think that is another angle to discuss regarding how to weigh these issues.
But the specific bias I was accusing some in the EA community of having is virtue signalling as having superior ethics, morals, and intelligence. I think that is a real bias. I don't want to accuse everyone in the community of not being altruistic but claiming to be an effective altruist could certainly be a career enhancer if you are an ethicist. Also, my experience with this community is some of them come off as misanthropes. A misanthrope might be motivated to argue elaborate contrarian viewpoints in order to cause harm or annoy people.
> If someone did that I think it's interesting what conclusions they might draw about how to spend their money or what job they might choose. It destroys much of the point of having a family.
I'm not sure they would. They might adopt instead of having their own kids, for instance.
> So the needs of random strangers would almost always outweigh the needs of your children assuming they are healthy and you live comfortably.
Not necessarily. Divide and conquer is one of our most effective problem solving strategies, and family units fall under this umbrella of dividing the work of raising the next generation.
> But the specific bias I was accusing some in the EA community of having is virtue signalling as having superior ethics, morals, and intelligence. I think that is a real bias.
It could be. It's also possible that they are more ethical and intelligent though. Not in the sense that becoming an EA makes you more intelligent, but in the sense that maybe EA attracts more intelligent and ethical people. This self-sorting effect would diminish as EA spreads in popularity of course.
Ultimately, whether they are more ethical or intelligent is an empirical claim that you can validate, and while it's reasonable to think it's implausible for it to be true, that's not a proof that it isn't true.
I'm not sure they would. They might adopt instead of having their own kids, for instance.
> So the needs of random strangers would almost always outweigh the needs of your children assuming they are healthy and you live comfortably.
Not necessarily. Divide and conquer is one of our most effective problem solving strategies, and family units fall under this umbrella of dividing the work of raising the next generation.
> But the specific bias I was accusing some in the EA community of having is virtue signalling as having superior ethics, morals, and intelligence. I think that is a real bias.
It could be. It's also possible that they are more ethical and intelligent though. Not in the sense that becoming an EA makes you more intelligent, but in the sense that maybe EA attracts more intelligent and ethical people. This self-sorting effect would diminish as EA spreads in popularity of course.
Ultimately, whether they are more ethical or intelligent is an empirical claim that you can validate, and while it's reasonable to think it's implausible for it to be true, that's not a proof that it isn't true.
> I doubt even the EA people would do that unless they could virtue signal about how smart they are and how stupid everyone else is online. Most of the time, these people just speculated about how they might spend their money if they were in such a scenario. It was like a hobby
I think it's clear you don't know any EA people in real life.
I think it's clear you don't know any EA people in real life.
> I will consistently try to save my family member
Sure, but that's because we all prefer living in a society where everyone abides by their obligations to the people closest to them - nothing wrong with this. That's special pleading, though.
Sure, but that's because we all prefer living in a society where everyone abides by their obligations to the people closest to them - nothing wrong with this. That's special pleading, though.
Is helping your family get through cancer considered altruistic?
> Why should I be asked to ignore my own family by annoying redditors who don't give a shit about me anyways?
You shouldn't; doing so would be irrational. There is a fairly decent chance that your relative, if they survive, will be in a position to help you in the future. But the random person on the other side of the planet? The chance of them randomly helping you in the future is about one in a few billion. Particularly if they don't know who their benefactor was, so they couldn't deliberately repay the favor to you.
So the rational/selfish position is to always put family and friends first, before strangers.
You shouldn't; doing so would be irrational. There is a fairly decent chance that your relative, if they survive, will be in a position to help you in the future. But the random person on the other side of the planet? The chance of them randomly helping you in the future is about one in a few billion. Particularly if they don't know who their benefactor was, so they couldn't deliberately repay the favor to you.
So the rational/selfish position is to always put family and friends first, before strangers.
That’s fine… but that’s _not the reason_, right? The reason is that you know the person and don’t want them to suffer. Trying to ignore that lived reality in a quest to be objective or rational seems to me like a recipe for mistaken thinking.
It's not the reason, but it is a reason. The reason (for me) would be the principle of family coming first. But as is often the case, you can reconstruct deontological principles within a utilitarian framework.
For instance; if everybody sticks to the principle of murder being wrong despite the consequences, then there will be fewer misguided mass murderers, and this is better for society in the long run than everybody going around committing murders they believe to be justified by some end.
For instance; if everybody sticks to the principle of murder being wrong despite the consequences, then there will be fewer misguided mass murderers, and this is better for society in the long run than everybody going around committing murders they believe to be justified by some end.
I don't know about "always", but yes, there's a strong rational element behind favouring friends and especially family, and in general others that you're more likely to interact with in the future. For one thing, I'd think confidence that you have family members that will ensure your children will thrive even after you're gone gives peace of mind for most parents. And even if I never actually need help from family or friends of the nature I give them now, the regular act of providing assistance is an important part of what maintains good relationships, which we all know are pretty vital to mental health etc.
I would say though that giving money is probably not generally the best way to achieve that end, and EA are primarily (solely?) about monetary donations.
Honestly I don't think there's any realistic way to decide in general whether giving assistance to person A vs person B is more "rational" or even more "effective", but it's good to at least be aware that some charities/NGOs/etc. are at least trustworthy and ensure a high % of donations do actively go towards providing help, rather being swallowed in marketing and admin budgets.
Honestly I don't think there's any realistic way to decide in general whether giving assistance to person A vs person B is more "rational" or even more "effective", but it's good to at least be aware that some charities/NGOs/etc. are at least trustworthy and ensure a high % of donations do actively go towards providing help, rather being swallowed in marketing and admin budgets.
In defense of marketing and admin budgets. What would you prefer, a nonprofit where $100 of your donation goes to "providing help" (I don't really know what that means, since money alone can't do anything) or a nonprofit where $30 of your $100 is invested in "marketing and admin" which results in:
- the charity has $500
- there are admin humans around to "provide help" (and you're paying them a decent or even above-average salary because you want them to do an amazing job, right?)
What if they took $60 out of your $100 and turned it into $5,000 (not unheard of if they invest in digital ads to a warm list, for example)
It's counterproductive to insist that nonprofits operate without operations budgets.
It's counterproductive to insist that nonprofits operate without operations budgets.
Yes, I think this is reasonable. I would also like to take a moment to reiterate why so many people are bitter about the marketing / admin angle:
I pay them money. Charity pays consultants to turn that into more money (plus admins to manage the consultants). Consultants determine that a fraction of that money should be used to solicit more donations. Research indicates best source of new donations is in fact existing donors.
At this point charity is spending money I gave them to extract further donations from me. The ice-cream cone is now licking itself :/
I pay them money. Charity pays consultants to turn that into more money (plus admins to manage the consultants). Consultants determine that a fraction of that money should be used to solicit more donations. Research indicates best source of new donations is in fact existing donors.
At this point charity is spending money I gave them to extract further donations from me. The ice-cream cone is now licking itself :/
I wouldn't consider 30% unreasonable or an example of money being swallowed - but there are known instances of non-profits where over 60% of donations go towards covering such costs (the Law Enforcement Legal Defense fund supposedly is one of the worst offenders).
>most altruism is not effective. Like oh your altruism is not that productive, but my altruism, well it's effective altruism.
I unironically think this. In Hungary where I live, if you don't care about these things much but still want to contribute somewhat to society by charity, the first lines of contact will mostly be crooks that feed on goodwill. There will be beggars, organized into groups, charities that don't exist or have such an overhead that only a single percent goes to the actual cause, corporations that take your donation and then boast with the amount as if they were the ones giving it, and charities that are popular but are actually fronts for ideologies, or churches, and frauds like people collecting clothes for the poor, only to sell them abroad. These entities grind up a large amount of goodwill, while producing little or negative value for mankind. So yes - the altruism of the people donating to entities like this does fuck-all, while I imagine that if someone researches a bit, they will be more likely to give to entities that contribute more.
With that being said, I don't like the EA people either. But the concept is valid.
I unironically think this. In Hungary where I live, if you don't care about these things much but still want to contribute somewhat to society by charity, the first lines of contact will mostly be crooks that feed on goodwill. There will be beggars, organized into groups, charities that don't exist or have such an overhead that only a single percent goes to the actual cause, corporations that take your donation and then boast with the amount as if they were the ones giving it, and charities that are popular but are actually fronts for ideologies, or churches, and frauds like people collecting clothes for the poor, only to sell them abroad. These entities grind up a large amount of goodwill, while producing little or negative value for mankind. So yes - the altruism of the people donating to entities like this does fuck-all, while I imagine that if someone researches a bit, they will be more likely to give to entities that contribute more.
With that being said, I don't like the EA people either. But the concept is valid.
[deleted]
> I don't like the EA people either.
Do you actually know any? The ones I know in real life are wonderful people. Intelligent, caring and hard working.
Do you actually know any? The ones I know in real life are wonderful people. Intelligent, caring and hard working.
I'm a bit ashamed because of the wording, I don't actually know any, not even online. I only have experience with the expression of the ideas, gathered from various websites.
Is there a term you would deem acceptable for seeking to give to others in a cost-effective manner?
This seems to be a perfectly good idea. Doesn't it seem problematic that giving it a simple, straightforward name seems inherently arrogant and insufferable?
This seems to be a perfectly good idea. Doesn't it seem problematic that giving it a simple, straightforward name seems inherently arrogant and insufferable?
The article mentions its philosophical underpinnings: utilitarianism. There’s a wealth of philosophical discussion of its own merits and problems. Reading the article but having been unfamiliar with the “effective altruism” term, I immediately recognized utilitarianism in it and the rest was predictable. It seemed a perfectly good idea to me too for some time, and still does with a slew of caveats.
Giving it a simple name is fine, but it has one. Giving it a manipulative name is, well, one of the potential philosophical problems with utilitarianism.
Giving it a simple name is fine, but it has one. Giving it a manipulative name is, well, one of the potential philosophical problems with utilitarianism.
What name is manipulative?
I assume the GP is talking about the name "effective altruism".
I don't know what else they should have picked? It seems like a fairly straightforward name for movement that focuses on charity effectiveness. I guess you could call it "effective giving" or something like that but caring about effectiveness is it core differentiating tenant and anything related to altruism/giving/charity is going to sound good.
I think the argument is that everyone else doing charity/altruism also (ostensibly) cares about effectiveness so it’s only a differentiating tenant if you somehow think your caring about effectiveness is somehow different to everyone else’s.
They care about effectiveness in the sense of if you asked if they cared about effectiveness they'd say yes.
They don't care about effectiveness in the sense that they are doing the math on how effective different outreach is, comparing that against other uses of the money, and sharing that with their donors.
I've been to 9-10 charity events and none of them talked about effectiveness in a dollar and cents way like the way givewell does. It's a real change from the status quo.
They don't care about effectiveness in the sense that they are doing the math on how effective different outreach is, comparing that against other uses of the money, and sharing that with their donors.
I've been to 9-10 charity events and none of them talked about effectiveness in a dollar and cents way like the way givewell does. It's a real change from the status quo.
Like someone else said and articles have said. Socialism is the most effective altruism. This doesn’t allow you to become worth $100M+ with the reasoning being you’ll be an effective altruist though. The upper EA people are suspect. As well as the thinking that making more money is better then a great society helping job. Not that most EA people are like this.
> Is there a term you would deem acceptable for seeking to give to others in a cost-effective manner?
Not the OP but State-imposed socialism might do the trick. Free housing, free decent education, free healthcare, what's there not to like?
The only thing is that you wouldn't be the one "doing" the giving as an individual, because that's not your job to do (again, as an individual).
Not the OP but State-imposed socialism might do the trick. Free housing, free decent education, free healthcare, what's there not to like?
The only thing is that you wouldn't be the one "doing" the giving as an individual, because that's not your job to do (again, as an individual).
Your reaction is a bit absolutist. The basic premise is that if you're capable of making 500k a year in the developed world then going to Africa to volunteer to build houses is not as effective as just donating your money to charities that send people to help, or hire people yourself.
What it evolved into is a bit controversial, overall they seem to be a bunch of kids who don't know what to do with the power and money they had at one point, as they all came too easy.
What it evolved into is a bit controversial, overall they seem to be a bunch of kids who don't know what to do with the power and money they had at one point, as they all came too easy.
[deleted]
> Maybe this is superficial, but I was put off by the name "effective altruism". By naming it such, it felt like an implicit dig that most altruism is not effective.
As an outsider, I feel the same way about "evidence based medicine". It'll be interesting to see, in 40 years, whether or not either moniker was earned.
As an outsider, I feel the same way about "evidence based medicine". It'll be interesting to see, in 40 years, whether or not either moniker was earned.
> By naming it such, it felt like an implicit dig that most altruism is not effective. Like oh your altruism is not that productive, but my altruism, well it's effective altruism.
But ... that's where effective altruism comes from. Have you not noticed the eroding trust in charities (https://www.theguardian.com/voluntary-sector-network/2015/se...) over the past decade? Effective altruism is just a counter-movement against wasteful charities and engenders a growing realization that not all donations are good and even that donations might not always be the best solution. It wasn't about lecturing anybody, it was about (for example) figuring out which charities are most effective and least wasteful with the money that they receive.
But ... that's where effective altruism comes from. Have you not noticed the eroding trust in charities (https://www.theguardian.com/voluntary-sector-network/2015/se...) over the past decade? Effective altruism is just a counter-movement against wasteful charities and engenders a growing realization that not all donations are good and even that donations might not always be the best solution. It wasn't about lecturing anybody, it was about (for example) figuring out which charities are most effective and least wasteful with the money that they receive.
> By naming it such, it felt like an implicit dig that most altruism is not effective. Like oh your altruism is not that productive, but my altruism, well it's effective altruism.
Pretty much. A subset of altruism is effective, a subset is ineffective. And there's another term for a third subset that overlaps with both: virtue signaling. Usually people treat the term like aligns closer to ineffective altruism though, like "oh that's just virtue signaling", even though it can be both effective and virtue signaling.
Pretty much. A subset of altruism is effective, a subset is ineffective. And there's another term for a third subset that overlaps with both: virtue signaling. Usually people treat the term like aligns closer to ineffective altruism though, like "oh that's just virtue signaling", even though it can be both effective and virtue signaling.
I feel like real effective altruism is achieved by sites like charity navigator, which give you a sense of how well your donations to a given cause are spent by the organization.
This is not really true. Charity Navigator incentivize that charities operate on a shoestring which is not very effective: https://www.ted.com/talks/dan_pallotta_the_way_we_think_abou...
I haven't watched the video (yet), but for example wikipedia has a stellar near-perfect rating, but still devote 15%/$12M of their budget to salaries which surely could be cut drastically if they wanted to really be on a shoestring.
The founders of the Effective Altruism movement are mostly non-technical people. You can't blame their arrogance and failures on the "techbros".
The people mentioned on the Effective Altruism Wikipedia page:
Peter Singer: Professor of Bioethics (philosopher)
Toby Ord: Philosopher. Philosophy PhD, Computer Science bachelors
Hilary Greaves: Philosopher, and he double-majored in physics in undergrad
William MacAskill: Philosopher/Ethicist. PhD in philosophy.
Eliezer Yudkowsky: self-described autodidact, no formal education or notable work history. Runs a "AI" "Research" center called MIRI.
Holden Karnofsky: Co-Founder of GiveWell, Harvard degree in Social Studies
Yew-Kwang Ng: Welfare Economist, has published biology, mathematics, philosophy, cosmology, psychology, and sociology papers.
Derek Parfit: Philosopher. Has a history degree.
Nick Bostrom: Philosopher. BA in philosophy and math, MA in philosophy and physics.
Benjamin Todd: Physics/Philosophy double major, worked at Kodak for eight months and at an investment firm for 1.25 years before joining the Centre for Effective Altruism.
The financial backers are more diverse:
Dustin Moskovitz: Co-Founder of Facebook, Co-Founder of Open Philanthropy
Cari Tuna: Dustin's wife, previously worked as a reporter before cofounding Open Philanthropy
Dan Smith: Professional poker player
Liv Boeree: Professional poker player. Physics degree
Sam Bankman-Fried: con-artist, thief, fraudster. Has a physics degree and a minor in math, interned/worked as a trader at Jane Street, and founded Alameda Capital and FTX.
The people mentioned on the Effective Altruism Wikipedia page:
Peter Singer: Professor of Bioethics (philosopher)
Toby Ord: Philosopher. Philosophy PhD, Computer Science bachelors
Hilary Greaves: Philosopher, and he double-majored in physics in undergrad
William MacAskill: Philosopher/Ethicist. PhD in philosophy.
Eliezer Yudkowsky: self-described autodidact, no formal education or notable work history. Runs a "AI" "Research" center called MIRI.
Holden Karnofsky: Co-Founder of GiveWell, Harvard degree in Social Studies
Yew-Kwang Ng: Welfare Economist, has published biology, mathematics, philosophy, cosmology, psychology, and sociology papers.
Derek Parfit: Philosopher. Has a history degree.
Nick Bostrom: Philosopher. BA in philosophy and math, MA in philosophy and physics.
Benjamin Todd: Physics/Philosophy double major, worked at Kodak for eight months and at an investment firm for 1.25 years before joining the Centre for Effective Altruism.
The financial backers are more diverse:
Dustin Moskovitz: Co-Founder of Facebook, Co-Founder of Open Philanthropy
Cari Tuna: Dustin's wife, previously worked as a reporter before cofounding Open Philanthropy
Dan Smith: Professional poker player
Liv Boeree: Professional poker player. Physics degree
Sam Bankman-Fried: con-artist, thief, fraudster. Has a physics degree and a minor in math, interned/worked as a trader at Jane Street, and founded Alameda Capital and FTX.
"TechBros" doesn't mean "programmers". It broadly means something like "people who think tech will solve everything but don't think very deeply"
Nah, it's an insult meant to lower the assumed position/opinion of whoever it was aimed at. Maybe in the past it meant what you describe it as, but I haven't seen that in any recent use.
It's more or less a term one would use to signal to others to dogpile the person and get them to leave the conversation/topic, mostly by being absurdly obtuse and misconstruing their words to it's most literal meaning with nothing left to deem charitable if engaging, or just speaking to other people entirely and not towards the subject whatsoever. Kind of like how most insults devolve into in pretty much all social media, just another word to pull out of the ad-hom bag.
It's more or less a term one would use to signal to others to dogpile the person and get them to leave the conversation/topic, mostly by being absurdly obtuse and misconstruing their words to it's most literal meaning with nothing left to deem charitable if engaging, or just speaking to other people entirely and not towards the subject whatsoever. Kind of like how most insults devolve into in pretty much all social media, just another word to pull out of the ad-hom bag.
And yet Effective Altruism in itself is not technology focused. There's no evidence that list of people (many philosophers) think tech will solve everything.
Longtermism explicitly is technology focused though:
> Many longtermists reckon that the antidote to the threat from technology is more technology; mastering artificial intelligence will forestall a malevolent AI from enslaving us. They also believe that technological acceleration is morally good in its own right, as it enables the universe to sustain more people. Bostrom laments the lives lost every second that technological advances are delayed. In a paper from 2003 he invites readers to imagine all the “unused energy…being flushed down black holes”, and all the suns beyond our own that are “illuminating and heating empty rooms”, because we lack the means to populate the planets orbiting them
And longtermism has taken over much of the EA movement:
> Disillusioned effective altruists are dismayed by the increasing predominance of “strong longtermism” ..[snip].. According to Benjamin Todd, a founder of 80,000 Hours, longtermism “might well turn out to be one of the most important discoveries of effective altruism so far” ..[snip].. Projects devoted to global health and poverty still garner the most funding, but their share of the total allocation is shrinking as long-term-risk research attracts more cash.
Both quotes from the linked article.
> Many longtermists reckon that the antidote to the threat from technology is more technology; mastering artificial intelligence will forestall a malevolent AI from enslaving us. They also believe that technological acceleration is morally good in its own right, as it enables the universe to sustain more people. Bostrom laments the lives lost every second that technological advances are delayed. In a paper from 2003 he invites readers to imagine all the “unused energy…being flushed down black holes”, and all the suns beyond our own that are “illuminating and heating empty rooms”, because we lack the means to populate the planets orbiting them
And longtermism has taken over much of the EA movement:
> Disillusioned effective altruists are dismayed by the increasing predominance of “strong longtermism” ..[snip].. According to Benjamin Todd, a founder of 80,000 Hours, longtermism “might well turn out to be one of the most important discoveries of effective altruism so far” ..[snip].. Projects devoted to global health and poverty still garner the most funding, but their share of the total allocation is shrinking as long-term-risk research attracts more cash.
Both quotes from the linked article.
> And longtermism has taken over much of the EA movement
Show me the money. Sci-Fi/Fantasy longtermst nerd concerns get a disproportionate amount of attention. But from what I can tell, most donations still go to charities dealing with malaria, deficiencies and vaccines: https://www.openphilanthropy.org/research/2021-allocation-to...
Sure, some people are getting paid thousands of dollars for navel gazing. But more than a billion dollars was donated to directly improve global health.
Show me the money. Sci-Fi/Fantasy longtermst nerd concerns get a disproportionate amount of attention. But from what I can tell, most donations still go to charities dealing with malaria, deficiencies and vaccines: https://www.openphilanthropy.org/research/2021-allocation-to...
Sure, some people are getting paid thousands of dollars for navel gazing. But more than a billion dollars was donated to directly improve global health.
It's great that over $1B was donated now for global health!
But the highest ever grant was $55M for "potential risks from AI". The 4th and 6th highest grants were similar.[1]
More recently[2] the highest ($10.7M) grant was for "Potential Risks from Advanced AI" and third highest grant ($2.3M) was for "longtermism" and there were 4 other similar grants on the first page.
[1] https://www.openphilanthropy.org/grants/?sort=high-to-low#ca...
[2] https://www.openphilanthropy.org/grants/?sort=recent#categor...
But the highest ever grant was $55M for "potential risks from AI". The 4th and 6th highest grants were similar.[1]
More recently[2] the highest ($10.7M) grant was for "Potential Risks from Advanced AI" and third highest grant ($2.3M) was for "longtermism" and there were 4 other similar grants on the first page.
[1] https://www.openphilanthropy.org/grants/?sort=high-to-low#ca...
[2] https://www.openphilanthropy.org/grants/?sort=recent#categor...
potential risks of AI sounds the opposite of the position of someone who would think technology can solve everything. You're shifting goalposts to protect the initial narrative.
> potential risks of AI sounds the opposite of the position of someone who would think technology can solve everything.
Yes, that's a logical conclusion, but it's not right because of the weird "longtermist" logic. Their "AI risk research" is about AI alignment, which is a "longtermist" priority.
To repost my previous quote:
> Many longtermists reckon that the antidote to the threat from technology is more technology; mastering artificial intelligence will forestall a malevolent AI from enslaving us. They also believe that technological acceleration is morally good in its own right.
This is the "acceleratist view" of AI research, of which Nick Bostrom is a prominent proponent: basically he views progress as inevitable, so it's best to accelerate AI that is aligned with human values. See https://nickbostrom.com/papers/openness.pdf
He also values future lives equally to existing lives (the "longtermist" view) so any timme where we aren't expanding to the stars is morally repugnant:
> One recent paper speculates, using loose theoretical considerations based on the rate of increase of entropy, that the loss of potential human lives in our own galactic supercluster is at least ~10^46 per century of delayed colonization. [snip] The effect on total value, then, seems greater for actions that accelerate technological development than for practically any other possible action.
Let me requote that:
The effect on total value, then, seems greater for actions that accelerate technological development than for practically any other possible action.
https://nickbostrom.com/astronomical/waste
I'd highly recommend you read the linked article this discussion is about. It goes into some depth about on these points.
Yes, that's a logical conclusion, but it's not right because of the weird "longtermist" logic. Their "AI risk research" is about AI alignment, which is a "longtermist" priority.
To repost my previous quote:
> Many longtermists reckon that the antidote to the threat from technology is more technology; mastering artificial intelligence will forestall a malevolent AI from enslaving us. They also believe that technological acceleration is morally good in its own right.
This is the "acceleratist view" of AI research, of which Nick Bostrom is a prominent proponent: basically he views progress as inevitable, so it's best to accelerate AI that is aligned with human values. See https://nickbostrom.com/papers/openness.pdf
He also values future lives equally to existing lives (the "longtermist" view) so any timme where we aren't expanding to the stars is morally repugnant:
> One recent paper speculates, using loose theoretical considerations based on the rate of increase of entropy, that the loss of potential human lives in our own galactic supercluster is at least ~10^46 per century of delayed colonization. [snip] The effect on total value, then, seems greater for actions that accelerate technological development than for practically any other possible action.
Let me requote that:
The effect on total value, then, seems greater for actions that accelerate technological development than for practically any other possible action.
https://nickbostrom.com/astronomical/waste
I'd highly recommend you read the linked article this discussion is about. It goes into some depth about on these points.
[deleted]
They've effectively poisoned the terms "effective", "altruism", "longterm", "rational", and "utilitarian".
Because it was all a hustle to continue doing exactly whatever they wanted to do. Here is a good book that talks about the mindset:
https://www.amazon.com/Winners-Take-All-Charade-Changing/dp/...
It's easy to say you're doing things "longterm" for "future generations". The non-existent are the easiest people to advocate for. They demand absolutely nothing of you. And if your projected "altruism" will only really be felt well after you're dead, you aren't on the hook to show results now. They claim this is utilitarian, but it's not. It never was.
What is known is that small real changes now can affect large changes later. Doing things to improve today will improve tomorrow, next year, next decade, etc. If you are looking to be effective with your altruism, the most rational, utilitarian thing you can do is solve the problems of today. Anyone who says otherwise is trying to separate you from your money.
Because it was all a hustle to continue doing exactly whatever they wanted to do. Here is a good book that talks about the mindset:
https://www.amazon.com/Winners-Take-All-Charade-Changing/dp/...
It's easy to say you're doing things "longterm" for "future generations". The non-existent are the easiest people to advocate for. They demand absolutely nothing of you. And if your projected "altruism" will only really be felt well after you're dead, you aren't on the hook to show results now. They claim this is utilitarian, but it's not. It never was.
What is known is that small real changes now can affect large changes later. Doing things to improve today will improve tomorrow, next year, next decade, etc. If you are looking to be effective with your altruism, the most rational, utilitarian thing you can do is solve the problems of today. Anyone who says otherwise is trying to separate you from your money.
This is very wrong.
Most effective altruists are working on relatively near term things, like education in India or parasitic worms in Nigeria, those just aren't controversial and so get discussed less.
But long term projects like switching away from CO2 and to renewables can still be worth doing.
Most effective altruists are working on relatively near term things, like education in India or parasitic worms in Nigeria, those just aren't controversial and so get discussed less.
But long term projects like switching away from CO2 and to renewables can still be worth doing.
Humans have devised many, many institutions and philosophies for helping each other grow into powerful moral people. Uncountable religions, self help movements, mind-body wellness systems, the list goes on and on.
Every single one, without exception, has examples of people at the top of the institution, widely regarded as masters of the system, who are miserable destructive human beings.
This makes me profoundly sad. I could understand if every single group of people had members who are toxic, but the fact that the selection systems for progression are so predictably bad...
IME, the best you can do is to find a small local scene that practices the system in a way that is healthy and nurturing for the members and live with the cognitive dissonance caused by the rot at the top. Whether it's your small neighborhood church, a martial arts dojo or your toastmasters club, these things can be great in the small even if they're awful in the large.
Every single one, without exception, has examples of people at the top of the institution, widely regarded as masters of the system, who are miserable destructive human beings.
This makes me profoundly sad. I could understand if every single group of people had members who are toxic, but the fact that the selection systems for progression are so predictably bad...
IME, the best you can do is to find a small local scene that practices the system in a way that is healthy and nurturing for the members and live with the cognitive dissonance caused by the rot at the top. Whether it's your small neighborhood church, a martial arts dojo or your toastmasters club, these things can be great in the small even if they're awful in the large.
The solution is to tax the shit out of rich people and hand it over to the rest of society. I mean the maths has been done: whatever the rich give out in charity is always less than what they would pay in taxes.
The whole effective altruism thing is an ego trip experience. Wow I get my name on a hospital wing!
Taxation creates its own set of perverse incentives, I suggest we use a money pit: https://www.theonion.com/in-the-know-should-the-government-s...
It's worse than that; wealth aside it seems like every time people come together they can't seem to reliably reform or reject the bad actors.
That said, systems for routinely rotating the top of the hierarchy (such as taxation or term limits) are helpful to be sure.
That said, systems for routinely rotating the top of the hierarchy (such as taxation or term limits) are helpful to be sure.
This provides good context: https://astralcodexten.substack.com/p/effective-altruism-as-...
EA is a very very tempting horse to beat, to the point it's their own internal favorite sport. There are large quantities of ink spilled criticizing it, from inside and outside. But it is all pretty worthless without taking a look at the foundations of the movement.
EA is a very very tempting horse to beat, to the point it's their own internal favorite sport. There are large quantities of ink spilled criticizing it, from inside and outside. But it is all pretty worthless without taking a look at the foundations of the movement.
That essay kind of straw-mans the whole thing. He uses EA to alternatively mean the foundational philosophy, or the specific organization and the choices the people running it make, interchangeably.
"I don't think AI is a risk, so I'm opposed to EA" charitably means that that person is opposed to EA, the organization headed by the people named in the article, and the choices they have made, who have made themselves the face of the movement with the same name. It doesn't mean they think that helping people is bad.
It's one of those fun language tricks groups get to play when there's a movement of some sort, and then a specific organization pops up with the exact same name. Black Lives Matter famously did the same thing; great movement, but the organization with the same name has done some questionable things with the money they were given.
"I don't think AI is a risk, so I'm opposed to EA" charitably means that that person is opposed to EA, the organization headed by the people named in the article, and the choices they have made, who have made themselves the face of the movement with the same name. It doesn't mean they think that helping people is bad.
It's one of those fun language tricks groups get to play when there's a movement of some sort, and then a specific organization pops up with the exact same name. Black Lives Matter famously did the same thing; great movement, but the organization with the same name has done some questionable things with the money they were given.
I don't have the feeling the criticism in this thread is directed at a specific organization. The top negative comment refers to the name "effective altruism", and most comments are pretty general.
> to the point it's their own internal favorite sport.
Surely this is a massive irony - the time wasted debating it is time where they could have been collectively earning millions or billions to donate to worthy causes.
Surely this is a massive irony - the time wasted debating it is time where they could have been collectively earning millions or billions to donate to worthy causes.
Yes, listening to criticism is all well and good... for a time. But eventually you'll realize you need to just block and ignore the naysayers and do you own thing.
Which, to be fair, the 95% of effective altruists that you never hear about do.
Which, to be fair, the 95% of effective altruists that you never hear about do.
To me, SBF embodied the worst stereotypes about the Effective Altruism movement. The not-so-subtle air of I'm-smarter-than-thou, the hubris to think whatever they have thought of haven't been thought and dismissed by others (probably for good reason), and a bull-headed focus on risk and return that can be exactly quantified, to the point of ignoring other, "softer" sides of the issue.
SBF isn't alone in this. In school, while interning for a competing hedge fund, I met a drove of Jane Street interns who thought exactly the same way, in terms of "EV" or expected value -- the mean outcome from a random event. For example, whenever we would go out for ice cream, they would try to flip a coin to see who pays, given the "expected cost" was same. This behavior is fine if you're a market maker betting 0.001% of your worth on a toss that you win 51% of the time, and I imagine this is why Jane Street tries to cultivate it. But unfortunately, in this hyperfocus on returns, they forgot that "risk"/"variance" exists, and sometimes flipping a coin will land you in such a deep hole that you can't continue to play the game long enough to dig yourself out of there.
You can read some thoughts by the man himself here, which flies in the face of all such work (Kelly bets, etc.) in variance minimization: https://nitter.net/SBF_FTX/status/1337250686870831107
SBF isn't alone in this. In school, while interning for a competing hedge fund, I met a drove of Jane Street interns who thought exactly the same way, in terms of "EV" or expected value -- the mean outcome from a random event. For example, whenever we would go out for ice cream, they would try to flip a coin to see who pays, given the "expected cost" was same. This behavior is fine if you're a market maker betting 0.001% of your worth on a toss that you win 51% of the time, and I imagine this is why Jane Street tries to cultivate it. But unfortunately, in this hyperfocus on returns, they forgot that "risk"/"variance" exists, and sometimes flipping a coin will land you in such a deep hole that you can't continue to play the game long enough to dig yourself out of there.
You can read some thoughts by the man himself here, which flies in the face of all such work (Kelly bets, etc.) in variance minimization: https://nitter.net/SBF_FTX/status/1337250686870831107
I don't think anything he said there flies in the face of kelly. Single event, he is explicit about his utility function and why.
He just doesn't seem very good at managing risk in the real world. If you spend any time at an investment bank or similar where you do a lot of trading, there are just hundreds of "gotchas" that those institutions are aware of and manage (or they aren't and don't and go bankrupt eventually).
There are lots of fields like this in the real world, just lots of gotchas. He didn't seem to understand that. I'd bet Jane Street understands this at the org level, but the younger folks likely just see a little bit of the whole picture, and have a lot of theory.
He just doesn't seem very good at managing risk in the real world. If you spend any time at an investment bank or similar where you do a lot of trading, there are just hundreds of "gotchas" that those institutions are aware of and manage (or they aren't and don't and go bankrupt eventually).
There are lots of fields like this in the real world, just lots of gotchas. He didn't seem to understand that. I'd bet Jane Street understands this at the org level, but the younger folks likely just see a little bit of the whole picture, and have a lot of theory.
Kelly maximizes the long run returns as the number of bets approaches infinity. If the variance is extremely high on f(x) and you get a single bet it can be rational to go very heavy. (Here x is the Bernoulli with p=0.75 and f(x) the payoff of 10000:1 on x = 1)
This problem is usually considered from the other angle: if variance is high but the odds are unfavorable you should favor fewer bigger bets over many smaller bets. Why? Because the more bets you make the more likely you are to converge onto the expectation. A single big bet leaves you either running way over expectation or slightly under it. All of this assumes that you have to bet though, the safest thing to do with negative expectation is to just not bet.
His example is unusual in that it has positive expectation (so you want to bet) and high variance (because you’re limited to a single bet). f(x) = 0b or 10000b so the single sample has a mean of 5000 and a variance of 50,000,000 but the E[f(x)] is 7500. A bigger than Kelly bet is rational here but if you had unlimited (or a large number of) bets you should revert to Kelly.
Food for thought.
This problem is usually considered from the other angle: if variance is high but the odds are unfavorable you should favor fewer bigger bets over many smaller bets. Why? Because the more bets you make the more likely you are to converge onto the expectation. A single big bet leaves you either running way over expectation or slightly under it. All of this assumes that you have to bet though, the safest thing to do with negative expectation is to just not bet.
His example is unusual in that it has positive expectation (so you want to bet) and high variance (because you’re limited to a single bet). f(x) = 0b or 10000b so the single sample has a mean of 5000 and a variance of 50,000,000 but the E[f(x)] is 7500. A bigger than Kelly bet is rational here but if you had unlimited (or a large number of) bets you should revert to Kelly.
Food for thought.
> The not-so-subtle air of I'm-smarter-than-thou, the hubris to think whatever they have thought of haven't been thought and dismissed by others (probably for good reason), and a bull-headed focus on risk and return that can be exactly quantified, to the point of ignoring other, "softer" sides of the issue.
It’s a little long for an HN tagline but it’ll do.
It’s a little long for an HN tagline but it’ll do.
expected value makes some sense when you're sampling a lot, but basically none for one offs.
personally i think if you're making big bets/one-offs, you probably should be computing something like minimax trees for all the possible outcomes before making a choice and maybe adjust the bet size such that the worst case is something you can swallow.
personally i think if you're making big bets/one-offs, you probably should be computing something like minimax trees for all the possible outcomes before making a choice and maybe adjust the bet size such that the worst case is something you can swallow.
[deleted]
If you are making money trading - you are "making money" - if you are buing lottery tickets with a 1:1,000,000 payout and are on losing ticket #500,000 you are making "High EV trades"
[deleted]
I would consider "Effective Altruism" to be a fairly broad abstract attitude of trying to effectively help living beings (oneself & others), as guided by one's own opinions. Self-examination, critical thinking & debate are core values, and as far as I can tell continue to be so.
It's unfortunate that it's being defined by the rightly controversial opinions of famous people..
To balance this article, consider checking out one of the main EA websites https://www.effectivealtruism.org/articles/introduction-to-e...
For example it indicates that longtermism is "a school of thinking within effective altruism". The popularity of such ideas will vary with time and they will be further refined with ongoing thinking & healthy debate. I for one fail to understand why it's important to maximise the number of future lives, which don't even exist - it seems very risky & unsustainable.
It's unfortunate that it's being defined by the rightly controversial opinions of famous people..
To balance this article, consider checking out one of the main EA websites https://www.effectivealtruism.org/articles/introduction-to-e...
For example it indicates that longtermism is "a school of thinking within effective altruism". The popularity of such ideas will vary with time and they will be further refined with ongoing thinking & healthy debate. I for one fail to understand why it's important to maximise the number of future lives, which don't even exist - it seems very risky & unsustainable.
I used to work for one of the largest software vendors in the non-profit space worldwide and it fundamentally changed the way I look at the non-profit world. The big ones have essentially become these weird quasi-think tanks that passively influence policy or actively engage the community to achieve an often political agenda.
For example, a large non-profit can't explicitly lobby congress to ban sex education in schools. But, they can go around and "buy out" small non-profits into fulfilling their agenda. This is usually pretty easy to do because small non-profits are almost always cash-starved, and it can be really hard to say "no" when some outside figure is willing to provide a massive percentage of your operating budget in perpetuity in exchange for avoiding certain topics (like mutual consent) in schools.
This very scenario happened to an acquaintance of mine. She wanted to teach consent in schools after the #metoo scandal, spent 2 years grinding to improve access to consent-based communication for kids and then someone offered her obscene money under the condition that she make tweaks (i.e. "don't talk about consent in school because that will encourage children to have sex", that kind of stuff). She took the money because she saw a way out of needing hustle so hard for cash, but of course she sold her soul at the end and regretted it.
And none of those interactions ever get recorded. Nobody is tracking what big non-profits and charitable funds ask (or demand) others to do, and they get to market themselves as doing good for the world. I'm sure some of them are, and I'm sure there are tons of well-meaning people working for some of them, but that world was so much darker than I ever imagined, at least from the periphery.
For example, a large non-profit can't explicitly lobby congress to ban sex education in schools. But, they can go around and "buy out" small non-profits into fulfilling their agenda. This is usually pretty easy to do because small non-profits are almost always cash-starved, and it can be really hard to say "no" when some outside figure is willing to provide a massive percentage of your operating budget in perpetuity in exchange for avoiding certain topics (like mutual consent) in schools.
This very scenario happened to an acquaintance of mine. She wanted to teach consent in schools after the #metoo scandal, spent 2 years grinding to improve access to consent-based communication for kids and then someone offered her obscene money under the condition that she make tweaks (i.e. "don't talk about consent in school because that will encourage children to have sex", that kind of stuff). She took the money because she saw a way out of needing hustle so hard for cash, but of course she sold her soul at the end and regretted it.
And none of those interactions ever get recorded. Nobody is tracking what big non-profits and charitable funds ask (or demand) others to do, and they get to market themselves as doing good for the world. I'm sure some of them are, and I'm sure there are tons of well-meaning people working for some of them, but that world was so much darker than I ever imagined, at least from the periphery.
While I agree with you that non-profits can grow to a point that they become sclerotic and lost in their own bureaucracy, I'll give a contrary opinion as to why non-profits that actually want to be effective care deeply about public policy.
For example, consider a simple animal rescue non-profit. While it may on its surface just seem like "this organization should just exist to try to find homes for pets needing adoption", it's actually more helpful to look at a more fundamental question: "What causes animals to be put up for adoption in the first place." It turns out that in most states there are a host of systemic, government-caused problems that are directly responsible for the reasons why many people need to surrender their pets to shelters in the first place. So yes, the animal rescue non-profit is still in the day-to-day business of finding homes for homeless animals, I would say a good organization would actually do more to try to change the status quo and fix the root cause problem, and that often intersects with government policy.
For example, consider a simple animal rescue non-profit. While it may on its surface just seem like "this organization should just exist to try to find homes for pets needing adoption", it's actually more helpful to look at a more fundamental question: "What causes animals to be put up for adoption in the first place." It turns out that in most states there are a host of systemic, government-caused problems that are directly responsible for the reasons why many people need to surrender their pets to shelters in the first place. So yes, the animal rescue non-profit is still in the day-to-day business of finding homes for homeless animals, I would say a good organization would actually do more to try to change the status quo and fix the root cause problem, and that often intersects with government policy.
> Nobody is tracking what big non-profits and charitable funds ask (or demand) others to do, and they get to market themselves as doing good for the world.
Well, if there are no written agreements - take the money and do your thing anyway, then?
Well, if there are no written agreements - take the money and do your thing anyway, then?
>Nobody is tracking what big non-profits and charitable funds ask (or demand) others to do, and they get to market themselves as doing good for the world.
Forget about tracking, the people paying even get a tax deduction.
Forget about tracking, the people paying even get a tax deduction.
that's an interesting top-level post... could she write about it anonymously?
[deleted]
Just the other day I was listening to this Dave Troy podcast "against long termism"[0] and it's the first time I ever heard the term "effective altruism", however within the last few days of having listened to EA has come up multiple times. I also remembered when listening to it that the title screen of the Baba Is You videogame on the nintendo switch promotes the "Giving what we can" charity. Very interesting stuff, I highly recommend checking out not only that podcast but the rest of the Oil, Gold, Crypto and Fascism series from Dave Troy.
[0] https://podcasts.apple.com/au/podcast/against-longtermism-wi...
[0] https://podcasts.apple.com/au/podcast/against-longtermism-wi...
Thanks for mentioning Dave Troy and his podcast!
Their quality-of-life points combined with long-term-ism reminds me of that simcity build that maximized the population to absurd levels. If you get 'points' for each life, past and future, than maximizing the number of future life is the easiest way to maximize your high score.
> Bostrom calculates that 10E29 potential lives are lost for every second that we fail to colonise the supercluster of galaxies containing the Milky Way.
I think an easy solution would be to normalize the score, as I think the sum of human hapiness would be "better" if there was only ever one human who gave his life a 10/10, than two humans who gave their life a 9/10.
> Bostrom calculates that 10E29 potential lives are lost for every second that we fail to colonise the supercluster of galaxies containing the Milky Way.
I think an easy solution would be to normalize the score, as I think the sum of human hapiness would be "better" if there was only ever one human who gave his life a 10/10, than two humans who gave their life a 9/10.
There exists another solution that is "obvious" to "everybody" who has worked in life insurance (or long-term investments in finance): the concept of present value (PV):
> https://en.wikipedia.org/wiki/Present_value
Without giving a derivation (which you can find easily on the web), the idea is that a gain of 1 USD that you get in 1 year has to be discounted with a factor of (1+i), where i is the (long-term stable) interest rate. For 3 years, the discount factor is, of course, (1+i)^3, i.e.
- a gain of 1 USD that materializes in 1 year is worth 1/(1+i) USD today
- a gain of 1 USD that materializes in 3 years is worth 1/(1+i)^3 USD today
Thus, including this discount factor for a future monetary flow (as in the PV calculation) makes gains that materialize in, say, 1000 years worth less today than those of the same monetary value that materialize (much) earlier.
> https://en.wikipedia.org/wiki/Present_value
Without giving a derivation (which you can find easily on the web), the idea is that a gain of 1 USD that you get in 1 year has to be discounted with a factor of (1+i), where i is the (long-term stable) interest rate. For 3 years, the discount factor is, of course, (1+i)^3, i.e.
- a gain of 1 USD that materializes in 1 year is worth 1/(1+i) USD today
- a gain of 1 USD that materializes in 3 years is worth 1/(1+i)^3 USD today
Thus, including this discount factor for a future monetary flow (as in the PV calculation) makes gains that materialize in, say, 1000 years worth less today than those of the same monetary value that materialize (much) earlier.
Seems like Bostrom isn't thinking big enough. If we could invent time travel and seed life earlier in the universe, figure out a way to slow down entropy and star death and make the current universe last longer, or create new universes wholesale, we can get far bigger total life numbers than that. Alternatively, since what we really care about is QALYs, not lives, making even just one person truly immortal gives us a literal infinity that outweighs the value of all mortal sentient creatures that can ever exist.
This also implies every fictional supervillain who ever tried to achieve immortality was the morally best possible person, no matter what evil they performed in the attempt, even if they failed. Since no probability is ever 0, and any small number multiplied by infinity is still infinity, trying to achieve immortality is the maximally good thing any person can do.
This also implies every fictional supervillain who ever tried to achieve immortality was the morally best possible person, no matter what evil they performed in the attempt, even if they failed. Since no probability is ever 0, and any small number multiplied by infinity is still infinity, trying to achieve immortality is the maximally good thing any person can do.
Nick Bostrom wouldn't have missed that, he's a very precise philosopher (I'm reading his book Anthropic Bias, and damn I'm glad I didn't have to tease out that mess by myself).
This quote comes from a particular thought experiment. As I'm sure you'll grant, thought experiments don't discuss what to do if that exact hypothetical were reality, they only discuss what attitude to take towards this whole category of situations. In concrete terms yes, some humongously big number of lives are lost every second, more than 10E29 of them, if time travel is possible or creating new universes is possible, but that doesn't change the conclusion.
And you miss two things as far as I can see: achieving immortality for yourself is not the maximal good, since for one thing it would clearly be even better to achieve immortality for everyone. Second, you're equating infinities. It's hard for humans to reason about infinities, but mathematicians have in fact described that there are infinities of different orders: you can still speak of some "infinite" quantities as being greater than other "infinite" quantities. It's easier if you just drop the word in favour of "zillion" and reason intuitively about that.
This quote comes from a particular thought experiment. As I'm sure you'll grant, thought experiments don't discuss what to do if that exact hypothetical were reality, they only discuss what attitude to take towards this whole category of situations. In concrete terms yes, some humongously big number of lives are lost every second, more than 10E29 of them, if time travel is possible or creating new universes is possible, but that doesn't change the conclusion.
And you miss two things as far as I can see: achieving immortality for yourself is not the maximal good, since for one thing it would clearly be even better to achieve immortality for everyone. Second, you're equating infinities. It's hard for humans to reason about infinities, but mathematicians have in fact described that there are infinities of different orders: you can still speak of some "infinite" quantities as being greater than other "infinite" quantities. It's easier if you just drop the word in favour of "zillion" and reason intuitively about that.
Yeah I think if you want to be an actual smartypants you need some kind of exponential discounting. There's a lot of monsters that could eat us along the way, so people that exist right now or are about to exist are WAY more important.
If you wanted to go all in maybe try to consider the number of lives weighted by the probability they exist will. Some kind of drake equation like thing.
If you wanted to go all in maybe try to consider the number of lives weighted by the probability they exist will. Some kind of drake equation like thing.
Admittedly the SBF debacle is the first time I’ve heard of EA, but I don’t think it’s surprising that it’s attracted such characters.
Frankly it strikes me as just sketchy moral accounting, where you can be as greedy or exploitative as you want to be as long as you’re contributing to something that you can justify as having a higher “moral score”, but in the end you’re doing all the calculations and all the numbers are made up.
Frankly it strikes me as just sketchy moral accounting, where you can be as greedy or exploitative as you want to be as long as you’re contributing to something that you can justify as having a higher “moral score”, but in the end you’re doing all the calculations and all the numbers are made up.
... and that's bad for the actual altruistic causes because?
Yes there are such people. I don't think they would be less greedy and exploitative if they didn't have a way to feel better about it. They would find another way to feel better, perhaps one that doesn't have the side effect of actually help out random people that would benefit from that help.
Yes there are such people. I don't think they would be less greedy and exploitative if they didn't have a way to feel better about it. They would find another way to feel better, perhaps one that doesn't have the side effect of actually help out random people that would benefit from that help.
> and that's bad for the actual altruistic causes because
Because if all the numbers are made up, you don’t actually know what is an “altruistic” choice. You can say, gamble with other peoples money, and justify it by donating 10% of the returns to charity. And if the math works out in your head, it’s all good.
> Yes there are such people. I don't think they would be less greedy and exploitative if they didn't have a way to feel better about it.
Lol, we’re arguing about a moral philosophy here, and your argument is that anything bad that people do, keeping in line with the philosophy is irrelevant?
It’s also not that “there are such people”. I’m saying that EA will disproportionately attract the greedy because fundamentally it argues that you can buy your way into altruism. It’s the Catholic Church selling indulgences for the 21st century.
Because if all the numbers are made up, you don’t actually know what is an “altruistic” choice. You can say, gamble with other peoples money, and justify it by donating 10% of the returns to charity. And if the math works out in your head, it’s all good.
> Yes there are such people. I don't think they would be less greedy and exploitative if they didn't have a way to feel better about it.
Lol, we’re arguing about a moral philosophy here, and your argument is that anything bad that people do, keeping in line with the philosophy is irrelevant?
It’s also not that “there are such people”. I’m saying that EA will disproportionately attract the greedy because fundamentally it argues that you can buy your way into altruism. It’s the Catholic Church selling indulgences for the 21st century.
are you saying that if it weren't for EA, the greedy people who it attracts would behave better because they wouldn't have a way to wash their sins?
Maybe not all of them, but since a lot of recruiting is done at universities, where people are in their formative years, I believe the previous poster does have a point.
There are probably quite a few people just who want to do good, and get suckered into longtermism which results in them doing AI research instead.
You can actually see some of them in the EA forums, where they discuss this struggle.
Of course, as a lot of things surrounding EA, this is ironically quite hard to quantify ;)
Btw. please don't take it as an attack on EA as a concept, I believe that it has much to give provided it is not mixed with too much pseudo long-term thinking as described in the article, which in the end has way too much uncertainty in it for my taste.
There are probably quite a few people just who want to do good, and get suckered into longtermism which results in them doing AI research instead.
You can actually see some of them in the EA forums, where they discuss this struggle.
Of course, as a lot of things surrounding EA, this is ironically quite hard to quantify ;)
Btw. please don't take it as an attack on EA as a concept, I believe that it has much to give provided it is not mixed with too much pseudo long-term thinking as described in the article, which in the end has way too much uncertainty in it for my taste.
Does the effectiveness of various forms of altruism actually form a total order? I've yet to see any kind of rigorous argument in favour, yet seems like a fundamental requirement if one is going to expend such incredible resources into attempts to maximise this value.
The reason I ask is that, when I first discovered Effective Altruism, finding the premise to be deeply compelling, I ended up staying up all night watching videos trying to reconcile / assimilate it into my world view. In one video, various trolley problem scenarios were enumerated, and the EA guests were advocating sparing a lamborghini to the detriment of a child's life so that the car could be sold off and the money donated to saving multiple children in the developing world. Furthermore, there was some suggestion of a threshold where the altruism of sparing X domesticated animals was 'higher' than sparing Y humans, which definitely felt like it fell into [citation needed] territory.
The reason I ask is that, when I first discovered Effective Altruism, finding the premise to be deeply compelling, I ended up staying up all night watching videos trying to reconcile / assimilate it into my world view. In one video, various trolley problem scenarios were enumerated, and the EA guests were advocating sparing a lamborghini to the detriment of a child's life so that the car could be sold off and the money donated to saving multiple children in the developing world. Furthermore, there was some suggestion of a threshold where the altruism of sparing X domesticated animals was 'higher' than sparing Y humans, which definitely felt like it fell into [citation needed] territory.
> Does the effectiveness of various forms of altruism actually form a total order?
Given a moral compass, yes.
EA is about not just looking at the immediate and the definite, the relative value of animals and humans is out of scope (though of course many individuals there will have opinions on it).
Given a moral compass, yes.
EA is about not just looking at the immediate and the definite, the relative value of animals and humans is out of scope (though of course many individuals there will have opinions on it).
Maybe you can totally rank order goals or principles of morality within the confines of a single person, but how could you hope to do that for large groups of people without forcing everyone to assume the same things as you? Long-termism being a great example, because I personally, think the amount we should care about future people decays rapidly as we move into the future. It's not as if their lives don't have value equal to our own, but how can we hope to assess how many of them there may be or what aspects of the realities of human life will change over the next century even? This is like trying to predict the future by reading tea leaves, except you have to pledge 10% of your income to whichever leaf talks about the risk of general AI the loudest.
So should everyone be forced to value all future human lives as being as concretely valuable as our own? In order to put them into a total ordering of morality one is forced to make a judgement call on how many people we think are likely to exist in the future as well as how relevant our musings of today actually capture what life will be like in that future.
Effective altruists like to use exponential growth to beat people over the head with the idea that the future is more important than the present. Do they ever acknowledge the fact that the far future is also exponentially more unpredictable than seconds into the future?
It just reeks of hubris to me. To be a really effective altruist you must first create a time travel device.
So should everyone be forced to value all future human lives as being as concretely valuable as our own? In order to put them into a total ordering of morality one is forced to make a judgement call on how many people we think are likely to exist in the future as well as how relevant our musings of today actually capture what life will be like in that future.
Effective altruists like to use exponential growth to beat people over the head with the idea that the future is more important than the present. Do they ever acknowledge the fact that the far future is also exponentially more unpredictable than seconds into the future?
It just reeks of hubris to me. To be a really effective altruist you must first create a time travel device.
> Effective altruists like to use exponential growth to beat people over the head with the idea that the future is more important than the present. Do they ever acknowledge the fact that the far future is also exponentially more unpredictable than seconds into the future?
Not in my experience and all the time respectively.
Not in my experience and all the time respectively.
> the amount we should care about future people decays rapidly as we move into the future
Better to save a life today, than a billion lives a century from now, because the unstoppable planet-killer asteroid is going to hit us in 99 years.
Better to save a life today, than a billion lives a century from now, because the unstoppable planet-killer asteroid is going to hit us in 99 years.
It seems like suggesting there is _no such threshold_ is even more in need of citation.
Boundlessness means that X could be all domesticated animals, and Y=1 human. Surely that seems like not the right call, given that it would ultimately kill even more humans.
Boundlessness means that X could be all domesticated animals, and Y=1 human. Surely that seems like not the right call, given that it would ultimately kill even more humans.
There are definitely scenarios where an X big enough to cause broad ecological consequences would encounter little rhetorical friction to overwhelm Y=1.
[Admittedly this is all error-prone hazy recollection, but] From what I understood, the original scenario in question was about a case where X was large, but not of critical strategic consequence to the availability of the animal(s) in question
[Admittedly this is all error-prone hazy recollection, but] From what I understood, the original scenario in question was about a case where X was large, but not of critical strategic consequence to the availability of the animal(s) in question
> Nick Beckstead, the chief executive of the ftx Foundation, wrote in a PhD dissertation completed in 2013 that “It now seems more plausible to me that saving a life in a rich country is substantially more important than saving a life in a poor country, other things being equal.” Why? The former has the potential to create more long-term value and therefore save more lives.
Bringing racism and eugenics to a whole new "intellectual" and rationalist level. Sad that it took the swindling of $10-15 billions for all of this to come to light.
Bringing racism and eugenics to a whole new "intellectual" and rationalist level. Sad that it took the swindling of $10-15 billions for all of this to come to light.
While I find the statement in bad taste and guilty of classism I can't find the racist and/or eugenics aspects anywhere. He talks about rich and poor, not race or genetics.
> He talks about rich and poor
Looking at the current distribution of wealthy vs poor countries and people is kind of disingenuous and rationalistic to ignore the racist undertones. More exactly saying "we'll choose the wealthy people vs the poor ones if it will come to that" while ignoring that most of the wealthy people across the globe are of one "race", while the other peoples are of other "races", is, yes, racist.
About eugenics, whenever cold numbers only come into play in decisions like these I think it's safe to call it that, i.e. eugenics. Choosing wealthy people against poor people based on numbers alone is eugenics. You're choosing the more "worthy" ones, as that Beckstead guy explicitly said. More to the point, saying that people A should have more chances to live compared to people B, because people A is more worthy ("has the potential to create more long-term value"), is the definition of eugenics.
Looking at the current distribution of wealthy vs poor countries and people is kind of disingenuous and rationalistic to ignore the racist undertones. More exactly saying "we'll choose the wealthy people vs the poor ones if it will come to that" while ignoring that most of the wealthy people across the globe are of one "race", while the other peoples are of other "races", is, yes, racist.
About eugenics, whenever cold numbers only come into play in decisions like these I think it's safe to call it that, i.e. eugenics. Choosing wealthy people against poor people based on numbers alone is eugenics. You're choosing the more "worthy" ones, as that Beckstead guy explicitly said. More to the point, saying that people A should have more chances to live compared to people B, because people A is more worthy ("has the potential to create more long-term value"), is the definition of eugenics.
Someone's commented saying "EA was basically Judas Iscariot's method" and it got flagged. I think that statement was inaccurate. A motive like EA ("that oil could have been sold and the proceeds given to the poor") was the front for his real method, which in the Gospel is to sell out Jesus for money, so he can buy some land. His intentions were ultimately proven to not be altruistic. Knowing I'm guilty of some of this inappropriate judging myself, I think it's bad to judge people today who say they want to do EA by reaching for such a quick, easy "call them Judas" gambit.
Wow, I took EA to mean try donating using analysis like givewell does.
Judging by the article and comments, it looks as if people are living by a creed to be effectively altruistic.
In general any philosophy where you think there is an optimal answer suffers from the risk that the answer could be wrong.
In the case of givewell, one limitation is that you only support causes that demonstrate impact -- you don't support causes that take risks.
I'm sure there is many other reasons not think that everything can be solved through the application of logic :)
Judging by the article and comments, it looks as if people are living by a creed to be effectively altruistic.
In general any philosophy where you think there is an optimal answer suffers from the risk that the answer could be wrong.
In the case of givewell, one limitation is that you only support causes that demonstrate impact -- you don't support causes that take risks.
I'm sure there is many other reasons not think that everything can be solved through the application of logic :)
You weren't wrong, EA is basically the mindset of donating using analysis. As you fear, at the start the movement suffered from overemphasis on causes that can demonstrate impact. But it's not incompatible with risk-taking, if you personally deem the potential payoff as more than enough. Lately the movement is more nuanced, allowing room for harder-to-measure, "softer" causes like moral circle expansion, even if they're riskier, and even if you outsource part of your judgment to people closer to the cause: https://guerrillafoundation.org/additional-thoughts-on-effec...
This post does not have a concrete conclusion, it is just some thoughts I had while reading the article.
---
EA seems to have a core belief in post-scarcity. I've recently been thinking about post-scarcity. To my knowledge, I have not been engaging with EA content; I wonder if it has become pervasive because of their efforts, or if the time is ripe for the concept.
I've also been thinking about the number of different times that religion has evolved in a population. Religion fulfills various psychological needs; for a while it was popular for atheist thinkers to say we have evolved past the need for religion. I am agnostic, but I think people still need 'religion' and they will in the future. It may not be a religion with a personal god or gods; the concept of the divine comes from the human mind, the human mind will always have a place for godliness of some kind. I could be wrong, religion could be a blip, but I don't think so.
Religions of the past have reflected the tribalism inherent in the human condition. There has always been a fight with another tribe that was the most pressing. Other fights are suspended or suppressed until the main fight is concluded or paused. From my lifetime, the main fight was the Cold War until that concluded, and then it was national politics. It's very hard to come to terms with the core beliefs and morality of the main opponent, no matter who or when.
I'm sure EA says something about religion and tribalism, but the human experience is also important. The human experience starts in a place, with some people. Those people are your tribe, until you find another tribe. Their religion is your religion, until you find another religion.
---
EA seems to have a core belief in post-scarcity. I've recently been thinking about post-scarcity. To my knowledge, I have not been engaging with EA content; I wonder if it has become pervasive because of their efforts, or if the time is ripe for the concept.
I've also been thinking about the number of different times that religion has evolved in a population. Religion fulfills various psychological needs; for a while it was popular for atheist thinkers to say we have evolved past the need for religion. I am agnostic, but I think people still need 'religion' and they will in the future. It may not be a religion with a personal god or gods; the concept of the divine comes from the human mind, the human mind will always have a place for godliness of some kind. I could be wrong, religion could be a blip, but I don't think so.
Religions of the past have reflected the tribalism inherent in the human condition. There has always been a fight with another tribe that was the most pressing. Other fights are suspended or suppressed until the main fight is concluded or paused. From my lifetime, the main fight was the Cold War until that concluded, and then it was national politics. It's very hard to come to terms with the core beliefs and morality of the main opponent, no matter who or when.
I'm sure EA says something about religion and tribalism, but the human experience is also important. The human experience starts in a place, with some people. Those people are your tribe, until you find another tribe. Their religion is your religion, until you find another religion.
One thing a lot of people miss is that effective altruism has worked! The idea of donating to distant causes that do the most good (which is what EA is and this notion long predates the 21st century) has massively reduced the number of deaths of malaria. Bill Gate's efforts to eradicate polio have done a world of good.
Are EAs now patting themselves in the back for all the altruism that came before them? That's weird but on-brand. "We are responsible for all altruism, wherever it happened in space or time!"
Eh? That seems entirely unrelated to the comment you are replying to.
Bill G's anti-malarial efforts predate the EA movement.
You're making assumptions bigger than the pacific tectonic plate here.
While I absolutely applaud Bill Gate's philanthropy, and have no doubt he's one of the most effective charity donors on this planet, you cannot simply use that as a ledger entry in some karmic balance.
Bill at the peak of of his business career wielded enormous power and played a pivotal role in the IP regime that now rules the world, one that places low income nations at huge disadvantage in life or death industries like medicine.
When you say "effective altruism worked" with Bill as an example, what you're actually saying is "we should welcome abusive or even illegal behavior from people like Bill if they give away money later" which is absurd. There's nothing about philanthropy that requires you to do business like a sociopath as a prequel.
While I absolutely applaud Bill Gate's philanthropy, and have no doubt he's one of the most effective charity donors on this planet, you cannot simply use that as a ledger entry in some karmic balance.
Bill at the peak of of his business career wielded enormous power and played a pivotal role in the IP regime that now rules the world, one that places low income nations at huge disadvantage in life or death industries like medicine.
When you say "effective altruism worked" with Bill as an example, what you're actually saying is "we should welcome abusive or even illegal behavior from people like Bill if they give away money later" which is absurd. There's nothing about philanthropy that requires you to do business like a sociopath as a prequel.
Bill Gates' effective charitable giving should be commended.
Because the outcome of arguing against Bill giving charitably is Bill not giving charitably, not Bill not engaging in predatory business practices. Microsoft was cheating competitors and pursuing lock-in long before Bill founded the Gates Foundation.
Because the outcome of arguing against Bill giving charitably is Bill not giving charitably, not Bill not engaging in predatory business practices. Microsoft was cheating competitors and pursuing lock-in long before Bill founded the Gates Foundation.
Reading this blog post caused me to believe that EA was not a big contributor: https://milkyeggs.com/?p=175
Seems like it was more gross mismanagement and excessive stimulant consumption.
Seems like it was more gross mismanagement and excessive stimulant consumption.
> Over the past two years, I’ve heard many stories of young, ambitious people who came to effective altruism wanting to change the world but grew disenchanted.
> This disillusionment was partly because the community expends so much effort raising money for the proliferation of institutes and think-tanks that host its most prominent thinkers.
> Open Philanthropy, a foundation that Dustin Moskovitz helped to create, funds 80,000 Hours (over $10m since 2017),
Is this the same 80,000 hours as https://80000hours.org/? A self help guide and a job board? Listing Sam Bankman-Fried as the #1 anecdote? $10,000,000? Hm.
> This disillusionment was partly because the community expends so much effort raising money for the proliferation of institutes and think-tanks that host its most prominent thinkers.
> Open Philanthropy, a foundation that Dustin Moskovitz helped to create, funds 80,000 Hours (over $10m since 2017),
Is this the same 80,000 hours as https://80000hours.org/? A self help guide and a job board? Listing Sam Bankman-Fried as the #1 anecdote? $10,000,000? Hm.
It's interesting that the article notes the similarity of EA to a religion. I don't just mean the 'cultist' element of people afraid to speak out, message control,... but on a more fundamental level: the sort of practical ethics where elite university professors tell you they've figured out what you ought to do with seemingly impenetrable analytical rigour, honed in countless seminars. It really struck me about Peter Singer already, who seems to be the spiritual ancestor to EA and all of this.
I feel this is an "actions speak louder than words" situation. Has there been any effective believer in "Effective Altruism" that has actually got past the "accumulation of wealth" stage? Where are their great works of human charity? It's been a concept in many wealthy circles long enough we should be seeing the payoff.
Or is it just a self-justification of their current actions, of wealth accumulation, to be conveniently ignored later?
Or is it just a self-justification of their current actions, of wealth accumulation, to be conveniently ignored later?
> More than 110,000 donors have trusted GiveWell to direct their donations. Together, they have given over $1 billion to the organizations we recommend.
https://www.givewell.org/about
https://www.givewell.org/about
I would think so, yes.
Like, I know Vitalik Buterin said “stop making me a continued recipient of whatever coin project you are creating and just make those go to an effective charity like givewell and such”, and people followed that instruction.
The great works of charity, well, look at the charities they endorse, like, against malaria foundation and givewell etc. , and look at what these organizations have achieved.
Like, I know Vitalik Buterin said “stop making me a continued recipient of whatever coin project you are creating and just make those go to an effective charity like givewell and such”, and people followed that instruction.
The great works of charity, well, look at the charities they endorse, like, against malaria foundation and givewell etc. , and look at what these organizations have achieved.
I imagine that the rare folks who actually do structure their life to keep doing such things, are:
1. low-key, and
2. try to stay out of the spotlight to the maximum extent possible.
1. low-key, and
2. try to stay out of the spotlight to the maximum extent possible.
Wow, these guys are really lost.
I used to respect them, and even during the past few days I thought this crisis was a major opportunity for them to reflect and find their way again.
So I posted a comment on their forum with (i) an observation that they could regain trust by doing their share (I suggested twice the money they received from FTX) to make financial fraud victims whole before resuming their regular program, and (ii) a link to https://www.lesswrong.com/posts/BZ6XaCwN4QGgH9CxF/the-kelly-... , the first paragraph of which might have stopped SBF from making the big wrong turn in his reasoning (note that almost all other arguments on the Internet against SBF's reasoning don't actually work). Nothing but downvotes.
I recommend against any further association with these people in the strongest possible terms. (I'll retract this if it turns out that the downvoters were the exception and the EA community actually proceeds to demonstrate good citizenship in something like the manner I suggested, but I'm not holding my breath.)
I used to respect them, and even during the past few days I thought this crisis was a major opportunity for them to reflect and find their way again.
So I posted a comment on their forum with (i) an observation that they could regain trust by doing their share (I suggested twice the money they received from FTX) to make financial fraud victims whole before resuming their regular program, and (ii) a link to https://www.lesswrong.com/posts/BZ6XaCwN4QGgH9CxF/the-kelly-... , the first paragraph of which might have stopped SBF from making the big wrong turn in his reasoning (note that almost all other arguments on the Internet against SBF's reasoning don't actually work). Nothing but downvotes.
I recommend against any further association with these people in the strongest possible terms. (I'll retract this if it turns out that the downvoters were the exception and the EA community actually proceeds to demonstrate good citizenship in something like the manner I suggested, but I'm not holding my breath.)
> “It really suited my character at the time to try to think about effectiveness and rigour in everyday life,”
So was there any of that… that made getting into crypto or AI a good idea?
What was the math there?
I’ve seen effective altruism mentioned on HN before and I don’t really understand how it turns into directing people to jobs or specific tech… at all.
I feel like if you’re trying to help people the moment you mix your business in it, now you have conflicts of interest.
So was there any of that… that made getting into crypto or AI a good idea?
What was the math there?
I’ve seen effective altruism mentioned on HN before and I don’t really understand how it turns into directing people to jobs or specific tech… at all.
I feel like if you’re trying to help people the moment you mix your business in it, now you have conflicts of interest.
Totally agree. There are plenty of EAs, including myself, who use EA to direct their giving towards mosquito nets and not towards charitable organizations with overpaid CEOs or an otherwise terrible ROI.
EA is best as as critique of the narcissism of most wealthy western ‘altruism’. But all this mixing of billionaires, crypto, and long-termism has reintroduced narcissism into the community.
EA is best as as critique of the narcissism of most wealthy western ‘altruism’. But all this mixing of billionaires, crypto, and long-termism has reintroduced narcissism into the community.
At its root lies our desire to make our lives maximally meaningful, to the point where the existence of billions of people in the future can be traced back to one maximally effective decision or act.
At once the lyrics from Taylor Swift’s new song makes so much more sense:
“Did you hear my covert narcissism I disguise as altruism”
At once the lyrics from Taylor Swift’s new song makes so much more sense:
“Did you hear my covert narcissism I disguise as altruism”
Forgive me if this is reductive, but could this mentality not be summarized as:
'Justifying the means by the ends, accumulate as much wealth as possible at any cost to some groups in order to distribute it to the other groups towards which /I/ feel altruistic.'
This was prior to _recent events_ but there was a very good EconTalk episode about EA recently that's worth listening to again - https://simplecast.econtalk.org/episodes/erik-hoel-on-effect...
Effective altruists would risk the 10% chance of making thousands of clients lose all their money if it gives them a 50% chance of donating more to charity later. Since it makes sense according to their utility function. That’s what we learned from the FTX debacle. Sort of reminds me of the paper clip manufacturer A.I. who ends up killing all humans due to a broken utility function.
That's an over-generalization... I would think most effective altruists recognise that acting unethically by for example, risking client money, is not justified due to the long-term damage that causes. Big lies usually get found out. Most effective altruists are outraged at what SBF apparently did..
Apart from the already financially secure, who can possibly afford to be an effective altruist in the current climate (in the sense of donating a significant proportion of one's income). Similarly, who can afford to work for these EA foundations and charities on lowish salaries based in major cities but trustafarians?
I thought this was a good article on Effective Altruism by a blogger that has been featured here previously: https://www.residentcontrarian.com/p/im-trying-to-figure-out...
In my view it's a movement for people making excuses for themselves. You don't need movements to be altruistic, you just do it or not.
Of course they end up getting tied up in knots because they're trying to project altruism through continuously more complex justifications.
Of course they end up getting tied up in knots because they're trying to project altruism through continuously more complex justifications.
Yes, that would make the movement suspect. But if you'd read MacAskill or other sources, you'd know the original motivation was pretty much the opposite. The were aghast that lots of people currently donate to charities that ensure e.g. 4 livers to save 4 lives for the same amount of money that could've saved 1000 lives in another country. To reiterate, all these donors are already altruistic, and the original EAs were horrified at the 996 lives lost through the altruists' lack of thought.
The emphasis is not on the altruism but on the effectiveness. The altruism is a given: if it wasn't, you'd not be moved by any of the material anyway.
The emphasis is not on the altruism but on the effectiveness. The altruism is a given: if it wasn't, you'd not be moved by any of the material anyway.
I get that, but I find that kind of thinking achieves nothing practical because it's endlessly scalable, and hence the tie in knots problem because it starts justifying "well I just need to extract another 500% profit because then I can really start helping." and as soon as an immoral justification is accepted it's very conflicting.
Obviously this is not a blanked outcome, but it seems to be where the wealthy SBF types wander into, especially if their wealth is not entirely 'clean' in the first place.
Reminds me of arguments like, say, the Sacklers' opioid crisis isn't all bad because they donated to charity, we should appreciate the good that came.
Obviously this is not a blanked outcome, but it seems to be where the wealthy SBF types wander into, especially if their wealth is not entirely 'clean' in the first place.
Reminds me of arguments like, say, the Sacklers' opioid crisis isn't all bad because they donated to charity, we should appreciate the good that came.
An interesting discussion to be sure. But can't "net good" be a satisfactory judge for the Sacklers' crisis?
I was sympathetic to the movement when I first heard about it in the early 2010s, but once it reoriented itself around "longtermism" and especially donating to politicians and NGOs, I knew that it had shed any objective sense of altruism and become just another way to increase the power and prestige of neoliberalism/the cathedral.
[deleted]
I mean, it broke because utility isn't additive, which is the issue with utilitarian ethics generally.
I agree that utilities between people aren’t really commensurate in a way that allows adding them to be particularly well-defined/meaningful, and that therefore utilitarianism is not the actual full truth of morality,
However, I don’t think that precludes using a kind of utilitarianism where you just kinda fudge over that “there isn’t a canonical way of adding these” bit, in order to produce something which can be a rather good heuristic for “does this provide more or less help to people than that?”.
Like, yes, you can’t really add utilities, but if you value helping people, you can consider how much you value different people getting different amounts of different kinds of help, and if you pretend that this is the same thing as “the utility you provide them”, then you can add up how much you value the individual things and assume that how much you value the stuff adds linearly. And, in some cases, morality should presumably impose some constraints regarding how much you should value providing one sort of help in some quantity to some person or persons, compared to how much you value providing some other kind of help in some quantity to some other person(s).
And so, while I don’t think utilitarianism is true exactly, I think it likely that morality probably demands something which in some ways looks at least a little bit like utilitarianism? Though there are still cases where it is important to recognize that utilitarianism can lead one astray.
However, I don’t think that precludes using a kind of utilitarianism where you just kinda fudge over that “there isn’t a canonical way of adding these” bit, in order to produce something which can be a rather good heuristic for “does this provide more or less help to people than that?”.
Like, yes, you can’t really add utilities, but if you value helping people, you can consider how much you value different people getting different amounts of different kinds of help, and if you pretend that this is the same thing as “the utility you provide them”, then you can add up how much you value the individual things and assume that how much you value the stuff adds linearly. And, in some cases, morality should presumably impose some constraints regarding how much you should value providing one sort of help in some quantity to some person or persons, compared to how much you value providing some other kind of help in some quantity to some other person(s).
And so, while I don’t think utilitarianism is true exactly, I think it likely that morality probably demands something which in some ways looks at least a little bit like utilitarianism? Though there are still cases where it is important to recognize that utilitarianism can lead one astray.
Aye, you argue for sane bounds to utility where EA, to establish superiority as a morality, ignores and takes to extremes.
Good listen: https://pca.st/episode/61ba0858-0eac-4a92-91a3-9fbbacc49e69
Good listen: https://pca.st/episode/61ba0858-0eac-4a92-91a3-9fbbacc49e69
[deleted]
It seems like Effective Altuism is just an excuse to be an asshole.
I've always found EA to be the most capitalist way of trying to do good. Instead of like being humble, not accumulating wealth for yourself, not thinking of yourself as better, etc. EA is pushing for this weird self-centered egotistical philanthropic avenue.
That's literally the least efficient way to help anyone.
This also always felt very American. I'll get rich by paying workers poorly, finding any advantageous loophole I can, screwing customers, but once I'm rich I'll donate some of that money to do good...
How does this make any sense?
If you want to do good, just work for others from the get go. Be nice, be kind, be humble, offer mentoring and encouragement, don't hord things for yourself to redistribute it later, instead work to create sustainable opportunities for others.
That's literally the least efficient way to help anyone.
This also always felt very American. I'll get rich by paying workers poorly, finding any advantageous loophole I can, screwing customers, but once I'm rich I'll donate some of that money to do good...
How does this make any sense?
If you want to do good, just work for others from the get go. Be nice, be kind, be humble, offer mentoring and encouragement, don't hord things for yourself to redistribute it later, instead work to create sustainable opportunities for others.
> This also always felt very American. I'll get rich by paying workers poorly, finding any advantageous loophole I can, screwing customers, but once I'm rich I'll donate some of that money to do good...
That also feels very inaccurate and misrepresentative of the philosophy.
Rather, I'd say: live mindfully in a broken system. Pick a job that meets your needs with minimal intentional side-effects, donate some surplus to bettering the world. Do not be a paperclip maximizer or "ends justify the means".
That also feels very inaccurate and misrepresentative of the philosophy.
Rather, I'd say: live mindfully in a broken system. Pick a job that meets your needs with minimal intentional side-effects, donate some surplus to bettering the world. Do not be a paperclip maximizer or "ends justify the means".
That is a good and fair philosophy, except in my personal experience EAs have been very much a "paperclip maximizer" towards their (self-defined) goals of "bringing the most good to the world". From my point of view, claiming that the self-serving class of consultants and cryptobros donating 10% of their is not "true EA" sounds very much like a no true scotsman fallacy.
> Rather, I'd say: live mindfully in a broken system. Pick a job that meets your needs with minimal intentional side-effects, donate some surplus to bettering the world. Do not be a paperclip maximizer or "ends justify the means".
I think that is a very valid philosophy and one I would agree with.
However, EA has a large overlap with longtermism, which has some very paperclip maximizer like features:
> Providing we are capable of colonising the rest of our galaxy and nearby ones, a 1% reduction in the risk of extinction on such a time horizon would be the equivalent of saving 1032 lives. In such a perspective, nothing else matters.
> That year, Bostrom founded the Future of Humanity Institute at Oxford, dedicated to mitigating risk. He elaborated on his theory of longtermism in a book in 2008, urging people to be “good ancestors” by practising “altruism toward our descendants”. He didn’t mean composting or stopping driving, but mitigating existential risks to humanity – “x-risks” in community parlance – from hazards such as advanced artificial intelligence and bio-hacking, that could potentially wipe out or severely deplete humankind.
(from the linked article).
I think this a very problematic view, because (as noted) almost everything is justifiable if you think you are avoiding extinction of the human race.
I think that is a very valid philosophy and one I would agree with.
However, EA has a large overlap with longtermism, which has some very paperclip maximizer like features:
> Providing we are capable of colonising the rest of our galaxy and nearby ones, a 1% reduction in the risk of extinction on such a time horizon would be the equivalent of saving 1032 lives. In such a perspective, nothing else matters.
> That year, Bostrom founded the Future of Humanity Institute at Oxford, dedicated to mitigating risk. He elaborated on his theory of longtermism in a book in 2008, urging people to be “good ancestors” by practising “altruism toward our descendants”. He didn’t mean composting or stopping driving, but mitigating existential risks to humanity – “x-risks” in community parlance – from hazards such as advanced artificial intelligence and bio-hacking, that could potentially wipe out or severely deplete humankind.
(from the linked article).
I think this a very problematic view, because (as noted) almost everything is justifiable if you think you are avoiding extinction of the human race.
> That also feels very inaccurate and misrepresentative of the philosophy.
I admit, I don't want to generalize, I'm sure there are many interpretation of the ideas of EA and different people practice it differently.
That said, in practice, at least the prominent figures and their prominent groups, my impression is that they fall into what I described.
Seems there's an appealing aspect to it that's about being able to be the most cut throat, dirty capitalist where ends justify the means, and where you can feel good about being filthy rich off the back of everyone else, because you'll be efficiently altruistic in donating some of it back.
Feels there's a dimension to it that seems to justify taking from the poor to give to the poorer, while keeping a cut for yourself.
I admit, I don't want to generalize, I'm sure there are many interpretation of the ideas of EA and different people practice it differently.
That said, in practice, at least the prominent figures and their prominent groups, my impression is that they fall into what I described.
Seems there's an appealing aspect to it that's about being able to be the most cut throat, dirty capitalist where ends justify the means, and where you can feel good about being filthy rich off the back of everyone else, because you'll be efficiently altruistic in donating some of it back.
Feels there's a dimension to it that seems to justify taking from the poor to give to the poorer, while keeping a cut for yourself.
>Instead of like being humble, not accumulating wealth for yourself, not thinking of yourself as better, etc.
The EA advice is that if you're good at something valuable, then the best strategy for being charitable is to do that valuable thing and then donate your money. Do you think it's incompatible with humility to believe that you're good at something valuable? I don't think there's any notion of "thinking of yourself as better" baked in here.
The EA advice is that if you're good at something valuable, then the best strategy for being charitable is to do that valuable thing and then donate your money. Do you think it's incompatible with humility to believe that you're good at something valuable? I don't think there's any notion of "thinking of yourself as better" baked in here.
While I definitely think you have a point, I'm still not sure I have a better idea of how to do good. Perhaps you have though? (genuinely asking)
I work in tech and definitely make more than I need. I work for a small company (and don't even have any employees under me) so I'd at least like to think that I'm not that involved in exploiting people in order to make money.
Now the question that I have is what do I do with the excess money I'm making, especially since I do actually agree with you about the not accumulating wealth part. One option is of course spending a lot of time analysing charities to figure out which one I think would be the best to donate to, but while I'm still not completely ruling it out, I feel like that would make analysing charities almost a part time job. Since at least at the moment i don't want to do that, I still settled on donating based on GiveWell's recommendations, essentially outsourcing the charities analysis. I am aware that GiveWell and I probably don't share 100% of our values, but I truly don't see a better option.
So really, if you, or anyone else, has a better idea of what I can do with the excess money I'm making, I'd be more than happy to listen.
I work in tech and definitely make more than I need. I work for a small company (and don't even have any employees under me) so I'd at least like to think that I'm not that involved in exploiting people in order to make money.
Now the question that I have is what do I do with the excess money I'm making, especially since I do actually agree with you about the not accumulating wealth part. One option is of course spending a lot of time analysing charities to figure out which one I think would be the best to donate to, but while I'm still not completely ruling it out, I feel like that would make analysing charities almost a part time job. Since at least at the moment i don't want to do that, I still settled on donating based on GiveWell's recommendations, essentially outsourcing the charities analysis. I am aware that GiveWell and I probably don't share 100% of our values, but I truly don't see a better option.
So really, if you, or anyone else, has a better idea of what I can do with the excess money I'm making, I'd be more than happy to listen.
> EA is pushing for this weird self-centered egotistical philanthropic avenue.
That's literally the least efficient way to help anyone.
What.
The stereotypical EA person is a banker who makes $150k/year after tax and donates $100k/year to the against malaria foundation. As opposed to the lawyer who quits working for corporate and becomes a gardner, which seems to be what you're advocating.
The former seems to do far more good in the world, saving hundreds of young lives.
What.
The stereotypical EA person is a banker who makes $150k/year after tax and donates $100k/year to the against malaria foundation. As opposed to the lawyer who quits working for corporate and becomes a gardner, which seems to be what you're advocating.
The former seems to do far more good in the world, saving hundreds of young lives.
This is starting to feel like Motte and Bailey fallacy to me
https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy
Either that or just confusion about what and/or who we are talking about.
EA is not a well defined set of concepts or ideas. I do know a lot of people who identify as EA and have argued for "earn-to-give" where the giving part is deferred far into the future.
I won't spur the confusion further but my own personal experience is that there are several people I have personally met who more closely match the parents description than the description you give.
I'm all for earn-to-give if you are doing the giving soon and to causes where the outcomes are easier to measure, like what you mentioned. This is very commendable.
Where EA-style thinking runs into trouble is where it places excess confidence in future predictions
either A) I will eventually donate this money B) This cause I'm donating to has no obvious impact now, but it will have immense impact in the future or C) Even though I can't see all the negative impacts my work is having, my prediction/inference is that the negative impacts will be smaller than the positive impacts of my altruism.
So I have nothing against the idea of trying to maximize good, I just think that I frequently see it used in conjunction with excessively high-confidence predictions or inferences.
In your case, I think it's important that the banker do some sort of accounting of whether he is having a negative impact on their current project/firm etc. There are bankers who have had a huge negative impact in the past and so this does need to be part of the accounting.
Either that or just confusion about what and/or who we are talking about.
EA is not a well defined set of concepts or ideas. I do know a lot of people who identify as EA and have argued for "earn-to-give" where the giving part is deferred far into the future.
I won't spur the confusion further but my own personal experience is that there are several people I have personally met who more closely match the parents description than the description you give.
I'm all for earn-to-give if you are doing the giving soon and to causes where the outcomes are easier to measure, like what you mentioned. This is very commendable.
Where EA-style thinking runs into trouble is where it places excess confidence in future predictions
either A) I will eventually donate this money B) This cause I'm donating to has no obvious impact now, but it will have immense impact in the future or C) Even though I can't see all the negative impacts my work is having, my prediction/inference is that the negative impacts will be smaller than the positive impacts of my altruism.
So I have nothing against the idea of trying to maximize good, I just think that I frequently see it used in conjunction with excessively high-confidence predictions or inferences.
In your case, I think it's important that the banker do some sort of accounting of whether he is having a negative impact on their current project/firm etc. There are bankers who have had a huge negative impact in the past and so this does need to be part of the accounting.
> The former seems to do far more good in the world, saving hundreds of young lives
I guess the example is incomplete, the banker also contributes to some system in a way, is the work harmful to others?
And if the work is beneficial, they're still part of a system where, why are they taking such a large salary in the first place?
Now, I'm actually not opposed to this form of EA. Like the other commenter said, donating money is fine, it's choosing to become a banker and trying to become the highest paid banker in history, so that you can then go and donate a larger sum of that income that I find problematic.
The latter form of EA seems wrong to me, because the idea that you'll take more money from people, to then personally decide who else you'll give it back too and how much of it you'll keep for yourself or not, that does not seem altruistic.
Instead I wish that people adopted a moral where making excessively more than others, or owning excessively more things than others was just seen as wrong to begin with. If you don't need more than 100k, then just charge less for your services, or pay your employees and staff more, or heck accept higher tax brackets and give more back to fund social institutions and infrastructure.
I guess the example is incomplete, the banker also contributes to some system in a way, is the work harmful to others?
And if the work is beneficial, they're still part of a system where, why are they taking such a large salary in the first place?
Now, I'm actually not opposed to this form of EA. Like the other commenter said, donating money is fine, it's choosing to become a banker and trying to become the highest paid banker in history, so that you can then go and donate a larger sum of that income that I find problematic.
The latter form of EA seems wrong to me, because the idea that you'll take more money from people, to then personally decide who else you'll give it back too and how much of it you'll keep for yourself or not, that does not seem altruistic.
Instead I wish that people adopted a moral where making excessively more than others, or owning excessively more things than others was just seen as wrong to begin with. If you don't need more than 100k, then just charge less for your services, or pay your employees and staff more, or heck accept higher tax brackets and give more back to fund social institutions and infrastructure.
> heck accept higher tax brackets and give more back to fund social institutions and infrastructure.
Am I reading this wrong or are you saying that the government is the most efficient form of charity?
Am I reading this wrong or are you saying that the government is the most efficient form of charity?
It’s because we threw away Christianity as a moral compass which many interpret similarly and the hole was filled with EA, woke, etc. These people just totally missed the point of why we ditched Christianity/religion in the first place. It wasn’t so we could replace it with a different socio-political hegemony or otherwise moralizing institution.
This pretty much hits it on the nose for me. I think with a little extra spice of thinking you know best and experts in the field are dumb, aka, white savior complex.
> I've always found EA to be the most capitalist way of trying to do good.
If by "capitalist", you mean the system that incentivized the most selfish assholes on the planet to accidentally lift billions of people out of poverty, then I agree.
> This also always felt very American. I'll get rich by paying workers poorly...
The median American gets paid more than their counterparts in all but a couple countries. When you account for the population size and diversity of America, it's a statistical miracle how well Americans get paid.
EA and capitalism focus on efficiency and not virtue. You can't substitute being unvirtuous with being hyper-efficient, and I don't think anyone is going to argue otherwise. There's an argument to be made that efficiency is crowding out virtue, especially when it comes to capitalism, but even then I think you're significantly overestimating this effect. I don't think this same argument can be applied to EA. Overall, I think it inspires rationalist-obsessing nerds to be more interested in virtuous behavior. One of them happens to be me. Another, happens to be a billionaire scam artist.
If by "capitalist", you mean the system that incentivized the most selfish assholes on the planet to accidentally lift billions of people out of poverty, then I agree.
> This also always felt very American. I'll get rich by paying workers poorly...
The median American gets paid more than their counterparts in all but a couple countries. When you account for the population size and diversity of America, it's a statistical miracle how well Americans get paid.
EA and capitalism focus on efficiency and not virtue. You can't substitute being unvirtuous with being hyper-efficient, and I don't think anyone is going to argue otherwise. There's an argument to be made that efficiency is crowding out virtue, especially when it comes to capitalism, but even then I think you're significantly overestimating this effect. I don't think this same argument can be applied to EA. Overall, I think it inspires rationalist-obsessing nerds to be more interested in virtuous behavior. One of them happens to be me. Another, happens to be a billionaire scam artist.
Mostly it's a method of assuaging the guilt and resolving the cognitive dissonance that comes with having left-wing values but also wanting to be very rich.
This seems correct. Thanks for putting it so succinctly.
Man, I miss good puns in article headlines!
We need a way for bored narcissistic rich people to feel good about themselves while letting the rest of use go on about our lives.
I wish they just partied and spent their money competing on luxury buys with each other.
Instead, we made that "bad" or something, and now they compete on who can signal the most and biggest virtues, and here we are.
I wish they just partied and spent their money competing on luxury buys with each other.
Instead, we made that "bad" or something, and now they compete on who can signal the most and biggest virtues, and here we are.
now we all learn about consequentialism done wrong, yay
I'm not familiar with Effective Altruism, but a takeaway as a tech person working in the clinical trial field:
Most drugs go to market with the idea of being effective, e.g. treating as many people / conditions as possible. This leads to a problem that most people deal with a slightly different problem from the next person, so you just end up with large groups of small numbers of people who are essentially overlooked, because they're not profitable enough to make a drug or treatment for. This class is called "orphan drugs"
Yes, this is mostly "how the industry / capitalism" works, but that's the point. By measuring how effective altruism is, and by only doing the things that are most effective, you'll essentially end up with many "orphan" categories at the end.
Most drugs go to market with the idea of being effective, e.g. treating as many people / conditions as possible. This leads to a problem that most people deal with a slightly different problem from the next person, so you just end up with large groups of small numbers of people who are essentially overlooked, because they're not profitable enough to make a drug or treatment for. This class is called "orphan drugs"
Yes, this is mostly "how the industry / capitalism" works, but that's the point. By measuring how effective altruism is, and by only doing the things that are most effective, you'll essentially end up with many "orphan" categories at the end.
Perhaps before 1983, but there's a strong case that since the Orphan Drug Act was passed in the US, orphan drugs are overincentivized and a massive misallocation of resources [0]. Orphan drugs represent nearly half of FDA approved drugs in some recent years and come with huge tax benefits, near monopoly conditions, and typically the most outrageous prices. Maximizing quality-adjusted life years would be better achieved by incentivizing single-dose therapeutics for common ailments. Not the best example but the first that comes to mind is upcoming psychedelic therapy, which could have major potential, but is seen as having a broken business model because most people won't want more than one treatment.
[0] https://www.npr.org/sections/health-shots/2017/01/17/5095068...
[0] https://www.npr.org/sections/health-shots/2017/01/17/5095068...
“Efficiency” isn’t a universally, objectively better goal, but it’s what technocrats focus on.
There are many other equally good (and often conflicting) possible goals: sufficiency, diversity, happiness, regret, etc.
The current economy focuses on efficiency because it’s about infinite capital accumulation using finite resources, and that’s what benefits the people at the helm.
There are many other equally good (and often conflicting) possible goals: sufficiency, diversity, happiness, regret, etc.
The current economy focuses on efficiency because it’s about infinite capital accumulation using finite resources, and that’s what benefits the people at the helm.
EA is basically Judas Iscariot's method.
Judas got 30 pieces of silver, and with Jesus being crucified he lived a life free of sin on humanities behalf, opening the gates of heaven for us all.
Who knew Judas was the good guy this whole time?
Who knew Judas was the good guy this whole time?
I was referring to the alabastar flask.
But yeah it applied to 30 pieces of silver as well.
But yeah it applied to 30 pieces of silver as well.
Sounds like a bunch of sociopathic, terminally online people found each other and started yet another hierarchical fund raising scheme.
What’s better than picking individual charities based on utility? Charity index funds. Also known as social democratic governments that collect taxes.
What’s better than picking individual charities based on utility? Charity index funds. Also known as social democratic governments that collect taxes.
> Also known as social democratic governments that collect taxes.
Well yes, but in the absence (or inadequacy) of such government action for causes you care about, doesn't it make sense to donate to the charities that most effectively do the thing you care about?
Well yes, but in the absence (or inadequacy) of such government action for causes you care about, doesn't it make sense to donate to the charities that most effectively do the thing you care about?
> Also known as social democratic governments that collect taxes.
That conclusion does not follow. Why do you think a social democratic government has any focus on the utility of its work?
That conclusion does not follow. Why do you think a social democratic government has any focus on the utility of its work?
>”But if he established a well-paid private practice in Britain, he would be able to “earn to give”: he’d save “considerably more” lives and still be comfortable… …As the community has expanded, it has also become more exclusive. Conferences, seminars and even picnics held by the Centre for Effective Altruism are application-only.
Yes, by application only. Because the near-intuitive next step in thinking is that, heck, we can’t all be the comfortable one, someone has got to be the person getting their hands dirty incrementally adding to all of those QALY’s and clearly They’re too enlightened in understanding this to be the one living uncomfortably with dirty hands.
So the mathematical inevitability is there needs to be some gate keeping, and why shouldn’t it be the people who realized that also make the decisions about access?
Self serving, self reinforcing bullshit. Probably a mixture of honest good intentions and rationalizing of guilt for being better off than others.
The modern age makes it just about impossible to not be aware of a lot of awful crap in the world, so it’s no surprise that some of the current generation, finding themselves at or adjacent to the pinnacle of wealth and opportunity through some admixture of at least a little ability and a lot of good fortune and circumstance and some degree— large or small— of actual empathy, become plagued by the cognitive dissonance. By living so well but knowing that right this moment countless people are at the absolute opposite most desperate and miserable end of that spectrum, many soon to be dead, many dead just in the time it took me to type this.
Maybe I have it easier, cognitive dissonance wise, by virtue of having just a little of my formative years that 1) preceded a time when easy knowledge of the systemic awfulness of much of the world was unavoidable before a person hit a double-digit age and 2) Had a fair bit of personal awfulness in them as well. awfulness that was, in terms of the world wide range, still thankfully minor but enough that the dissonance is small.
So it’s no surprise that a somewhat cliquish group of the generation around the cultural/economic/academic centers of the world that, even just by virtue of their location, afford a great deal of opportunity have come up with a line of thinking that justifies their place in the world.
My hope is that an extra decade or two of experience and living and seeing how consequences play out slices off some critical mass of those folks who are a bit more self aware. At least enough to recognize that figuring out how to live your life as a decent person isn’t always going to be a problem that’s algorithmicaly or mathematically solvable.
Because I’d really prefer the next crop of people who find themselves in positions of wealth, power, and influence— again through some combination of (hopefully) a little merit and a lot of good fortune and circumstance… well, I’d prefer them to not be well intentioned but tragically arrogant assholes trying to figure out things like:
Is it better to suffer the inconvenience of saving the fleeing refugee on my doorstep? Or should I instead work an extra hour or two a month in order to earn a calculated incremental bonus in pay that might be donated to a cause that might be able to save 10 refugees?
I don’t want people in positions of power and influence to think that is, in any way, an interesting or worthwhile question to pursue as a mature intellectual use of their brain power rather than a shitty waste of time.
Yes, by application only. Because the near-intuitive next step in thinking is that, heck, we can’t all be the comfortable one, someone has got to be the person getting their hands dirty incrementally adding to all of those QALY’s and clearly They’re too enlightened in understanding this to be the one living uncomfortably with dirty hands.
So the mathematical inevitability is there needs to be some gate keeping, and why shouldn’t it be the people who realized that also make the decisions about access?
Self serving, self reinforcing bullshit. Probably a mixture of honest good intentions and rationalizing of guilt for being better off than others.
The modern age makes it just about impossible to not be aware of a lot of awful crap in the world, so it’s no surprise that some of the current generation, finding themselves at or adjacent to the pinnacle of wealth and opportunity through some admixture of at least a little ability and a lot of good fortune and circumstance and some degree— large or small— of actual empathy, become plagued by the cognitive dissonance. By living so well but knowing that right this moment countless people are at the absolute opposite most desperate and miserable end of that spectrum, many soon to be dead, many dead just in the time it took me to type this.
Maybe I have it easier, cognitive dissonance wise, by virtue of having just a little of my formative years that 1) preceded a time when easy knowledge of the systemic awfulness of much of the world was unavoidable before a person hit a double-digit age and 2) Had a fair bit of personal awfulness in them as well. awfulness that was, in terms of the world wide range, still thankfully minor but enough that the dissonance is small.
So it’s no surprise that a somewhat cliquish group of the generation around the cultural/economic/academic centers of the world that, even just by virtue of their location, afford a great deal of opportunity have come up with a line of thinking that justifies their place in the world.
My hope is that an extra decade or two of experience and living and seeing how consequences play out slices off some critical mass of those folks who are a bit more self aware. At least enough to recognize that figuring out how to live your life as a decent person isn’t always going to be a problem that’s algorithmicaly or mathematically solvable.
Because I’d really prefer the next crop of people who find themselves in positions of wealth, power, and influence— again through some combination of (hopefully) a little merit and a lot of good fortune and circumstance… well, I’d prefer them to not be well intentioned but tragically arrogant assholes trying to figure out things like:
Is it better to suffer the inconvenience of saving the fleeing refugee on my doorstep? Or should I instead work an extra hour or two a month in order to earn a calculated incremental bonus in pay that might be donated to a cause that might be able to save 10 refugees?
I don’t want people in positions of power and influence to think that is, in any way, an interesting or worthwhile question to pursue as a mature intellectual use of their brain power rather than a shitty waste of time.
And a brief addendum: it is vastly both telling and unsettling that the core movement 1) recognizes that their most brutal “calculations” would be looked on in horror and absurdity (rationalized no doubt as those who simply are not enlightened or intelligent enough to understand) and therefore 2) work to silence adherents who start to explore them in any type of open forum, necessitating 3) a PR campaign that blithely pays lips service to welcoming all ideas and open discussion at the same time that 4) it is clear there are aspects or implications of their philosophy that they are only willing to talk about behind closed doors.
It really, really, is not hard to see where applying quasi mathematical calculation will get you when you start weighing hypothetical future lives against the resources directed today towards various EA projects.
Anything can be justified when you’re trying to do mathematical ethics against vastly unlikely but catastrophically bad scenarios, all while ignoring the fact that if you don’t address the “right now” systemic and intractable problems we’ll get to much more likely existential crisis long before we have to worry about runaway ai-complete malevolent entities causing harm.
And that is actually an apt example— there is a certain amount of overlap between the EAS movement and the rational discussion communities that came up the idea of Roko’s Basilisk.
It really, really, is not hard to see where applying quasi mathematical calculation will get you when you start weighing hypothetical future lives against the resources directed today towards various EA projects.
Anything can be justified when you’re trying to do mathematical ethics against vastly unlikely but catastrophically bad scenarios, all while ignoring the fact that if you don’t address the “right now” systemic and intractable problems we’ll get to much more likely existential crisis long before we have to worry about runaway ai-complete malevolent entities causing harm.
And that is actually an apt example— there is a certain amount of overlap between the EAS movement and the rational discussion communities that came up the idea of Roko’s Basilisk.
I've seen this criticism a few times and figured it was just ambiguous.... while the FTX affair is obviously bad for the institutions that represent the modern EA movement, to me this doesn't represent a real critique of EA as a philosophy. "Most EA supporters are techbros and FTX reinforces the view that techbros are bad, therefore EA is bad" is not a real argument, it's a sneer. Is the implication that I should start donating money to the Red Cross rather than GiveWell? I don't think so. At the margin yes I should be more suspicious that my money won't be used well, but I don't believe that all of these institutions are corrupt.
The more material argument is whether from an EA perspective running massive scams and funneling the proceeds to EA causes should be applauded or not. Obviously the fact that this discussion is even being surfaced will cause brand damage, but again it says nothing to which perspective is correct, just that these discussions are subject to optics-based constraints. It's like the open discussions in the rationalist community over whether various horrible civilizational collapses happening to humanity might actually be desirable because it will prevent (or at least delay) an AI from killing us. There's serious discussions but it never plays well to bring this to a "normie" audience. But pretty much every movement has its Straussian elements where the "insider" narratives contradict the "outsider" narratives.
The more material argument is whether from an EA perspective running massive scams and funneling the proceeds to EA causes should be applauded or not. Obviously the fact that this discussion is even being surfaced will cause brand damage, but again it says nothing to which perspective is correct, just that these discussions are subject to optics-based constraints. It's like the open discussions in the rationalist community over whether various horrible civilizational collapses happening to humanity might actually be desirable because it will prevent (or at least delay) an AI from killing us. There's serious discussions but it never plays well to bring this to a "normie" audience. But pretty much every movement has its Straussian elements where the "insider" narratives contradict the "outsider" narratives.
> The more material argument is whether from an EA perspective running massive scams and funneling the proceeds to EA causes should be applauded or not.
Is that really the material argument? It seems obvious to this normie that the answer ought to be "hell no".
Is that really the material argument? It seems obvious to this normie that the answer ought to be "hell no".
[deleted]
[deleted]
"but my _intentions_ were pure" lol who falls for that anymore
Not sure how this relates to my comment, unless you misread my point about "institutions" as being about "intentions"...
I've been convinced for sometime now that "effective altruism" (Lesswrong-ism? Yudkowskianism?) is the communism of the 21st century.
Lots of parallels here beyond the obvious common thread of utilitarian collectivism:
- Censorship: the concept of "infohazards" is super-popular https://nickbostrom.com/information-hazards.pdf https://www.lesswrong.com/tag/information-hazards
- Social control + the creation of an 'other': They are absolutely convinced that sentient AGI will inevitably kill everyone without any real evidence. However, if we manage to create sentient AGI, what these guys plan to do with it is best described as modern slavery https://www.lesswrong.com/tag/ai-boxing-containment#:~:text=...
- Surveillance: They justify it using Bostrom's "vulnerable world hypothesis": https://nickbostrom.com/papers/vulnerable.pdf?platform=hoots... (TFA also discusses it well)
- Desire for government power: TFA is super clear about that. More interestingly, here's the FTX Future Fund on space governance: "We’d love to see workshops or a mock constitutional convention where sharp people think hard about how to structure international governance institutions for the long-term future, or how to govern space settlement. As explained in more detail on our areas of interest page, we believe that the onset of space settlement could be a watershed moment in human history. We want people to start thinking about how it should work." https://ftxfuturefund.org/projects/alternative-voting-system...
Also: https://80000hours.org/problem-profiles/space-governance/
- Entryism/influence operations: Pretty much every career profile on 80,000 Hours encourages taking a job at government agencies/influential private organizations to push the EA agenda https://80000hours.org/career-reviews/
Also, https://en.wikipedia.org/wiki/Roko%27s_basilisk. No one with morals and a spine should fear this. (FWIW, the same justification of fearing retribution by an evil power for refusing to collaborate could be used by people in occupied countries who collaborated with the Nazis.)
Lots of parallels here beyond the obvious common thread of utilitarian collectivism:
- Censorship: the concept of "infohazards" is super-popular https://nickbostrom.com/information-hazards.pdf https://www.lesswrong.com/tag/information-hazards
- Social control + the creation of an 'other': They are absolutely convinced that sentient AGI will inevitably kill everyone without any real evidence. However, if we manage to create sentient AGI, what these guys plan to do with it is best described as modern slavery https://www.lesswrong.com/tag/ai-boxing-containment#:~:text=...
- Surveillance: They justify it using Bostrom's "vulnerable world hypothesis": https://nickbostrom.com/papers/vulnerable.pdf?platform=hoots... (TFA also discusses it well)
- Desire for government power: TFA is super clear about that. More interestingly, here's the FTX Future Fund on space governance: "We’d love to see workshops or a mock constitutional convention where sharp people think hard about how to structure international governance institutions for the long-term future, or how to govern space settlement. As explained in more detail on our areas of interest page, we believe that the onset of space settlement could be a watershed moment in human history. We want people to start thinking about how it should work." https://ftxfuturefund.org/projects/alternative-voting-system...
Also: https://80000hours.org/problem-profiles/space-governance/
- Entryism/influence operations: Pretty much every career profile on 80,000 Hours encourages taking a job at government agencies/influential private organizations to push the EA agenda https://80000hours.org/career-reviews/
Also, https://en.wikipedia.org/wiki/Roko%27s_basilisk. No one with morals and a spine should fear this. (FWIW, the same justification of fearing retribution by an evil power for refusing to collaborate could be used by people in occupied countries who collaborated with the Nazis.)
The base model Roko's basilisk is pretty stupid, but I have talked to people who have worked on "improving" the idea, so it actually works in more realistic and diverse decision theories. It's like base model MOND vs improved versions of MOND, the idea is still pretty wacky but they did some fudging to get it to work better "empirically" (with that word being used very loosely, of course). I think some people don't want anyone talking about the improved basilisks, but if anyone cares I can try to dig them up. Probably don't care though, because it's really stupid.
>I've been convinced for sometime now that "effective altruism" (Lesswrong-ism? Yudkowskianism?) is the communism of the 21st century.
I mean, maybe sorta.. the eschatology is far more interesting with EA, at least.
I mean, maybe sorta.. the eschatology is far more interesting with EA, at least.
The EA eschatology is pretty much Christian eschatology, except they aren't at all sure their savior (a Friendly, Strong AI) will come. There's a good chance of just getting the Beast and the Harlot instead.
Hence the obsession with AI safety - Christians just have to work to bring Christ's kingdom to where they are. The EAs have to make sure nobody summons a vengeful demon overlord while they're slowly crafting their own all-loving, all-powerful digital deity.
Hence the obsession with AI safety - Christians just have to work to bring Christ's kingdom to where they are. The EAs have to make sure nobody summons a vengeful demon overlord while they're slowly crafting their own all-loving, all-powerful digital deity.
There is a lot of scepticism about EA in this thread, and for the most part I share all of it. It feels like a cult with an ulterior motive.
But: they haven't been caught actually doing anything remotely bad (they have some assholes as followers, some of whom are also crooks, but I struggle to blame EA for it) and perhaps, maybe, they really are donating tons of money to good causes?
Are there any estimates of how much money EA raised for the world? I mean, if each and every one of the donates 10% of their income, they are far ahead of me, and I'll shut up and put up.
But: they haven't been caught actually doing anything remotely bad (they have some assholes as followers, some of whom are also crooks, but I struggle to blame EA for it) and perhaps, maybe, they really are donating tons of money to good causes?
Are there any estimates of how much money EA raised for the world? I mean, if each and every one of the donates 10% of their income, they are far ahead of me, and I'll shut up and put up.
[deleted]
The title is unfairly conflating the idea with one organization. Plenty of people quietly donate some fraction of income to effective charities (effective altruism), without engaging with the Effective Altruism institution.
Though, yeah wow, I didn't realize how cliquey Effective Altruism had become.
Though, yeah wow, I didn't realize how cliquey Effective Altruism had become.
It is sad! Distributing mosquito nets in malaria-afflicted areas saves lives on the cheap and is universally acclaimed. How better to describe this than "effective altruism"?
Why do perfectly nice general descriptive terms so often become tightly bound with one small group and their whole very particular worldview?
Why do perfectly nice general descriptive terms so often become tightly bound with one small group and their whole very particular worldview?
> Distributing mosquito nets in malaria-afflicted areas saves lives on the cheap and is universally acclaimed. How better to describe this than "effective altruism"?
Philanthropy? Charity? Those words fit perfectly.
Does anyone sincerely set out to be ineffective in those endeavors? I can't imagine anyone who was knowingly insincere would have trouble also claiming the moniker of effective altruism. Nonetheless, I think the term "effective altruism" is also intended to encompass and imply some additional--if optional--characteristics about the giver or the form of giving, such as leading a more sustainable lifestyle or the consideration given to one's choices in choosing particular acts of charity. But we don't need that term merely for the act of distributing or funding the distribution of mosquito nets--philanthropy and charity are more than sufficient. And, technically speaking, charity has usually sufficed in any event--e.g. clergy who give their entire lives over to charity work weren't in search of more descriptive nomenclature, AFAIK.
Philanthropy? Charity? Those words fit perfectly.
Does anyone sincerely set out to be ineffective in those endeavors? I can't imagine anyone who was knowingly insincere would have trouble also claiming the moniker of effective altruism. Nonetheless, I think the term "effective altruism" is also intended to encompass and imply some additional--if optional--characteristics about the giver or the form of giving, such as leading a more sustainable lifestyle or the consideration given to one's choices in choosing particular acts of charity. But we don't need that term merely for the act of distributing or funding the distribution of mosquito nets--philanthropy and charity are more than sufficient. And, technically speaking, charity has usually sufficed in any event--e.g. clergy who give their entire lives over to charity work weren't in search of more descriptive nomenclature, AFAIK.
>Does anyone sincerely set out to be ineffective in those endeavors? I can't imagine anyone who was knowingly insincere would have trouble also claiming the moniker of effective altruism.
I mean, I'll bite the bullet and say yes. People will always choose to have more effectiveness in a ceteris peribus sense, but the best you could say for the actual observed pattern of charitable contributions is that people are being swayed in their giving by all sorts of concerns (social pressures, ignorance, bigotry, etc.) that cause them to trade off effectiveness for those things. And the message of EA is basically "recognize and call out these biases unless they're truly reflective of your values."
I mean, I'll bite the bullet and say yes. People will always choose to have more effectiveness in a ceteris peribus sense, but the best you could say for the actual observed pattern of charitable contributions is that people are being swayed in their giving by all sorts of concerns (social pressures, ignorance, bigotry, etc.) that cause them to trade off effectiveness for those things. And the message of EA is basically "recognize and call out these biases unless they're truly reflective of your values."
Not all philanthropy and charity are the same though. It's not all aimed through rigorous cost-benefit analysis. Which is not to say everyone should do that! But there's a difference between caring a lot about quantified impact and being guided by more qualitative considerations. What's the word to describe the difference? Quantitative altruism? Moneyball philanthropy?
Maybe you feel we don't need a word. But I find it odd, and kind of suspicious, when we insist we can't give a distinct name to something that is clearly a thing.
Maybe you feel we don't need a word. But I find it odd, and kind of suspicious, when we insist we can't give a distinct name to something that is clearly a thing.
> But there's a difference between caring a lot about quantified impact and being guided by more qualitative considerations. What's the word to describe the difference?
Sure, if you needed a term to categorize. We can build ontologies (and identities) all day if we want. But I ask this: can you quantify the difference between a qualitative vs quantitative emphasis in charity work? Point being: I'm not sure the distinction has much substance or utility. If one's aims are merely to feel good, without any concern for concrete, identifiable impacts, that doesn't fit the definition of altruism or charity or philanthropy. And with mixed motives (as they always are), it's really only the selfless aspect that counts as any of those when we're being pedantic. Even the tax code recognizes this--something is a charitable deduction only to the extent you don't personally derive value from it, and the deduction value is (in principle) the actual value imparted.
But if being able to identify as an "effective altruist" motivates more people to perform more charity, then who am I to argue ;)
Sure, if you needed a term to categorize. We can build ontologies (and identities) all day if we want. But I ask this: can you quantify the difference between a qualitative vs quantitative emphasis in charity work? Point being: I'm not sure the distinction has much substance or utility. If one's aims are merely to feel good, without any concern for concrete, identifiable impacts, that doesn't fit the definition of altruism or charity or philanthropy. And with mixed motives (as they always are), it's really only the selfless aspect that counts as any of those when we're being pedantic. Even the tax code recognizes this--something is a charitable deduction only to the extent you don't personally derive value from it, and the deduction value is (in principle) the actual value imparted.
But if being able to identify as an "effective altruist" motivates more people to perform more charity, then who am I to argue ;)
A charity organization that converts 70% of the money you give it into e.g. saved human lives is better than one that converts 5%.
EA was inspired by the observation that most people don't know where their money is going when they give it to charity.
[deleted]
Yeah it’s basically “give me your money and I promise I’ll do really cool things with it”.
No thanks.
No thanks.
kazinator(2)
"effective altruism" was never either effective nor altruistic. If people are finally working that out, good for them.
Ok, but please don't post shallow dismissals to Hacker News. We're hoping for something more than putdowns and supercilious swipes.
https://news.ycombinator.com/newsguidelines.html
p.s. You've unfortunately been posting a bunch of unsubstantive/baity comments lately. Can you please not? It's not what this site is for, and it destroys what it is for.
https://news.ycombinator.com/newsguidelines.html
p.s. You've unfortunately been posting a bunch of unsubstantive/baity comments lately. Can you please not? It's not what this site is for, and it destroys what it is for.
How did it fall short?
It justifies doing bad things for "the greater good".
Maximizing earnings by any means is verifiably not effectively altruistic, because if everyone did this... no one would ever engage in true charitable causes. Young people would compete simply to rip off the gullibility of older people looking to scratch the charitable itch.
Saw it referred to as "Indulgences for billionaires" which seems apt.
idk if it's all bad, I remember reading about it maybe a decade ago when rationalists talked about it. Like one good idea is donating to charities that are more efficient. But it seems like mostly BS wrapped up in mathematics to make it seem more rational. Some assumptions are not necessarily true but if you disagreed then you were immoral and irrational. It also comes with a form of very elitist virtue signalling like forming clubs where people bloviate about this.
Also a problem with charity, especially in the US, is as the IRS defines it, it's not necessarily charitable. So it's easy to confuse altruism with plain old politics, vanity, and for-profit business disguised as a tax-exempt and then argue that anyone that disagree is not rational.
Also a problem with charity, especially in the US, is as the IRS defines it, it's not necessarily charitable. So it's easy to confuse altruism with plain old politics, vanity, and for-profit business disguised as a tax-exempt and then argue that anyone that disagree is not rational.
It is purely people justifying doing the evil things they personally desire by claiming some unverifiable benefit that accrues to hypothetical future people who may never even exist.
Aaaand this is why time discounts exist and are steep. Not because we are incapable of or unwilling to comprehend future rewards, but because we don't know if they'll come to pass. It's a dampening parameter - if you don't do it, you fall prey to Basilisks.
Why do we see replications of religious concepts and actions in all these modern replacement movements? Just call it tithing, for example.
Please stop repackaging the same garbage and calling it new.
Please stop repackaging the same garbage and calling it new.
Well, I do believe that some of these charities explicitly recommend giving 10-20% of your income..
Effective Altruism is bullshit and here is why; because, the invisible hand has already shown itself to be a far better lever of human good through the acceptance of human nature. "Helping people" implies you know what people need. It's a power position to begin with and thus designed to fulfill the doers goals of doing good not the receivers 'need' of being helped.
Each of us, individually, pursuing our own self-interests produce value to society and others as a whole. Whether through producing products, offering services, performing labor, or consuming goods, we all seek our own interests to the betterment of the whole.
EA is just a new sort of chivalry for a new wannabe class of Nobles. That's it.
Each of us, individually, pursuing our own self-interests produce value to society and others as a whole. Whether through producing products, offering services, performing labor, or consuming goods, we all seek our own interests to the betterment of the whole.
EA is just a new sort of chivalry for a new wannabe class of Nobles. That's it.
>"Helping people" implies you know what people need.
No it doesn't. I'm content to outsource my charitable decision-making to recommenders that I think align with my values. A competitor is free to come along and convince me that they'd make a better recommendation and I'll shift my money accordingly, just like buying any other product. They can even skim a bit off the top if I really trust them and give to their funds! That's the invisible hand.
No it doesn't. I'm content to outsource my charitable decision-making to recommenders that I think align with my values. A competitor is free to come along and convince me that they'd make a better recommendation and I'll shift my money accordingly, just like buying any other product. They can even skim a bit off the top if I really trust them and give to their funds! That's the invisible hand.
The concept that seeking our own interests is to the betterment of the whole seems to be demonstrably false.. For example consider the tobacco and fossil fuel executives who are trying to delay the phase-out of their products. Or the people who buy/sell meat in spite of the resulting ecological damage & animal suffering.
By the way, a charity quite popular in mainstream Effective Altruism is Give Directly - which aims to avoid trying to figure out what poor people need by giving them cash.
By the way, a charity quite popular in mainstream Effective Altruism is Give Directly - which aims to avoid trying to figure out what poor people need by giving them cash.
> For example consider the tobacco
Smelly, dead hands, liberal
> fossil fuel
Polluting, dead hands, liberal
> buy sell meat
Greasy, dead hands, liberal
On good terms I would have permitted the fossil fuel claim, but in the context of the others I'd rather be a militant.
Smelly, dead hands, liberal
> fossil fuel
Polluting, dead hands, liberal
> buy sell meat
Greasy, dead hands, liberal
On good terms I would have permitted the fossil fuel claim, but in the context of the others I'd rather be a militant.
EA is about the most "invisible hand"-friendly school of philanthropic thought going, complete with ethical arguments generally siding in favour of individuals preferring to take the best paid private sector job going to raise funds rather than working directly for nonprofits.
How’d you like Atlas Shrugged? That speech goes on and on, doesn’t it.
Never read it. But, I did buy my brother a 1st edition cause he liked it. I'm aware of the theme.