Guess I'm a rationalist now(scottaaronson.blog)
scottaaronson.blog
Guess I'm a rationalist now
https://scottaaronson.blog/?p=8908
1,183 comments
They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value.
There is a term for this. "Getting stuck up your own butt." It wouldn't be so bad except that said people often take on an air of absolute superiority because they used "only logic" and in their head they can not be wrong. Many people end up thinking like this as teenagers or 20 somethings, but most will have someone in their life who smacks them over the head and tells them to stop being so foolish, but if you have enough money and the Internet you can insulate yourself from that kind of oversight.
There is a term for this. "Getting stuck up your own butt." It wouldn't be so bad except that said people often take on an air of absolute superiority because they used "only logic" and in their head they can not be wrong. Many people end up thinking like this as teenagers or 20 somethings, but most will have someone in their life who smacks them over the head and tells them to stop being so foolish, but if you have enough money and the Internet you can insulate yourself from that kind of oversight.
The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current people, and that the number of future people vastly outweighs everyone alive today (or who has ever lived), then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority.
People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.
People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.
Intelligence and rational thought is useful, but like any strategy it has its tradeoffs and limitations. No amount of intelligence can overcome the chaos of long time horizons, especially when we're talking about human civilization. IMHO it's reasonable to pick a long-term problem/risk and focus on solving it. But it's pure hubris to think rationality will give you anything approaching high confidence of what the biggest problems and risks actually are on a 20-50 year time horizon, let alone 200-500 years or longer.
The whole reason we even have time to think this way is because we are at the peak of an industrial civilization that has created a level of abundance that allows a lot of people a lot of time to think. But the whole situation that we live in is not stable at all, "progress" could continue, or we could hit a peak and regress. As much as we can see a lot of long-term trajectories (eg. peak oil, global warming), we really have no idea what will be the triggers and inflection points that change the social fabric in ways that are unforeseeable and quickly invalidate whatever prior assumptions all that deep thinking was resting upon. I mean 50 years ago we thought overpopulation was the biggest risk, and that thinking has completely flipped even without a major trajectory change for industrial civilization in that time.
The whole reason we even have time to think this way is because we are at the peak of an industrial civilization that has created a level of abundance that allows a lot of people a lot of time to think. But the whole situation that we live in is not stable at all, "progress" could continue, or we could hit a peak and regress. As much as we can see a lot of long-term trajectories (eg. peak oil, global warming), we really have no idea what will be the triggers and inflection points that change the social fabric in ways that are unforeseeable and quickly invalidate whatever prior assumptions all that deep thinking was resting upon. I mean 50 years ago we thought overpopulation was the biggest risk, and that thinking has completely flipped even without a major trajectory change for industrial civilization in that time.
I think one can levy a much more specific critique of rationalism: rationalism is in some sense self-defeating. If you are rational you will necessarily conclude that the fundamental dynamic that drives the (interesting parts of) the universe is Darwinian evolution, which is not rational. It blindly selects for reproductive fitness at the expense of all else. If you are a gene, you can probably produce more offspring in an already-industrialized environment by making brains that lean more towards misogyny and sexual promiscuity than gender equality and intellectual achievement.
The real conflict here is between Darwinism and enlightenment ideals. But I have yet to see any self-styled Rationalists take this seriously.
The real conflict here is between Darwinism and enlightenment ideals. But I have yet to see any self-styled Rationalists take this seriously.
I always liken this to that we’re all asteroids floating in space. There’s no free will and everything is determined. We just see the whole thing unfold from one conscious perspective.
Emotionally I don’t subscribe to this view. Rationally I do.
My critique for rational people is that they don’t seem to fully take experience into account. It’s assumptions + rationality + experience/data + whatever strong inclinations one has that seems to be the full picture for me.
Emotionally I don’t subscribe to this view. Rationally I do.
My critique for rational people is that they don’t seem to fully take experience into account. It’s assumptions + rationality + experience/data + whatever strong inclinations one has that seems to be the full picture for me.
> no free will
That always seemed like a meaningless argument to me. To an outside observer free will is indistinguishable from a random process over some range of possibilities. You aren’t going to randomly go to sleep with your hand in a fire, there’s some hard coded biology preventing that choice but that only means human behavior isn’t completely random, hardly a groundbreaking discovery.
At the other end we have no issues making an arbitrary decision where there’s no way to predict what the better choice is. So what exactly does free will bring to the table that we’re missing without it? Some sort of mystical soul, well what if that’s also deterministic? Unpredictability is useful in game theory, but computers can get that from a hardware RNG based on quantum processes like radioactive decay, so it doesn’t mean much.
Finally, subjectively the answer isn’t clear so what difference does it make?
That always seemed like a meaningless argument to me. To an outside observer free will is indistinguishable from a random process over some range of possibilities. You aren’t going to randomly go to sleep with your hand in a fire, there’s some hard coded biology preventing that choice but that only means human behavior isn’t completely random, hardly a groundbreaking discovery.
At the other end we have no issues making an arbitrary decision where there’s no way to predict what the better choice is. So what exactly does free will bring to the table that we’re missing without it? Some sort of mystical soul, well what if that’s also deterministic? Unpredictability is useful in game theory, but computers can get that from a hardware RNG based on quantum processes like radioactive decay, so it doesn’t mean much.
Finally, subjectively the answer isn’t clear so what difference does it make?
> That always seemed like a meaningless argument to me.
Same as that is not the lived experience. I notice that I care about free choice.
The idea that there's no free will may be a pessimistic outlook to some but to me it's a strictly neutral one. It used to be a bit negative, until I looked more closely that there's a difference between looking at a situation objectively and having a lived experience. When it comes to my inclinations and how I want to live life, lived experience takes precedence.
I don't have my thoughts sharp on it, but I don't think the concept even exists philosophically, but I think that's also what you're getting at. It's a conceptual remnant from the past.
Same as that is not the lived experience. I notice that I care about free choice.
The idea that there's no free will may be a pessimistic outlook to some but to me it's a strictly neutral one. It used to be a bit negative, until I looked more closely that there's a difference between looking at a situation objectively and having a lived experience. When it comes to my inclinations and how I want to live life, lived experience takes precedence.
I don't have my thoughts sharp on it, but I don't think the concept even exists philosophically, but I think that's also what you're getting at. It's a conceptual remnant from the past.
"Free choice" is the first step towards the solution to this paradox: free will is what a deterministic choice feels like from the inside. The popular notion of free will is that our decisions are undetermined, which must imply that there is a random element to them.
But though that is the colloquial meaning, it doesn't line up with what people say they want: you want to make your choice according to your own reasons. You want free choice. But unless your own reasoning includes a literal throw of the dice, your justifications deterministically decide the outcome.
"Free will" is the ability to make your own choices, and for most people most of the time, those choices are deterministic given the options and knowledge available. Free will and determinism are not only compatible, but necessarily so. If your choices weren't deterministic, it wouldn't be free will.
But though that is the colloquial meaning, it doesn't line up with what people say they want: you want to make your choice according to your own reasons. You want free choice. But unless your own reasoning includes a literal throw of the dice, your justifications deterministically decide the outcome.
"Free will" is the ability to make your own choices, and for most people most of the time, those choices are deterministic given the options and knowledge available. Free will and determinism are not only compatible, but necessarily so. If your choices weren't deterministic, it wouldn't be free will.
This is the position that is literally called compatibilist.
But when you probe people, while a lot of people will argue in ways that a philosopher might call compatibilist, my experience is that people will also strongly resist the notion that the only options are randomness and determinism. A lot of people have what boils down to a religious belief in a third category that is not merely a combination of those two, but infuses some mysterious third options where they "choose" that they can't explain.
Most of the time, people who believe there is no free will (and can't be), like me, take positions similar to what you described, that - again - a proponent of free will might describe as compatibilist, but sometimes we oppose the term for the reason above: A lot of people genuinely believe in a "third option" for choices are made.
And so there are really two separate debates on free will: Does the "third option" exist or not, and does "compatibilist free will" exist or not. I don't think I've ever met anyone who seriously disagrees that "free will" the way compatibilists define it exists, so when compatibilists get into arguments over this, it's almost always a misunderstanding...
But I have met plenty of people who disagree with the notion that things are deterministic "from the outside".
But when you probe people, while a lot of people will argue in ways that a philosopher might call compatibilist, my experience is that people will also strongly resist the notion that the only options are randomness and determinism. A lot of people have what boils down to a religious belief in a third category that is not merely a combination of those two, but infuses some mysterious third options where they "choose" that they can't explain.
Most of the time, people who believe there is no free will (and can't be), like me, take positions similar to what you described, that - again - a proponent of free will might describe as compatibilist, but sometimes we oppose the term for the reason above: A lot of people genuinely believe in a "third option" for choices are made.
And so there are really two separate debates on free will: Does the "third option" exist or not, and does "compatibilist free will" exist or not. I don't think I've ever met anyone who seriously disagrees that "free will" the way compatibilists define it exists, so when compatibilists get into arguments over this, it's almost always a misunderstanding...
But I have met plenty of people who disagree with the notion that things are deterministic "from the outside".
I'm a regular practitioner of magic, have written essays about it on Quora, and I can identify this mysterious third option as "the universe responding to your needs." You can use any number of religious terms to refer it to, like serendipity and the like, but none of them can capture the full texture of precisely how free will operates.
Approaching this subject from a rational perspective divorces you from subject and makes it impossible to perceive. You have to immerse yourself in it and one way to do that is magical practice. Having direct experience of the universe responding to your actions and mindset eventually makes it absurdly clear that the universe bears intelligence and it's in this intelligence that free will operates.
I'd never thought before now to connect magic this directly to free will. Thanks for the opportunity to think this through! If you're interested in a deeper discussion, happy to jump on a call.
Approaching this subject from a rational perspective divorces you from subject and makes it impossible to perceive. You have to immerse yourself in it and one way to do that is magical practice. Having direct experience of the universe responding to your actions and mindset eventually makes it absurdly clear that the universe bears intelligence and it's in this intelligence that free will operates.
I'd never thought before now to connect magic this directly to free will. Thanks for the opportunity to think this through! If you're interested in a deeper discussion, happy to jump on a call.
lmao, magic? seriously?
It is stronger than compatibilism. Compatibilism argues that free will and determinism are orthogonal. The argument I summarized is that free will is and must necessarily imply determinism..
I think that is a distinction without difference in as much as it's an excuse not to deal with it. But compatibilist "free will" must imply determinism unless some "magic" third alternative exists, because there isn't another option, and there is no evidence to suggest such a third alternative exists, so in practice every compatibilist I've had this discussion with have fallen back on arguing free will is compatible with determinism.
Definition doesn't tell all implications. In practice compatibilists do deeper analysis that reveals that determinism is required for free will.
Opposing the term gives a wrong result too, as people jump to hard determinism.
However "hard" your determinism, there is no support for the notion of agency, and that is all that matters. Without agency, free will is nothing but an illusion, with same moral consequences.
I think a reasonable interpretation of the colloquial sense of incompatibilist free will is that people want to be (or have the experience that they are) their own causal origins or prime movers. That they originate an action that is not (purely) the effect of all other actions that have occurred, but in such a way that they decided what that action was.
From the outside, this is indistinguishable from randomness. But from the inside, the difference is that the individual had a say in what the action would be.
Where this tends to get tangled up with notions of a soul, I think, is that one could argue that such a free choice needs some kind of internal state. If not, then the grounds by which the person makes the choice is a combination of something that is fixed and their environment, which then intuitively seems to reduce the free-will process to a combination of determined and random. So the natural thing to do is then to assign the required "being-ness" (or internal state if you will) to a soul.
But there may exist subtle philosophical arguments that sidestep this dilemma. I am not a philosopher: this is just my impression of what commonsense notions of free will mean.
From the outside, this is indistinguishable from randomness. But from the inside, the difference is that the individual had a say in what the action would be.
Where this tends to get tangled up with notions of a soul, I think, is that one could argue that such a free choice needs some kind of internal state. If not, then the grounds by which the person makes the choice is a combination of something that is fixed and their environment, which then intuitively seems to reduce the free-will process to a combination of determined and random. So the natural thing to do is then to assign the required "being-ness" (or internal state if you will) to a soul.
But there may exist subtle philosophical arguments that sidestep this dilemma. I am not a philosopher: this is just my impression of what commonsense notions of free will mean.
My point is that from the outside this doesn’t look like randomness at all, unless you are mistaking ignorance of their motives as a random oracle. If you can infer what set of goals drives their decision making, and the decision making process itself (e.g. ADHD brain vs careful considered action) you can very much predict their decisions. Marketing and PR people do this every single day. People don’t behave like random oracles, they behave like deterministic decision makers with complex, partly unknown goals so our predictions of their behavior are not always correct. That’s not the same thing as random.
People get emotional about free will because if you come to believe there is no free will it makes you question a lot of things that are emotionally difficult.
E.g. punishment for the sake of retribution is near impossible to morally justify if you don't believe in free will because it means you're punishing someone for something the had no agency over.
Similarly, wealth disparities can't be excused by someone choosing to work harder, because they had no agency in the "decision".
You can still justify some degree of punishment and reward, but a lack of free will changes which justifications are reasonable very substantially.
E.g. punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals, and that has emotionally difficult consequences. For example, for non-premeditated murders carried out out of passion rather than e.g. gang crimes, the odds of someone carrying out another is extremely low and the odds that the fear of a long prison sentence is an actual deterrence is generally low, and and so long prison terms are hard to justify once vengeance is off the table.
And so holding on to a belief in free will is easier to a lot of people than the alternative.
My experience is that there are few issues where people so easily get angry than if you suggest we don't have free will once they start thinking through the consequences (and some imagined ones...).
E.g. punishment for the sake of retribution is near impossible to morally justify if you don't believe in free will because it means you're punishing someone for something the had no agency over.
Similarly, wealth disparities can't be excused by someone choosing to work harder, because they had no agency in the "decision".
You can still justify some degree of punishment and reward, but a lack of free will changes which justifications are reasonable very substantially.
E.g. punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals, and that has emotionally difficult consequences. For example, for non-premeditated murders carried out out of passion rather than e.g. gang crimes, the odds of someone carrying out another is extremely low and the odds that the fear of a long prison sentence is an actual deterrence is generally low, and and so long prison terms are hard to justify once vengeance is off the table.
And so holding on to a belief in free will is easier to a lot of people than the alternative.
My experience is that there are few issues where people so easily get angry than if you suggest we don't have free will once they start thinking through the consequences (and some imagined ones...).
If there is no free will, thoughts about free will are predetermined and so is punishment. The punishers don’t have agency either. You seem to say that punishers do have free will, but criminals don’t?
I didn't say anything about whether free will exists or not, actually. The comment was specifically worded to explain why some people react to coming to believe there is no free will.
But, sure, I personally do not believe in free will. I'm saying there is no rational basis for thinking anyone has free will ever. I'm saying there is no evidence to suggest free will is possible. In fact, I'll go so far as to say that believing in free will is a religious belief with no support.
But that doesn't mean that events does not have effects on what happens next, just that we don't have agency. That an IF ... THEN ... ELSE ... statement is purely deterministic for deterministic inputs does not mean that changing the inputs won't affect the outputs.
If you "choose" to lay down and do nothing because you decide nothing matters because you don't have free will, you will still lose your job and starve. That it wasn't a "true" "free" choice does not change the fact that it has consequences.
One of the consequences of coming to accept that free will is an illusion is that you need to come to terms with what that means for your beliefs about a wide range of things.
Including that vengeance which might seem moral to some extent if the person who did something to you or others had agency suddenly become immoral. But we still have the feelings and impulses. Reconciling that is hard for a lot of people, and so a lot of people in my experience when faced with a claim like the one I made above that we have no free will tend to react emotionally to the idea of the consequences of it.
But, sure, I personally do not believe in free will. I'm saying there is no rational basis for thinking anyone has free will ever. I'm saying there is no evidence to suggest free will is possible. In fact, I'll go so far as to say that believing in free will is a religious belief with no support.
But that doesn't mean that events does not have effects on what happens next, just that we don't have agency. That an IF ... THEN ... ELSE ... statement is purely deterministic for deterministic inputs does not mean that changing the inputs won't affect the outputs.
If you "choose" to lay down and do nothing because you decide nothing matters because you don't have free will, you will still lose your job and starve. That it wasn't a "true" "free" choice does not change the fact that it has consequences.
One of the consequences of coming to accept that free will is an illusion is that you need to come to terms with what that means for your beliefs about a wide range of things.
Including that vengeance which might seem moral to some extent if the person who did something to you or others had agency suddenly become immoral. But we still have the feelings and impulses. Reconciling that is hard for a lot of people, and so a lot of people in my experience when faced with a claim like the one I made above that we have no free will tend to react emotionally to the idea of the consequences of it.
Are there deterministic solutions to the three body problem? Or the double pendulum? Or can you tell the t° at any point on earth for say, a millisecond in, say, 6h (feel free to chose a prefered point and time)? And what precision could you realistically produce in that?
If there are non deterministic processes that can be proven to exist, and those interact with deterministic processes, doesn't it follow that the deterministic process becomes non deterministic (since the result of the interaction is necessarily non deterministic), and that it is not continually deterministic.
So - can you see how nothing can be deterministic other than in isolation (or thought experiment really)?
Edit0: typo
If there are non deterministic processes that can be proven to exist, and those interact with deterministic processes, doesn't it follow that the deterministic process becomes non deterministic (since the result of the interaction is necessarily non deterministic), and that it is not continually deterministic.
So - can you see how nothing can be deterministic other than in isolation (or thought experiment really)?
Edit0: typo
There are deterministic solutions to the three body problem or the double pendulum in Newtonian mechanics.
We can’t measure things to arbitrary precision due to quantum mechanics, but Philosophy isn’t bound by the actual physical universe we inhabit. Arbitrary physical models allow for the possibility of infinite precision in measurement and calculation resulting in perfect prediction of future states forever. Alternatively, you could have a universe of finite precision (think Minecraft) which also allows for perfect calculation of all future states from initial starting conditions.
We can’t measure things to arbitrary precision due to quantum mechanics, but Philosophy isn’t bound by the actual physical universe we inhabit. Arbitrary physical models allow for the possibility of infinite precision in measurement and calculation resulting in perfect prediction of future states forever. Alternatively, you could have a universe of finite precision (think Minecraft) which also allows for perfect calculation of all future states from initial starting conditions.
That’s more effective as an argument to get rid of the most extreme forms of punishment (eg drawn and quartered) not all forms of retribution.
In a world without free will crimes of passion are simply the result of the situation which means that person would always chose murder in that situation. People who would respond with murder in an unacceptably wide range of situations is an edge case worth consideration without free will. Alternatively if we want nobody to respond with murder in a crime of passion situation evolutionary pressure could eventually work even without free will.
> E.g. punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals, and that has emotionally difficult consequences. For example, for non-premeditated murders carried out out of passion rather than e.g. gang crimes, the odds of someone carrying out another is extremely low and the odds that the fear of a long prison sentence is an actual deterrence is generally low, and and so long prison terms are hard to justify once vengeance is off the table.
That’s assuming absolute certainty about what happened. Punishments may make sense as a logical argument even if it’s only useful in a subset of cases if you can’t be absolutely sure which case something happened to be.
Uncertainty does a lot to align emotional heuristics and logical actions.
In a world without free will crimes of passion are simply the result of the situation which means that person would always chose murder in that situation. People who would respond with murder in an unacceptably wide range of situations is an edge case worth consideration without free will. Alternatively if we want nobody to respond with murder in a crime of passion situation evolutionary pressure could eventually work even without free will.
> E.g. punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals, and that has emotionally difficult consequences. For example, for non-premeditated murders carried out out of passion rather than e.g. gang crimes, the odds of someone carrying out another is extremely low and the odds that the fear of a long prison sentence is an actual deterrence is generally low, and and so long prison terms are hard to justify once vengeance is off the table.
That’s assuming absolute certainty about what happened. Punishments may make sense as a logical argument even if it’s only useful in a subset of cases if you can’t be absolutely sure which case something happened to be.
Uncertainty does a lot to align emotional heuristics and logical actions.
Whether or not you have free will is not relevant, as I had described in other comments.
> In a world without free will crimes of passion are simply the result of the situation which means that person would always chose murder in that situation. People who would respond with murder in an unacceptably wide range of situations is an edge case worth consideration without free will.
This is a significant argument. However, there is also worth considering if that is actually accurate, and if such a situation will occur (in a case where whoever would be killed would not effectively protect themself from this).
> That’s assuming absolute certainty about what happened. Punishments may make sense as a logical argument even if it’s only useful in a subset of cases if you can’t be absolutely sure which case something happened to be.
It is true that you do not have absolute certainty, but neither should you arrest someone who is not guilty.
> Uncertainty does a lot to align emotional heuristics and logical actions.
In some cases, yes, but it is not always valid. But, even if it is, this does not mean that you should not consider it logically if you are able to do so.
> In a world without free will crimes of passion are simply the result of the situation which means that person would always chose murder in that situation. People who would respond with murder in an unacceptably wide range of situations is an edge case worth consideration without free will.
This is a significant argument. However, there is also worth considering if that is actually accurate, and if such a situation will occur (in a case where whoever would be killed would not effectively protect themself from this).
> That’s assuming absolute certainty about what happened. Punishments may make sense as a logical argument even if it’s only useful in a subset of cases if you can’t be absolutely sure which case something happened to be.
It is true that you do not have absolute certainty, but neither should you arrest someone who is not guilty.
> Uncertainty does a lot to align emotional heuristics and logical actions.
In some cases, yes, but it is not always valid. But, even if it is, this does not mean that you should not consider it logically if you are able to do so.
I think that whether or not you have free will is not so important when making these considerations.
Whether or not you have a choice and free will, you can influence and be influenced by other stuff, since that is how anything is doing.
> punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals
I do agree with that, and I think that whether or not you have free will is not significant. Being emotionally difficult is not what makes it good or bad in this case (and it also does not seem to be so emotionally difficult to me, anyways). Reducing reoffending rates is what is important.
(Another issue is knowing if they are actually guilty (you shouldn't arrest people who are not actually guilty of murder); this is not always certain, either.)
I also think that it should mean that prisoners should not be treated badly and that prison sentences should not be too long. (Also, they shouldn't take up too much space by the prisons, since they should have free space for natural lands and for other buildings and purposes, but that is not quite the same issue, though.)
However, there may be cases where a fine might be appropriate, in order to pay for damages (although if someone else is willing to forgive them then such a fine may not be required). This does not justify a prison sentence or stuff like that, though.
Also, some people will just not like them anymore if they are accused of murder, even if they are not put in prison and not fined. This is not the issue for police and legal things; it is just what it will be. And, if it becomes known, people who disagree with the risk assessment can try to avoid someone.
And, if someone does commit a crime again and may have opportunity to do again in future, then this can be considered as being wrong the first time and this time hopefully you can know better.
Whether or not you have a choice and free will, you can influence and be influenced by other stuff, since that is how anything is doing.
> punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals
I do agree with that, and I think that whether or not you have free will is not significant. Being emotionally difficult is not what makes it good or bad in this case (and it also does not seem to be so emotionally difficult to me, anyways). Reducing reoffending rates is what is important.
(Another issue is knowing if they are actually guilty (you shouldn't arrest people who are not actually guilty of murder); this is not always certain, either.)
I also think that it should mean that prisoners should not be treated badly and that prison sentences should not be too long. (Also, they shouldn't take up too much space by the prisons, since they should have free space for natural lands and for other buildings and purposes, but that is not quite the same issue, though.)
However, there may be cases where a fine might be appropriate, in order to pay for damages (although if someone else is willing to forgive them then such a fine may not be required). This does not justify a prison sentence or stuff like that, though.
Also, some people will just not like them anymore if they are accused of murder, even if they are not put in prison and not fined. This is not the issue for police and legal things; it is just what it will be. And, if it becomes known, people who disagree with the risk assessment can try to avoid someone.
And, if someone does commit a crime again and may have opportunity to do again in future, then this can be considered as being wrong the first time and this time hopefully you can know better.
I don't find the consequences very hard to bear:
For example
> E.g. punishment for the sake of retribution is near impossible to morally justify if you don't believe in free will because it means you're punishing someone for something the had no agency over.
and
> E.g. punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals
are simply logical to me (even without assuming any lack of free will).
So what is emotionally difficult about this, as you claim?
For example
> E.g. punishment for the sake of retribution is near impossible to morally justify if you don't believe in free will because it means you're punishing someone for something the had no agency over.
and
> E.g. punishment might still be justified from the point of view of reducing offending and reoffending rates, but if that is the goal then it is only justified to the extent that it actually achieves those goals
are simply logical to me (even without assuming any lack of free will).
So what is emotionally difficult about this, as you claim?
I agree; they seem logical to me too, whether or not you have free will.
However, it would seem that not everyone believes that, though.
(It is not quite as simple as it might seem, because the situation is not necessarily always that clear, but other than that, I would agree that it is logical and reasonable, that punishment is only justified from the point of view of reducing offending and reoffending rates and only if it actually achieves those goals.)
However, it would seem that not everyone believes that, though.
(It is not quite as simple as it might seem, because the situation is not necessarily always that clear, but other than that, I would agree that it is logical and reasonable, that punishment is only justified from the point of view of reducing offending and reoffending rates and only if it actually achieves those goals.)
Then you're highly unusual (in a good way). Look at the amount of comments on social media with outcries over "too short" sentences for example, and the lack of political support for shortening sentences or improving prison standards.
I'm saying it's emotionally difficult to people because I've had this discussion many times over then last 30+ years and I've seen first hand how most people I have this conversation with tend to get angry and agitated over the prospect of not having moral cover for vengeance.
I'm saying it's emotionally difficult to people because I've had this discussion many times over then last 30+ years and I've seen first hand how most people I have this conversation with tend to get angry and agitated over the prospect of not having moral cover for vengeance.
> Then you're highly unusual (in a good way). Look at the amount of comments on social media with outcries over "too short" sentences for example, and the lack of political support for shortening sentences or improving prison standards.
I live in Germany.
When I observe the whole societal and political situation in the USA from the outside, it seems to me that it is rather "two blocks where in each of these there is somewhat an internal consensus regarding a quite some political positions. On the other hand, each of these two blocks is actively fighting the other one."
On the other hand, for Germany, I would claim that the opinions in society rather consist of "lots of very diverse stances (though in contrary to the USA less pronounced on the extreme ends) on a lot of topics that make it hard to reach a larger set of followers or some consensus in a larger group, i.e. in-fighting about all kinds of topics without these positions forming political camps (and the fractions for different opinions can easily change when the topic changes)."
Thus, in the given example, this means for a person out-crying "too short" sentences on social media, you will very likely find one who is out-crying the opposite position.
I live in Germany.
When I observe the whole societal and political situation in the USA from the outside, it seems to me that it is rather "two blocks where in each of these there is somewhat an internal consensus regarding a quite some political positions. On the other hand, each of these two blocks is actively fighting the other one."
On the other hand, for Germany, I would claim that the opinions in society rather consist of "lots of very diverse stances (though in contrary to the USA less pronounced on the extreme ends) on a lot of topics that make it hard to reach a larger set of followers or some consensus in a larger group, i.e. in-fighting about all kinds of topics without these positions forming political camps (and the fractions for different opinions can easily change when the topic changes)."
Thus, in the given example, this means for a person out-crying "too short" sentences on social media, you will very likely find one who is out-crying the opposite position.
“ E.g. punishment for the sake of retribution is near impossible to morally justify if you don't believe in free will because it means you're punishing someone for something the had no agency over.”
False, the punisher also has no will, so it doesn’t matter.
False, the punisher also has no will, so it doesn’t matter.
I have much less patience for C++ than I would in a world with free will.
Since there's no free will, outcomes are determined by luck, and what matters is how lucky we can make people through pit-of-success environments. Rust makes people luckier than C++ does.
I also have much less patience for blame than I do in a world with free will. I believe, for example, that blameless postmortems lead to much better outcomes than trying to pretend people had free will to make mistakes, and therefore blaming them for those mistakes.
You can get to these positions through means other than rejection of free will, but the most robust grounds for them are fundamentally deterministic.
Since there's no free will, outcomes are determined by luck, and what matters is how lucky we can make people through pit-of-success environments. Rust makes people luckier than C++ does.
I also have much less patience for blame than I do in a world with free will. I believe, for example, that blameless postmortems lead to much better outcomes than trying to pretend people had free will to make mistakes, and therefore blaming them for those mistakes.
You can get to these positions through means other than rejection of free will, but the most robust grounds for them are fundamentally deterministic.
If there is no free will, then all arguments about what should be done are irrelevant, since every outcome is either predetermined or random, so you have no influence on whether the project at work will choose Rust or C++. This choice was either made 13 billion years ago at the Big Bang, or it is an entirely random process.
> If there is no free will, then all arguments about what should be done are irrelevant, since every outcome is either predetermined or random, so you have no influence on whether the project at work will choose Rust or C++.
This is not correct. Whether or not you have free will, stuff influences and is influenced by other stuff, so these arguments are not meaningless or worthless.
> This choice was either made 13 billion years ago at the Big Bang, or it is an entirely random process.
I had thought of this before, and what I had decided is that both of these are also independent of having free will. For example, if the initial state includes unknown and uncomputable transcendental numbers which can somehow "encode" free will and then the working of physics is deterministic, then it is still possible (although not necessarily mandatory) to have free will, even though it is deterministic.
This is not correct. Whether or not you have free will, stuff influences and is influenced by other stuff, so these arguments are not meaningless or worthless.
> This choice was either made 13 billion years ago at the Big Bang, or it is an entirely random process.
I had thought of this before, and what I had decided is that both of these are also independent of having free will. For example, if the initial state includes unknown and uncomputable transcendental numbers which can somehow "encode" free will and then the working of physics is deterministic, then it is still possible (although not necessarily mandatory) to have free will, even though it is deterministic.
Lack of free will doesn’t prevent logical arguments from seeming to work.
Depends on whether you consider facts or theory. Facts don't prevent logical arguments from seeming to work, but lack of free will is theory. When theory doesn't match facts, theory is wrong.
We have built systems that don’t have free will and respond to logical arguments, so no theory is required here.
Random processes can’t use logic.
Fuzzy logic deals with truth values between 0 and 1. You can for example map water temperatures in such systems without having arbitrarily important cutoff points.
Such system often deal with uncertainty quite well including random noise on their inputs. The output ends up a function of both logic and randomness, but can still be useful.
Such system often deal with uncertainty quite well including random noise on their inputs. The output ends up a function of both logic and randomness, but can still be useful.
This is a strawman argument extended by those who rely on supernatural explanations. In reality, people's utterances and actions are part of the environment that determines future actions, just like everything else.
Sure, but that still doesn't matter: the fact that I wrote my previous comment is what caused you to write your response, but it's not like I had a choice to write that comment or some other: the fact that I wrote that comment, as well as everything that led to me writing it (conversations with teachers, my parents letting me watch English cartoons so I learned English, etc), were predetermined the moment the Big Bang happened, or they're just a quantum fluctuation.
What I'm saying is that there's no logical point to the concept "should" unless you have some concept of free will: everything that happens must happen, or is entirely random.
What I'm saying is that there's no logical point to the concept "should" unless you have some concept of free will: everything that happens must happen, or is entirely random.
Divorced from a religious context, it doesn't make any difference.
Which religious context, and why?
Randomness based free will still gives you a non-inevitable future.
So does a computer without free will acting on a physical RNG. Therefore it’s the RNG that matters not free will.
For naturalistic libertarians , free.will is.partly constituted by indtermimksm, not something entirely different.
If you get down to the quantum level there is no such thing as objective reality. Our perception that the world is made of classical objects that actually exist at particular places at particular times and have continuity of identity is an illusion. But it's a really compelling illusion, and you won't go far wrong treating it as if it were the truth in 99% of real-world situations. Likewise, free will is an illusion, nothing more than a reflection of our ignorance of how our brains work. But it is a really compelling illusion, and you won't go far wrong treating it as if it were the truth, at least some of the time.
> If you get down to the quantum level there is no such thing as objective reality.
What do you mean by that? It still exists doesn't it? Albeit in a probabilistic sense that becomes non-probabilistic at larger scales.
I don't know much about quantum other than the high level conceptual stuff.
What do you mean by that? It still exists doesn't it? Albeit in a probabilistic sense that becomes non-probabilistic at larger scales.
I don't know much about quantum other than the high level conceptual stuff.
> It still exists doesn't it?
It's controversial, but here is the argument that the answer is "no": See https://flownet.com/ron/QM.pdf
Or if you prefer a video: https://www.youtube.com/watch?v=dEaecUuEqfc
It's controversial, but here is the argument that the answer is "no": See https://flownet.com/ron/QM.pdf
Or if you prefer a video: https://www.youtube.com/watch?v=dEaecUuEqfc
That's non sequitur.
>Under QIT, a measurement is just the propagation of a mutually entangled state to a large number of particles.
eyeroll so it's MWI in disguise, but MWI is quantum realism. Illusion they talk about is that the observed macroscopic state is a part of the bigger superposition (incomplete observation). But that's dumb, even if it's a part of a bigger state, it's still real, because it's not made up, but observed.
>Under QIT, a measurement is just the propagation of a mutually entangled state to a large number of particles.
eyeroll so it's MWI in disguise, but MWI is quantum realism. Illusion they talk about is that the observed macroscopic state is a part of the bigger superposition (incomplete observation). But that's dumb, even if it's a part of a bigger state, it's still real, because it's not made up, but observed.
> it's MWI in disguise
That's kind of like saying that GRW is Copenhagen in disguise. It's not wrong, but only because it's making the word "disguise" do some pretty heavy lifting.
> MWI is quantum realism
No, it isn't because it can't account for the Born rule. See:
https://blog.rongarret.info/2019/07/the-trouble-with-many-wo...
That's kind of like saying that GRW is Copenhagen in disguise. It's not wrong, but only because it's making the word "disguise" do some pretty heavy lifting.
> MWI is quantum realism
No, it isn't because it can't account for the Born rule. See:
https://blog.rongarret.info/2019/07/the-trouble-with-many-wo...
It's a strange conclusion. You seemingly consider one measurement and expect to see Born rule, and when it doesn't manifest, then MWI is wrong? But Born rule doesn't manifest at sample size one in any interpretation, it manifests only in a long string of measurements. If you consider a long string of measurements, you will see Born rule as <Ψ|Born rule> = 1 - O(exp(-N)), which is basically a definition of empirical tendency.
Well, now I see that QIT isn't quite there. You say classical behavior emerges by tracing, mathematically, not as a physical process? In MWI classical behavior emerges as a physical process, not by tracing. That "look at part of the system (in which case you see classical behavior)" is provided by linear independence of different branches, so each observer naturally observes their branch from inside, and it looks isolated from other branches.
Well, now I see that QIT isn't quite there. You say classical behavior emerges by tracing, mathematically, not as a physical process? In MWI classical behavior emerges as a physical process, not by tracing. That "look at part of the system (in which case you see classical behavior)" is provided by linear independence of different branches, so each observer naturally observes their branch from inside, and it looks isolated from other branches.
If you insist that MWI must mean "a discrete number of clearly separated worlds", then yes, such interpretation would have a problem with the Born rule.
(That is apparently the definition the author of the linked article uses, guessing by his reaction: "Wait, what??? There is no 'well defined notion of how many branches there are?'")
I can only say that I have never met a proponent of MWI who meant this.
(That is apparently the definition the author of the linked article uses, guessing by his reaction: "Wait, what??? There is no 'well defined notion of how many branches there are?'")
I can only say that I have never met a proponent of MWI who meant this.
It’s probabilistic at all length scales. For example our solar system may suddenly come undone according to simulations.
There is no local realism. That doesn't at all add up to all-in-the-head idealism.
That's true. There is a metaphysical reality "out there", but it is radically different from what we perceive. Hence: an illusion. Note that an illusion is emphatically NOT the same thing as a delusion. Illusions are real sensory experiences common to nearly all humans. They just happen not to correspond to reality.
How would you know? If all that's known is either known through the senses or drawn out by reason from what is known through the senses, then by declaring that sense data do not reflect reality, you've cut yourself off form the possibility of knowing reality altogether.
> How would you know?
Because that is the best explanation for what I observe.
> by declaring that sense data do not reflect reality, you've cut yourself off form the possibility of knowing reality altogether
That is true, but only in the uninteresting sense that I can never completely eliminate the possibility that I am living in the Matrix. So yes, it's possible that I'm wrong about the existence of objective reality. But if objective reality is itself an illusion, it's a sufficiently compelling illusion that I'm not going to go far wrong by acting as if it were real.
Because that is the best explanation for what I observe.
> by declaring that sense data do not reflect reality, you've cut yourself off form the possibility of knowing reality altogether
That is true, but only in the uninteresting sense that I can never completely eliminate the possibility that I am living in the Matrix. So yes, it's possible that I'm wrong about the existence of objective reality. But if objective reality is itself an illusion, it's a sufficiently compelling illusion that I'm not going to go far wrong by acting as if it were real.
> That is true, but only in the uninteresting sense that I can never completely eliminate the possibility that I am living in the Matrix. [...] But if objective reality is itself an illusion, it's a sufficiently compelling illusion that I'm not going to go far wrong by acting as if it were real.
That seems squishy, as what constitutes "going far wrong" is not meaningful under skeptical assumptions.
A better stance is one of cognitive optimism that avoids the irrationality of skepticism. Skepticism is irrational, because it leads to incoherence, and because there is no rational warrant to categorically doubt the senses. For doubt to be rational, there must be a reason for it. To doubt without reason is not to be rational, but to be willful, and willful beliefs cannot be reasoned with; they are not the product of evidence or inference — and they certainly aren't self-evident — but rather the product of arbitrary choice. The logical possibility of living in the Matrix is no reason for doubting the senses, just as the logical possibility of there being poison in your sandwich is no reason for doubting you'll survive eating it.
The difference between our positions is that I begin from a position of natural trust toward the senses and toward reason as the only rational possibility and default. I have no choice but to reason well or to reason poorly. I recognize that my senses and my inferences can err, but it does not follow that they always err. Indeed, the very claim that they can err presumes I can tell when they do.
So, if my inferences lead me to a position that undermines their own coherence, then I must conclude that my inferences are wrong (including those that led me to adopt a certain interpretation of, say, scientific measurements).
> Because that is the best explanation for what I observe.
But if your explanation involves contradiction of what you observe, then that is not only not the best explanation, but no explanation at all! An explanation cannot deny the thing it seeks to explain. Thus, by denying the objective reality of what you perceive, you are barred from inferring that denial.
That seems squishy, as what constitutes "going far wrong" is not meaningful under skeptical assumptions.
A better stance is one of cognitive optimism that avoids the irrationality of skepticism. Skepticism is irrational, because it leads to incoherence, and because there is no rational warrant to categorically doubt the senses. For doubt to be rational, there must be a reason for it. To doubt without reason is not to be rational, but to be willful, and willful beliefs cannot be reasoned with; they are not the product of evidence or inference — and they certainly aren't self-evident — but rather the product of arbitrary choice. The logical possibility of living in the Matrix is no reason for doubting the senses, just as the logical possibility of there being poison in your sandwich is no reason for doubting you'll survive eating it.
The difference between our positions is that I begin from a position of natural trust toward the senses and toward reason as the only rational possibility and default. I have no choice but to reason well or to reason poorly. I recognize that my senses and my inferences can err, but it does not follow that they always err. Indeed, the very claim that they can err presumes I can tell when they do.
So, if my inferences lead me to a position that undermines their own coherence, then I must conclude that my inferences are wrong (including those that led me to adopt a certain interpretation of, say, scientific measurements).
> Because that is the best explanation for what I observe.
But if your explanation involves contradiction of what you observe, then that is not only not the best explanation, but no explanation at all! An explanation cannot deny the thing it seeks to explain. Thus, by denying the objective reality of what you perceive, you are barred from inferring that denial.
It can't be that different, either, or our senses would be of no practical use.
That's not true. What our senses perceive (classical reality) is an emergent phenomenon of the underlying metaphysical truth (quantum mechanics). Those two things are about as radically different as you can get. That's why the measurement problem is a thing. But that doesn't mean our senses are of no practical use.
“ There’s no free will and everything is determined.”
Objects without free will aren’t able to come to conclusions like this.
Objects without free will aren’t able to come to conclusions like this.
I'd like to believe that there is no such thing as free will, but I just can't decide.
Look into Chaos Theory - the universe is not deterministic, you're good.
Or for a tldr look for the three body problem or try to find a solution to a double pendulum!
Or for a tldr look for the three body problem or try to find a solution to a double pendulum!
To the contrary, here's a series of essays on the subject of evolutionary game theory, the incentives created by competition, and its consequences for human wellbeing:
https://www.lesswrong.com/s/kNANcHLNtJt5qeuSS
"Moloch hasn't won" is a lengthy critique of the argument you are making here.
https://www.lesswrong.com/s/kNANcHLNtJt5qeuSS
"Moloch hasn't won" is a lengthy critique of the argument you are making here.
That doesn't seem to be on point to me. I'm not talking about being "caught in bad equilibria". My assertion is that rationalism itself is not stable, that the (apparent) triumph of rationalism since the Enlightenment was a transient, not an equilibrium. And one of the reasons it was a transient is that self-styled rationalists believed (and apparently still believe) that rationalism will inevitably triumph because it is rational, because it is in more intimate contact with reality than religion and superstition. But this is wrong because it overlooks the fact that what triumphs in the long run is simply reproductive fitness. Being in contact with reality can be actively harmful to reproductive fitness if it leads you to, say, decide not to have kids because you are pessimistic about the future.
> it overlooks the fact that what triumphs in the long run is simply reproductive fitness.
Why can't that observation be taken into account? Isn't the entire point of the approach accounting for all inputs to the extent possible?
I think you are making invalid assumptions about the motivations or goals or internal state or etc of the actors which you are then conflating with the approach itself. That there are certain conditions under which the approach is not an optimal strategy does not imply that it is never competitive under any.
The observation is then that rationalism requires certain prerequisites before it can reliably out compete other approaches. That seems reasonable enough when you consider that a fruit fly is unlikely to be able to successfully employ higher level reasoning as a survival strategy.
Why can't that observation be taken into account? Isn't the entire point of the approach accounting for all inputs to the extent possible?
I think you are making invalid assumptions about the motivations or goals or internal state or etc of the actors which you are then conflating with the approach itself. That there are certain conditions under which the approach is not an optimal strategy does not imply that it is never competitive under any.
The observation is then that rationalism requires certain prerequisites before it can reliably out compete other approaches. That seems reasonable enough when you consider that a fruit fly is unlikely to be able to successfully employ higher level reasoning as a survival strategy.
> Why can't that observation be taken into account?
Of course it can be. I'm saying that AFAICT it generally isn't.
> rationalism requires certain prerequisites before it can reliably out compete other approaches
Yes. And one of those, IMHO, is explicit recognition that rationalism does not triumph simply because it is rational, and coming up with strategies to compensate. But the rationalist community seems too hung up on things like malicious AI and Roko's basilisk to put much effort into that.
Of course it can be. I'm saying that AFAICT it generally isn't.
> rationalism requires certain prerequisites before it can reliably out compete other approaches
Yes. And one of those, IMHO, is explicit recognition that rationalism does not triumph simply because it is rational, and coming up with strategies to compensate. But the rationalist community seems too hung up on things like malicious AI and Roko's basilisk to put much effort into that.
This argument proves too much. If rationalism can't "triumph" (presumably over other modes of thought) because evolution makes moral realism unobservable, then no epistemic framework will help you - does empirically observing the brutality of evolution lead to better results? Or perhaps we should hypothesise that it's brutal and then test that prediction against what we observe?
I'm sympathetic to the idea that we know nothing because of the reproductive impulse to avoid doing or thinking about things that led our ancestors to avoid procreation, but such a conclusion can't be total because otherwise it is self defeating because is is contingent on rationalist assumptions about the mind's capacity to model knowledge.
I'm sympathetic to the idea that we know nothing because of the reproductive impulse to avoid doing or thinking about things that led our ancestors to avoid procreation, but such a conclusion can't be total because otherwise it is self defeating because is is contingent on rationalist assumptions about the mind's capacity to model knowledge.
The point being made is that rationalism is a framework. Having a framework does not imply competent execution. At lower levels of competence other strategies win out. At higher levels of competence we expect rationalism to win out.
Even then that might not always be the case. Sometimes there are severe time or bandwidth or energy or other constraints that preclude carefully collecting data and thinking things through. In those cases a heuristic that is very obviously not derived from any sort of critical thought process might well be the winning strategy.
There will also be cases where the answer provided by the rational approach will be to conform to some other framework. For example where cult type ingroup dynamics are involved across a large portion of the population.
Even then that might not always be the case. Sometimes there are severe time or bandwidth or energy or other constraints that preclude carefully collecting data and thinking things through. In those cases a heuristic that is very obviously not derived from any sort of critical thought process might well be the winning strategy.
There will also be cases where the answer provided by the rational approach will be to conform to some other framework. For example where cult type ingroup dynamics are involved across a large portion of the population.
> Having a framework does not imply competent execution,
Exactly right. It is not rationalism per se that is the problem, it is the way that The Rationalists are implementing it, the things they are choosing to focus their attention on. They are worried about things like hostile AI and Roko's Basilisk when what they should be worried about is MAGA, because that is not being driven by rationalism, it is being driven by Christian nationalism. MAGA is busily (and openly!) undermining every last hint of rationalism in the U.S. government, but the Rationalist community seems oddly unconcerned with this. Many self-styled Rationalists are even Trump supporters.
Exactly right. It is not rationalism per se that is the problem, it is the way that The Rationalists are implementing it, the things they are choosing to focus their attention on. They are worried about things like hostile AI and Roko's Basilisk when what they should be worried about is MAGA, because that is not being driven by rationalism, it is being driven by Christian nationalism. MAGA is busily (and openly!) undermining every last hint of rationalism in the U.S. government, but the Rationalist community seems oddly unconcerned with this. Many self-styled Rationalists are even Trump supporters.
> Being in contact with reality can be actively harmful to reproductive fitness if it leads you to, say, decide not to have kids because you are pessimistic about the future.
The fact that you can write this sentence, consider it to be true, and yet still hold in your head the idea that the future might be bad but it's still important to have children suggests that "contact with reality" is not a curse.
The fact that you can write this sentence, consider it to be true, and yet still hold in your head the idea that the future might be bad but it's still important to have children suggests that "contact with reality" is not a curse.
You got Darwinism exactly backwards. Darwinism and nature do not select like an algorithm. There is no cost function in reality and no population selection and reproduction algorithm. What you're seeing is the illusion of selection due to selection bias.
If gender equality and intellectual achievement don't produce children, then that isn't "darwinism selecting rationality out". You can't expect the continued existence of finite lifespan organisms if there are no replacement organisms. Raising children is hard work. The people who believe in gender equality and intellectual achievement made the decision to not want more of themselves, particularly when their belief in gender equality entails not wanting male offspring. The alternative is essentially freeloading and expecting others, who do not share the beliefs, to produce children for you and also to teach them the "enlightened" belief of forcing "enlightened" beliefs onto others (note the circularity, the initial conditions are usually irrelevant and often just a fig leaf to perpetuate the status quo).
If gender equality and intellectual achievement don't produce children, then that isn't "darwinism selecting rationality out". You can't expect the continued existence of finite lifespan organisms if there are no replacement organisms. Raising children is hard work. The people who believe in gender equality and intellectual achievement made the decision to not want more of themselves, particularly when their belief in gender equality entails not wanting male offspring. The alternative is essentially freeloading and expecting others, who do not share the beliefs, to produce children for you and also to teach them the "enlightened" belief of forcing "enlightened" beliefs onto others (note the circularity, the initial conditions are usually irrelevant and often just a fig leaf to perpetuate the status quo).
> no population selection and reproduction algorithm
I never said there was. Darwin said it because he didn't know anything about genes, but that mistake was corrected by Dawkins.
> If gender equality and intellectual achievement don't produce children, then that isn't "darwinism selecting rationality out".
Why not?
> The people who believe in gender equality and intellectual achievement made the decision to not want more of themselves
That's the wrong way to look at it. Individuals are not the unit of reproduction. Genes are. Genes that build brains that want to have (and raise) children are more likely to propagate than genes that build brains that don't, all else being equial. So it is not rationality per se that is the problem -- rationality can provide a reproductive advantage because it lets you, for example, build technology. The problem is that non-rational brains can parasitically benefit from the phenotype of rational brains, at least for a while. But in the long run this is not a stable equilibrium.
I never said there was. Darwin said it because he didn't know anything about genes, but that mistake was corrected by Dawkins.
> If gender equality and intellectual achievement don't produce children, then that isn't "darwinism selecting rationality out".
Why not?
> The people who believe in gender equality and intellectual achievement made the decision to not want more of themselves
That's the wrong way to look at it. Individuals are not the unit of reproduction. Genes are. Genes that build brains that want to have (and raise) children are more likely to propagate than genes that build brains that don't, all else being equial. So it is not rationality per se that is the problem -- rationality can provide a reproductive advantage because it lets you, for example, build technology. The problem is that non-rational brains can parasitically benefit from the phenotype of rational brains, at least for a while. But in the long run this is not a stable equilibrium.
Couple of point to that, mainly why should one have to follow darwinian evolution just because we are a product of that. It's similar to the natural law argument against homosexuality, that unnatural sex is wrong. The argument against that is natural biology does not inform what is good or what we should do.
I'm sure you would be able to predict what a rationalise will say when you ask them what future they prefer: one where we maximises for the number of humans or one with fewer humans but better lives
I'm sure you would be able to predict what a rationalise will say when you ask them what future they prefer: one where we maximises for the number of humans or one with fewer humans but better lives
> why should one have to follow darwinian evolution
That depends on what you mean by "follow". You have to "follow" Darwinian evolution for the same reason you have to "follow", say, the law of gravity. That doesn't mean you can't build airplanes and spacecraft, but you still have to acknowledge and "follow" the law of gravity. You can't just glue feathers to your arms, jump off a cliff, and hope for the best. (Actually, rationalists aren't even gluing feathers to their arms. They are doing the equivalent of jumping off a cliff because they just don't believe gravity applies to them.)
[UPDATE]
> unnatural sex is wrong
The problem with that argument is that homosexuality is not unnatural. Many, many species have homosexual relations. Accounting for this is a little bit challenging, but the fact is undeniable.
https://en.wikipedia.org/wiki/Homosexual_behavior_in_animals
That depends on what you mean by "follow". You have to "follow" Darwinian evolution for the same reason you have to "follow", say, the law of gravity. That doesn't mean you can't build airplanes and spacecraft, but you still have to acknowledge and "follow" the law of gravity. You can't just glue feathers to your arms, jump off a cliff, and hope for the best. (Actually, rationalists aren't even gluing feathers to their arms. They are doing the equivalent of jumping off a cliff because they just don't believe gravity applies to them.)
[UPDATE]
> unnatural sex is wrong
The problem with that argument is that homosexuality is not unnatural. Many, many species have homosexual relations. Accounting for this is a little bit challenging, but the fact is undeniable.
https://en.wikipedia.org/wiki/Homosexual_behavior_in_animals
Pointing to other species can be futile since they also eat their young and are horrible partners :)
Well, yeah, the whole "against nature" argument is bogus to begin with. But I think a counter-argument is stronger if it can be made even while accepting the other side's premises.
An issue with this line of reasoning is that Darwinian evolution fails to accurately describe real evolutionary processes.
A counterexample is meiotic drive, where alleles disrupt the meiotic process in order to favour their own transmission, even if the alleles in question ultimately produce a less fit organism.
Whilst this is not an inherently positive observation, I think it does illustrate that the fatalistic picture you're painting here is incorrect. There's room for tentative optimism.
A counterexample is meiotic drive, where alleles disrupt the meiotic process in order to favour their own transmission, even if the alleles in question ultimately produce a less fit organism.
Whilst this is not an inherently positive observation, I think it does illustrate that the fatalistic picture you're painting here is incorrect. There's room for tentative optimism.
> Darwinian evolution fails to accurately describe real evolutionary processes.
That is not correct. Darwin did make a mistake, but it was not the fundamental dynamics of the process, but that he chose the wrong unit of selection. Darwin thought that selection selected for individuals or species when in fact it selects for genes. Richard Dawkins is the person who figured this out, but Darwin knew nothing about genes (OoS was published only three years after Gregor Mendel's work) so he still gets the credit nothwithsanding this mistake.
That is not correct. Darwin did make a mistake, but it was not the fundamental dynamics of the process, but that he chose the wrong unit of selection. Darwin thought that selection selected for individuals or species when in fact it selects for genes. Richard Dawkins is the person who figured this out, but Darwin knew nothing about genes (OoS was published only three years after Gregor Mendel's work) so he still gets the credit nothwithsanding this mistake.
I don't think you've understood my point. I'm not suggesting that selection doesn't occur, but rather that selection is only part of the story. Have a look at gene flow or genetic drift if you want to know more, they're both examples of non-selective processes that influence the genetic makeup of a population.
> selection is only part of the story
Of course. Variation is the other part.
> gene flow or genetic drift
Those are two mechanisms by which variation occurs. This is not something Darwin got wrong, he just didn't have all the data. Genes were unknown in Darwin's time.
Of course. Variation is the other part.
> gene flow or genetic drift
Those are two mechanisms by which variation occurs. This is not something Darwin got wrong, he just didn't have all the data. Genes were unknown in Darwin's time.
Which of selection or variation does meiotic drive fall under?
Clearly it isn't Darwinian selection, because it can favour transmission of genes which are not conducive to survival.
Clearly it isn't variation, because it tends to reduce the degree of genetic polymorphism within a population.
Clearly it isn't Darwinian selection, because it can favour transmission of genes which are not conducive to survival.
Clearly it isn't variation, because it tends to reduce the degree of genetic polymorphism within a population.
Meiotic drive is selection.
> it isn't Darwinian selection, because it can favour transmission of genes which are not conducive to survival
This is the one thing that Darwin got wrong: he thought that the unit of selection was the organism, but it's not. The unit of selection is the gene. Richard Dawkins is the one who figured this out. Meiotic drive is not Darinian selection, but it is selection. But Darwin could not possibly have known this because genes were unknown at the time.
> it isn't Darwinian selection, because it can favour transmission of genes which are not conducive to survival
This is the one thing that Darwin got wrong: he thought that the unit of selection was the organism, but it's not. The unit of selection is the gene. Richard Dawkins is the one who figured this out. Meiotic drive is not Darinian selection, but it is selection. But Darwin could not possibly have known this because genes were unknown at the time.
Darwinism isn't a weakness of rationality, teleology has fine tuning problem, while darwinism is minimally fine tuned to work from scratch, which can be said to be optimal. Also darwinism doesn't select for reproductive fitness, it's only a proxy goal; true goal is survival, so you can produce more offspring only in a way compatible with true goal.
> Darwinism isn't a weakness of rationality
I didn't say it was. I said that the Rationalist community is not taking the implications of Darwinism into account when they choose where to focus their attention. This is what leads them to fixate on hostile AI and the MWI when what they should be worried about is the rise of MAGA. But not only is that not what they are worried about, many self-styled Rationalists are Trump supporters.
I didn't say it was. I said that the Rationalist community is not taking the implications of Darwinism into account when they choose where to focus their attention. This is what leads them to fixate on hostile AI and the MWI when what they should be worried about is the rise of MAGA. But not only is that not what they are worried about, many self-styled Rationalists are Trump supporters.
> If you are a gene, you can probably produce more offspring in an already-industrialized environment by making brains that lean more towards misogyny and sexual promiscuity than gender equality and intellectual achievement.
The fact that humans are intelligent at all and Enlightenment peoples currently dominate the world suggests otherwise.
The fact that humans are intelligent at all and Enlightenment peoples currently dominate the world suggests otherwise.
> Enlightenment peoples currently dominate the world
Huh??? How do you figure? AFAICT the world is dominated by Donald Trump, Xi Jinping, and Vladimir Putin (if you reckon by power) or Christians and Muslims (if you reckon by population). None of these individuals or groups can be properly categorized as "Enlightenment peoples", certainly not with a capital E.
https://en.wikipedia.org/wiki/Age_of_Enlightenment
Huh??? How do you figure? AFAICT the world is dominated by Donald Trump, Xi Jinping, and Vladimir Putin (if you reckon by power) or Christians and Muslims (if you reckon by population). None of these individuals or groups can be properly categorized as "Enlightenment peoples", certainly not with a capital E.
https://en.wikipedia.org/wiki/Age_of_Enlightenment
Western nations have dominated the world for centuries, in science, technology and cultural dominance. Russia is a bit player and has been for some time. Even if China were to ascend to equal prominence as the US, it would be an outlier in an otherwise clear trend.
> Western nations have dominated the world for centuries
I guess that depends on what you consider "the world". It makes no sense to even talk about the West dominating "the world" before 1492. The first truly global Western empire was Britain, but it was also the last. It was replaced by the U.S. but it was never really global. Even at the height of its power after WW2 the USSR was a credible rival. After the fall of the USSR in 1991 the U.S. was the sole undisputed superpower for a little while, but that came to an abrupt end on September 11, 2001 and the subsequent wars in Afghanistan and Iraq.
I think you are over-extrapolating the past into the future. The mindset and culture that produced U.S. hegemony in the 20th century seems to me to be mostly extinct. The U.S. was indeed ruled by rationalism (more or less) from the time of its founding through the mid-to-late 20th century, but there is precious little of that left today. Certainly the power structure in the U.S. today is actively hostile to rationalism, and I don't see a whole lot of rationalism in play in the opposition either.
I guess that depends on what you consider "the world". It makes no sense to even talk about the West dominating "the world" before 1492. The first truly global Western empire was Britain, but it was also the last. It was replaced by the U.S. but it was never really global. Even at the height of its power after WW2 the USSR was a credible rival. After the fall of the USSR in 1991 the U.S. was the sole undisputed superpower for a little while, but that came to an abrupt end on September 11, 2001 and the subsequent wars in Afghanistan and Iraq.
I think you are over-extrapolating the past into the future. The mindset and culture that produced U.S. hegemony in the 20th century seems to me to be mostly extinct. The U.S. was indeed ruled by rationalism (more or less) from the time of its founding through the mid-to-late 20th century, but there is precious little of that left today. Certainly the power structure in the U.S. today is actively hostile to rationalism, and I don't see a whole lot of rationalism in play in the opposition either.
But rationalism , if the modern sort, isn't supposed to be descriptive, it's supposed to be normative -- rationality lets you win.
The problem is not with rationalism, it's with Rationalism (with a capital R), the cult-like phenomenon that has grown up around rationalism that fetishizes things like Bayes's theorem, hostile AI, Roko's basilisk, and the MWI.
I hesitate to nitpick, but Darwinism (as far as I know) is not really the term to use because Darwin's theory was limited to life on earth. Only later was the concept generalised into "natural selection" or "survival of the fittest".
I'm not sure I entirely understand what you're arguing here, but I absolutely do agree that the most powerful force in the universe is natural selection.
I'm not sure I entirely understand what you're arguing here, but I absolutely do agree that the most powerful force in the universe is natural selection.
The modern understanding of Darwin's theory (even the original theory, not necessarily neo-Darwinian extensions of it) apply to the origins of life and non-biological systems as well. Darwin himself was largely concerned with biology and restricted his published writings to that topic, but even he saw the application to the origin of life, and implications for religion. Even if he hadn't, we generally still use the discoverer's name to a theory even when applied to a domain outside their original area of concern.
The term "survival of the fittest" predates Darwin's Origin of Species, and was adopted by Darwin within his lifetime, btw.
The term "survival of the fittest" predates Darwin's Origin of Species, and was adopted by Darwin within his lifetime, btw.
The term "Darwinian evolution" applies to any process that comprises iterated replication with random mutation followed selection for some quality metric. Darwin himself would not have defined it that way, but he still deserves the credit for being the first to recognize and document the power of this simple process.
I used to think that massive, very long term droughts might cause serious instability, but I have since changed my mind. In highly developed nations, the amount of irrigation infrastructure built in the last 100 years is simple stunning. Plus, national agriculture research programmes are always researching how to use less water and grow the same amount of product. About drinking water: Rich places just build desalination plants. Sure, it is much more expensive than natural water sources (rivers, lakes, aquifers), but not expensive enough to cause political instability or serious economic harm. To be clear: Everything I wrote is from the perspective of highly developed nations. In middle income nations and below, droughts are incredibly challenging to overcome. The political and economic impacts can be enormous.
'America Is Using Up Its Groundwater Like There’s No Tomorrow' (https://waterwatch.org/america-is-using-up-its-groundwater-l...) to support that stunning irrigation infrastructure.
You should not interpret that historical success to imply future success as it depended on non-sustainable groundwater extraction.
Eg, https://en.wikipedia.org/wiki/Ogallala_Aquifer
> Many farmers in the Texas High Plains, which rely particularly on groundwater, are now turning away from irrigated agriculture as pumping costs have risen and as they have become aware of the hazards of overpumping.
> Sixty years of intensive farming using huge center-pivot irrigators has emptied parts of the High Plains Aquifer.
> as the water consumption efficiency of the center-pivot irrigator improved over the years, farmers chose to plant more intensively, irrigate more land, and grow thirstier crops rather than reduce water consumption--an example of the Jevons Paradox in practice
How will the Great Plains farmers get water once the remaining groundwater is too expensive to extract?
Salt Lake City cannot simply build desalination plants to fix its water problem.
I expect the bad experiences of Okies during the internal migration of the Dust Bowl will be replicated once the temporary (albeit century-long) relief of using fossil water is exhausted.
You should not interpret that historical success to imply future success as it depended on non-sustainable groundwater extraction.
Eg, https://en.wikipedia.org/wiki/Ogallala_Aquifer
> Many farmers in the Texas High Plains, which rely particularly on groundwater, are now turning away from irrigated agriculture as pumping costs have risen and as they have become aware of the hazards of overpumping.
> Sixty years of intensive farming using huge center-pivot irrigators has emptied parts of the High Plains Aquifer.
> as the water consumption efficiency of the center-pivot irrigator improved over the years, farmers chose to plant more intensively, irrigate more land, and grow thirstier crops rather than reduce water consumption--an example of the Jevons Paradox in practice
How will the Great Plains farmers get water once the remaining groundwater is too expensive to extract?
Salt Lake City cannot simply build desalination plants to fix its water problem.
I expect the bad experiences of Okies during the internal migration of the Dust Bowl will be replicated once the temporary (albeit century-long) relief of using fossil water is exhausted.
You raise an interesting point about inland population centers. My guess: Household daily water usage is a tiny fraction compared to Ag. As a result, at some point, I assume that states will slowly raise the cost of Ag water, until the most wasteful practices disappear. Example: They could move to drip irrigation that Israel uses, but it must at least 10x more expensive.
My limited understanding of the irrigation history in the US tells makes me extremely pessimistic about the likelihood of your prediction.
I think you only need to look at the water politics of the Great Salt Lake to see the difficulty.
Look at how little water use has changed during the last 25 years of the southwestern North American megadrought.
The US policy appears to be to pray for rain.
I think you only need to look at the water politics of the Great Salt Lake to see the difficulty.
Look at how little water use has changed during the last 25 years of the southwestern North American megadrought.
The US policy appears to be to pray for rain.
> Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority.
Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are.
Suppose you clock "malicious AI" as a huge risk and then hamper AI, but it turns out the bigger risk is not doing space exploration, which AI would have accelerated, because something catastrophic yet already-inevitable is going to happen to the Earth in a few hundred years and if we're not sustainably multi-planetary by then it's all over.
The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one.
Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are.
Suppose you clock "malicious AI" as a huge risk and then hamper AI, but it turns out the bigger risk is not doing space exploration, which AI would have accelerated, because something catastrophic yet already-inevitable is going to happen to the Earth in a few hundred years and if we're not sustainably multi-planetary by then it's all over.
The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one.
> Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are.
Rationalist community understands that very well. They even know how to put bounds on the unknowns and their own lack of information.
> The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one.
Right. Good thing they'd agree with you 100% on this.
Rationalist community understands that very well. They even know how to put bounds on the unknowns and their own lack of information.
> The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one.
Right. Good thing they'd agree with you 100% on this.
> They even know how to put bounds on the unknowns and their own lack of information.
No they don't. They think they can do this because they've accidentally reinvented the philosophy "logical positivism", which philosophers gave up on because it doesn't work. (This is similar to how they accidentally reinvented reconstructing arguments and called it "steelmanning".)
https://metarationality.com/probability-limitations
No they don't. They think they can do this because they've accidentally reinvented the philosophy "logical positivism", which philosophers gave up on because it doesn't work. (This is similar to how they accidentally reinvented reconstructing arguments and called it "steelmanning".)
https://metarationality.com/probability-limitations
The nature of unknowns is that you don't know them.
What's the probability of AI singularity? It has never happened before so you have no priors and any number you assign will be pure speculation.
What's the probability of AI singularity? It has never happened before so you have no priors and any number you assign will be pure speculation.
Same is true about anything you're trying to forecast, by definition of it being in the future. And yet people have figured out how to make predictions more narrow than shrugging.
"It is difficult to make predictions, especially about the future."
Most of the time we make predictions based on how similar events happened in the past. For completely novel situations it's close to impossible to make a prediction and reckless to base policy on such a prediction.
Most of the time we make predictions based on how similar events happened in the past. For completely novel situations it's close to impossible to make a prediction and reckless to base policy on such a prediction.
That's strictly true, but I feel like you're misunderstanding something. Most people aren't actually doing anything truly novel, hence very few people ever actually have to even attempt to predict things in this way.
But it was necessary at the beginning of flight and the flight to the moon would've never been possible either without a few talented people being able to make predictions about scenarios they knew little about.
There are just way too many people around nowadays, which is why most of us never get confronted with such novel topics and consequently we don't know how to reason about it
But it was necessary at the beginning of flight and the flight to the moon would've never been possible either without a few talented people being able to make predictions about scenarios they knew little about.
There are just way too many people around nowadays, which is why most of us never get confronted with such novel topics and consequently we don't know how to reason about it
>> It has never happened before
> Same is true about anything you're trying to forecast, by definition of it being in the future
There might be some flaws in this line of reasoning...
> Same is true about anything you're trying to forecast, by definition of it being in the future
There might be some flaws in this line of reasoning...
Yes, but making it more nuanced still doesn't change the point.
Singularity obviously never happened before, and if anyone bothered to read up on what they're talking about, they'd realize that no one is trying to predict what happens then, because the singularity is defined as the time when changes accelerate to such a degree that we have no baseline to make any predictions whatsoever.
So when people speculate on when that is, they're trying to forecast the point forecasting breaks; they do it by extrapolating from known examples and trends, to which we do have baselines.
Or, in short: we know how it is to ride an exponent, we just never rode one long enough to fall off of it; predicting singularity is predicting when the exponent gets steep enough we can't follow, which is not unlike predicting any other trend people do. Same methods and caveats apply.
Singularity obviously never happened before, and if anyone bothered to read up on what they're talking about, they'd realize that no one is trying to predict what happens then, because the singularity is defined as the time when changes accelerate to such a degree that we have no baseline to make any predictions whatsoever.
So when people speculate on when that is, they're trying to forecast the point forecasting breaks; they do it by extrapolating from known examples and trends, to which we do have baselines.
Or, in short: we know how it is to ride an exponent, we just never rode one long enough to fall off of it; predicting singularity is predicting when the exponent gets steep enough we can't follow, which is not unlike predicting any other trend people do. Same methods and caveats apply.
"And the general absolutist tone of the community. The people involved all seem very... Full of themselves ?"
>And yet people have figured out how to make predictions more narrow than shrugging
And?
>And yet people have figured out how to make predictions more narrow than shrugging
And?
That's only one flaw in the theory.
There are others, such as the unproven, narcissistic and frankly unlikely-to-be-true assumption that humanity continuing to exist is a net positive in the long run.
There are others, such as the unproven, narcissistic and frankly unlikely-to-be-true assumption that humanity continuing to exist is a net positive in the long run.
"net positive" requires a human being existing to judge.
> "net positive" requires a human being existing to judge.
This is effectively a religious belief you are espousing.
This is effectively a religious belief you are espousing.
What do you mean? Religion seems like a total nonsequitur here. If anything I would argue the opposite: you would need religion or something like it for moral statements to have any meaning without the existence of sentient beings.
A net positive for whom?
Everything. of which humanity is a miniscule part.
In what sense are people in those communities "quite smart"? Stupid is as stupid does. There are plenty of people who get good grades and score highly on standardized tests, but are in fact nothing but pontificating blowhards and useless wankers.
They're members of a religion which says that if you do math in your head the right way you'll be correct about everything, and so they think they're correct about everything.
They also secondarily believe everyone has an IQ which is their DBZ power level; they believe anything they see that has math in it, and IQ is math, so they believe anything they see about IQ. So if you avoid trying to find out your own IQ you can just believe it's really high and then you're good.
Unfortunately this lead them to the conclusion that computers have more IQ than them and so would automatically win any intellectual DBZ laser beam fight against them / enslave them / take over the world.
They also secondarily believe everyone has an IQ which is their DBZ power level; they believe anything they see that has math in it, and IQ is math, so they believe anything they see about IQ. So if you avoid trying to find out your own IQ you can just believe it's really high and then you're good.
Unfortunately this lead them to the conclusion that computers have more IQ than them and so would automatically win any intellectual DBZ laser beam fight against them / enslave them / take over the world.
If only I could +1 this more than once! I have learned valuable things occasionally from people in the rationalist community but this overall lack of humility —and strangely blinkered view of humanities and important topics like say history of science relevant to “STEM”—ultimately turned me off to the movement as a whole. And I love science and math! It just shouldn’t belong to people with this (imo) childish model of people, IQ, etc.
According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence. Meanwhile building a single giant super AI is technologically feasible, so they weigh the intelligence of a single person vs all AIs operating as a collective hivemind.
> According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence.
An actual argument would be that intelligence doesn't work like that. Two people with IQ 100 cooperating together does not produce an IQ 200 solution.
There is the "wisdom of crowds". If a random member of a group is more than 50% likely to be correct, the average of the group is more likely to be correct than its members individually. But that has a few assumptions, for example that each member tries to figure out things independently (as opposed to everyone waiting for the highest-status member to express their opinion, and then agreeing with it -- in that case the entire group is only as smart as the highest-status member).
But you cannot leverage this to simply invite 1000 random people in your group and ask them to invent a Theory of Everything; because the assumption that each member is more than 50% likely to be correct does not apply in this case. So that is one of the limits of people working together.
(And this already conveniently ignores many other problems found in real life, such as conflict of interests, etc.)
An actual argument would be that intelligence doesn't work like that. Two people with IQ 100 cooperating together does not produce an IQ 200 solution.
There is the "wisdom of crowds". If a random member of a group is more than 50% likely to be correct, the average of the group is more likely to be correct than its members individually. But that has a few assumptions, for example that each member tries to figure out things independently (as opposed to everyone waiting for the highest-status member to express their opinion, and then agreeing with it -- in that case the entire group is only as smart as the highest-status member).
But you cannot leverage this to simply invite 1000 random people in your group and ask them to invent a Theory of Everything; because the assumption that each member is more than 50% likely to be correct does not apply in this case. So that is one of the limits of people working together.
(And this already conveniently ignores many other problems found in real life, such as conflict of interests, etc.)
For anyone who takes Effective Altruism or Rationalism seriously I strongly recommend reading "You Are Not a Gadget" It was written more than ten years ago and was so prescient about the issues of social media and also contained one of the most devastating problems with EA, the idea of a circle of empathy.
You don't have to agree with any of this. I am not defending every idea the author has. But I recommend that book.
You don't have to agree with any of this. I am not defending every idea the author has. But I recommend that book.
Thanks for the reco.
> then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority.
This is the logic of someone who has failed to comprehend the core ideas of Calculus 101. You cannot use intuitive reasoning when it comes to infinite sums of numbers with extremely large uncertainties. All that results is making a fool out of yourself.
This is the logic of someone who has failed to comprehend the core ideas of Calculus 101. You cannot use intuitive reasoning when it comes to infinite sums of numbers with extremely large uncertainties. All that results is making a fool out of yourself.
This summarizes what irks me about the community.
They use technical terms (eg expected value, KL divergence) in their verbal reasoning only to sound rational, but don’t ever mean to use those terms technically.
They use technical terms (eg expected value, KL divergence) in their verbal reasoning only to sound rational, but don’t ever mean to use those terms technically.
Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with.
I get the feeling these people often want to seem smarter than they are, regardless of how smart they are. And they want to get money to ostensibly "consider these issues", but really they want money for nothing.
If they wanted to do right by the future masses, they should be looking to the things that are affecting us right now. But they treat those issues as if they'll work out in the wash.
I get the feeling these people often want to seem smarter than they are, regardless of how smart they are. And they want to get money to ostensibly "consider these issues", but really they want money for nothing.
If they wanted to do right by the future masses, they should be looking to the things that are affecting us right now. But they treat those issues as if they'll work out in the wash.
> Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with.
The current sums invested and donated in altruist causes are rounding errors themselves compared to GDPs of countries, so the revealed preferences of those investing and donating to altruist causes is to care about the future and the present also.
Are you saying that they should give a greater preference to help those who already exist rather than those who may exist in the future?
I see a lot of Peter Singer’s ideas in modern “effective” altruism, but I get the sense from your comment that you don’t think that they have good reasons for doing what they do, or that their reason leads them to support well-meaning but ineffective solutions. I am trying to understand your position without misrepresenting your point or goals. Are you naysaying or do you have an alternative?
https://en.wikipedia.org/wiki/Peter_Singer
The current sums invested and donated in altruist causes are rounding errors themselves compared to GDPs of countries, so the revealed preferences of those investing and donating to altruist causes is to care about the future and the present also.
Are you saying that they should give a greater preference to help those who already exist rather than those who may exist in the future?
I see a lot of Peter Singer’s ideas in modern “effective” altruism, but I get the sense from your comment that you don’t think that they have good reasons for doing what they do, or that their reason leads them to support well-meaning but ineffective solutions. I am trying to understand your position without misrepresenting your point or goals. Are you naysaying or do you have an alternative?
https://en.wikipedia.org/wiki/Peter_Singer
I think it's essentially a grift. An excuse to do nothing while looking like you care and reaping rewards.
If they wanted to help, they should be focused on the now. Global poverty, climate change, despotic world leaders. They should be aligning themselves against such things.
But instead what we see is essentially not that. Effective altruism is a lot like the Democratic People's Republic of Korea, a bit of a misnomer.
If they wanted to help, they should be focused on the now. Global poverty, climate change, despotic world leaders. They should be aligning themselves against such things.
But instead what we see is essentially not that. Effective altruism is a lot like the Democratic People's Republic of Korea, a bit of a misnomer.
They have aligned themselves in favor of global poverty, climate change and despotic world leaders.
A lot of them argue that poor countries essentially don't matter. Climate change is not an extinction event and there should an authoritarian world government to prevent nuclear conflict to minimize the risk of nuclear extinction.
>In his dissertation On the Overwhelming Importance of Shaping the Far Future (2013), supposedly “one of the best texts on existential risks,”[9] Nicholas Beckstead meditates on the “ripple effects” a human life might have for future generations and concludes “that saving a life in a rich country is substantially more important than saving a life in a poor country” due to the higher level of innovation and economic productivity attained in these countries.[10]
https://umbau.hfg-karlsruhe.de/posts/philosophy-against-the-...
A lot of them argue that poor countries essentially don't matter. Climate change is not an extinction event and there should an authoritarian world government to prevent nuclear conflict to minimize the risk of nuclear extinction.
>In his dissertation On the Overwhelming Importance of Shaping the Far Future (2013), supposedly “one of the best texts on existential risks,”[9] Nicholas Beckstead meditates on the “ripple effects” a human life might have for future generations and concludes “that saving a life in a rich country is substantially more important than saving a life in a poor country” due to the higher level of innovation and economic productivity attained in these countries.[10]
https://umbau.hfg-karlsruhe.de/posts/philosophy-against-the-...
The site you reference quotes Beckstead out of context, and either reading the context or looking at what he spent the next decade of his life working on would make it clear that he thinks the marginal dollar is better spent on saving lives in poor countries than rich ones. He well understands that his "other things being equal" in his dissertation essentially never holds in practice, and was writing for a philosophy audience where this kind of hypothetical is expected.
Meanwhile, on an actual EA website: https://www.givewell.org/charities/top-charities
Agreed.
To be pedantic, DPRK is run via the will of the people to a degree comparable to any country. A bigger misnomer is the west calling liberal “democracy”, just democracy.
To be pedantic, DPRK is run via the will of the people to a degree comparable to any country. A bigger misnomer is the west calling liberal “democracy”, just democracy.
Even if I understand what you mean, DPRK is very far away from democratic countries. It's not comparable at all.
[deleted]
My point is that most people who see that think a country like America or Europe are democracies and that is partly why it would be a misnomer. When it is arguable that DPRK state is more a will of the people than the west.
The elite letting the people choose between a few candidates is not a democracy. There are no “democratic” countries in that way.
The elite letting the people choose between a few candidates is not a democracy. There are no “democratic” countries in that way.
> When it is arguable that DPRK state is more a will of the people than the west.
It's not arguable, it's simply wrong. But to understand that you would have to understand much more about the DPRK.
> The elite letting the people choose between a few candidates is not a democracy.
It is, but more importantly, your framing is wrong. Every democracy has several levels of democratic institutions (local, state, federal etc.), there are often 'surprise' election winners; there are also non-elective ways for non-elite people to influence policy. DPRK has none of this.
It's not arguable, it's simply wrong. But to understand that you would have to understand much more about the DPRK.
> The elite letting the people choose between a few candidates is not a democracy.
It is, but more importantly, your framing is wrong. Every democracy has several levels of democratic institutions (local, state, federal etc.), there are often 'surprise' election winners; there are also non-elective ways for non-elite people to influence policy. DPRK has none of this.
> But to understand that you would have to understand much more about the DPRK.
I have never come across some one like you who knows about DPRK outside of what western state depts say and of course they swallow it all up. But people do sometimes lie and say they know so much like you right now
I have never come across some one like you who knows about DPRK outside of what western state depts say and of course they swallow it all up. But people do sometimes lie and say they know so much like you right now
Now you did come across someone like that, so you should update your assumptions. But more importantly study more and don't be a tankie.
“ When it is arguable that DPRK state is more a will of the people than the west.”
Ba da Ching
Ba da Ching
What do people like you use to prove the DPRK is auth, a dictatorship, and so bad while so smugly believing they have “freedom” and “democracy” better than the Global South? Western state depts and western state directed media manufacturing consent - ba da ching
Don’t forget — the enemies of the west are all bad and evil. And the west is free and fair and democratic and not a scourge on the rest of the world.
Don’t forget — the enemies of the west are all bad and evil. And the west is free and fair and democratic and not a scourge on the rest of the world.
Because when doctors go there to cure the blind from easily treatable malnutrition the people drop to the floor in fear praising the moon lord leader.
> But they treat those issues as if they'll work out in the wash.
there's something pathologically, virally sick about wealth accumulation's primary function being the further accumulation of wealth. for a movement rooted in "rationalism," EA seems pretty irrationally focused on excusing antisocial behavior.
there's something pathologically, virally sick about wealth accumulation's primary function being the further accumulation of wealth. for a movement rooted in "rationalism," EA seems pretty irrationally focused on excusing antisocial behavior.
> wealth accumulation's primary function being the further accumulation of wealth
Then we are lucky that EA promotes giving more to charity as the primary function of accumulation of wealth.
> EA seems pretty irrationally focused on excusing antisocial behavior.
Is a guy getting a well-paid job at Microsoft and donating half of his salary to African charities really your best example of antisocial behavior?
Then we are lucky that EA promotes giving more to charity as the primary function of accumulation of wealth.
> EA seems pretty irrationally focused on excusing antisocial behavior.
Is a guy getting a well-paid job at Microsoft and donating half of his salary to African charities really your best example of antisocial behavior?
What it promotes and what happens are two different things.
“The purpose of a system is what it does.”
“The purpose of a system is what it does.”
> Then we are lucky that EA promotes giving more to charity as the primary function of accumulation of wealth.
no, we are not lucky. EA-good-because-charity-good is a brain-pretzel way of lobbying against equitable taxation.
> Is a guy getting a well-paid job at Microsoft and donating half of his salary to African charities really your best example of antisocial behavior?
you're inventing a competitive debate regarding a hypothetical "best example of antisocial behavior". i didn't target anyone specifically with any part of my post.
no, we are not lucky. EA-good-because-charity-good is a brain-pretzel way of lobbying against equitable taxation.
> Is a guy getting a well-paid job at Microsoft and donating half of his salary to African charities really your best example of antisocial behavior?
you're inventing a competitive debate regarding a hypothetical "best example of antisocial behavior". i didn't target anyone specifically with any part of my post.
Well, I base my opinion about EA on specific people I happen to know, such as https://www.jefftk.com/news/ea , but of course that shouldn't stop anyone from making up edgy interpretations and posting them as a fact. Because what is the point of discussing actual effective altruists when imaginary villains are a much more interesting topic.
What's the seductive-but-wrong part of EA? As far as I can tell, the vast majority of the opposition to it boils down to "Maybe you shouldn't donate money to pet shelters when people are dying of preventable diseases" vs "But donating money to pet shelters feels better!"
That hasn't defined EA for at least 10 years or so. EA started with "you should donate to NGOs distributing mosquito nets instead of the local pet shelter".
It then moved into "you should work a soulless investment banking job so you can give more".
More recently it was "you should excise all expensive fun things from your life, and give 100% of your disposable income to a weird poly sex cult and/or their fraudulent paper hedge fund because they're smarter than you."
It then moved into "you should work a soulless investment banking job so you can give more".
More recently it was "you should excise all expensive fun things from your life, and give 100% of your disposable income to a weird poly sex cult and/or their fraudulent paper hedge fund because they're smarter than you."
[deleted]
This is what some EAs believe, I don't think there was ever a broad consensus on those latter claims. As such, it doesn't seem like a fair criticism of EA.
You can’t paint a broad brush of a whole movement, but it’s true if the leadership of the EA organization. Once they went all-in on “AI x-risk”, there ceased being a meaningful difference between them and the fringe of the LW ratsphere.
Which organization?
https://forum.effectivealtruism.org/posts/f6kg8T2Lp6rDqxWwG/...
https://forum.effectivealtruism.org/posts/f6kg8T2Lp6rDqxWwG/...
> That hasn't defined EA for at least 10 years or so.
Meanwhile, on an actual EA website: https://www.givewell.org/charities/top-charities
* Medicine to prevent malaria
* Nets to prevent malaria
* Supplements to prevent vitamin A deficiency
* Cash incentives for routine childhood vaccines
Meanwhile, on an actual EA website: https://www.givewell.org/charities/top-charities
* Medicine to prevent malaria
* Nets to prevent malaria
* Supplements to prevent vitamin A deficiency
* Cash incentives for routine childhood vaccines
GiveWell is an example of the short-termist end of EA. At the long-termist end people pay their friends to fantasize about Skynet at 'independent research institutes' like MIRI and Apollo Research. At the "trendy way to get rich people to donate" end you get buying a retreat center in Berkley, a stately home in England, and a castle in Czechia so Effective Altruists can relax and network.
Its important to know which type of EA organization you are supporting before you donate, because the movement includes all three.
Its important to know which type of EA organization you are supporting before you donate, because the movement includes all three.
I assume that GiveWell is the most popular of them. I mean, if you donate to MIRI, it is because you know about MIRI and because you specifically believe in their cause. But if you are just "hey, I have some money I want to donate, show me a list of effective charities", then GiveWell is that list.
(And I assume that GiveWell top charities receive orders of magnitude more money, but I haven't actually checked the numbers.)
(And I assume that GiveWell top charities receive orders of magnitude more money, but I haven't actually checked the numbers.)
Even GiveWell partnered with the long-termist/hypothetical risk type of EA by funding something called Open Philanthropy. And there are EA organizations which talk about "animal welfare" and mean "what if we replaced the biosphere with something where nothing with a spinal cord ever gets eaten?" So you can't trust "if it calls itself EA, it must be highly efficient at turning donations into measurable good." EA orgs have literally hired personal assistants and bought stately homes for the use of the people running the orgs!
It's not all bad, some conclusions are ok. But it also has ideas like "don't donate to charity, it's better to invest that money in, like, an oil fund, and grow it 1000x, and then you can donate so much more! Bill Gates has done much more for humanity than some Red Cross doctor!". Which is basically just a way to make yourself feel good about becoming richer, much like "prosperity gospel" would for the religious.
This is not a commonly-held position in EA.
It might be the parts that lead a person to commit large scale fraud with the idea that the good they can do with the stolen money outweighs all the negatives. Or, at least, that’s the popular idea of what happened to Sam Bankman-Fried. I have no idea what was actually going through that man’s mind.
In any case, EA smells strongly of “the ends justify the means” which most popular moral philosophies reject with strong arguments. One which resonates with me is that there are no “ends.” The path itself is the goal.
In any case, EA smells strongly of “the ends justify the means” which most popular moral philosophies reject with strong arguments. One which resonates with me is that there are no “ends.” The path itself is the goal.
> the ends justify the means” which most popular moral philosophies reject with strong arguments.
This is a false statement. Our entire modern world is built on the basis of the ends justify the means. Every time money is spent on long term infrastructure vs giving poor kids food right now, every time a war is fought, every time a doctor triages injuries at a disaster.
This is a false statement. Our entire modern world is built on the basis of the ends justify the means. Every time money is spent on long term infrastructure vs giving poor kids food right now, every time a war is fought, every time a doctor triages injuries at a disaster.
I don't think it's useful to conflate "the ends justify the means" with "cost-benefit analysis". You sometimes use the latter to justify certain means, but you don't have to, that's why they're different. When you believe that the ends justify the means, you can also just give no consideration at all to the ethics of the means. No doctor triaging patients would ever shoot a patient in the head so he could move onto one they thought was more important. Yes they might let patients die, but that's different than actively killing them.
In the framework of modern civilian medicine sure.
I'm sure exactly what you described was done plenty of times in ww1 and similar around that era, and seen as perfectly moral and rational.
I'm sure exactly what you described was done plenty of times in ww1 and similar around that era, and seen as perfectly moral and rational.
I've noticed this with a lot of radical streamers on both sides. They don't care about principles, they care about "winning" by any means necessary.
Winning at things that align with your principle is a principle. If you don't care about principles, you don't care about what you're winning at, thereby making every victory hollow and meaningless. That is how you turn into a loser at everything you do.
Winning at things that align with your principle is a principle. If you don't care about principles, you don't care about what you're winning at, thereby making every victory hollow and meaningless. That is how you turn into a loser at everything you do.
> there are no “ends.” The path itself is the goal.
How does this apply to actual charity? "Curing malaria is not the goal. Our experiences during voluntourism are the true goal."
How does this apply to actual charity? "Curing malaria is not the goal. Our experiences during voluntourism are the true goal."
Yes, but they’re smug about it, which means they’re wrong.
Of course it sounds ridiculous when you spell it out this way.
Of course it sounds ridiculous when you spell it out this way.
Or something like focusing on high paying job and donating money does absolutely nothing to solve any root or structural problems which defeats the supposed purpose of caring about future people or being effective altruist in the first place
Of course the way your comment is written makes criticism sound silly.
Of course the way your comment is written makes criticism sound silly.
Pet shelters is just the pretty facade. The same ideology would have you step over homeless elderly ladies and buy mosquito nets instead, because the elderly lady cannot have children so mosquito nets maximize the number of future humans more efficiently.
This tracks based on my limited contact with LessWrong during the whole Roko's Basilisk thing.
I quickly lost interest in Roko's Basilisk, but that is what brought me in the door and started me looking around the discussions. At first, it was quite seductive. There was a strange fearlessness there, a willingness to say and admit some things about humanity, our limitations and how we tend to think that other great thinkers maybe danced around in the past. After awhile it became clear that while there were a select few individuals who had found some balance between purely rational thinking and how reality actually works, most of the rest had their heads so far up their asses that they'd fart and call it a cool breeze. Reminded me of my brief obsession with Game Theory and realizing that even it's creators knew it's utility was not quite as advertised to the layman (as in it would not really help you predict or plan for anything at all, just model how decisions might be made).
I quickly lost interest in Roko's Basilisk, but that is what brought me in the door and started me looking around the discussions. At first, it was quite seductive. There was a strange fearlessness there, a willingness to say and admit some things about humanity, our limitations and how we tend to think that other great thinkers maybe danced around in the past. After awhile it became clear that while there were a select few individuals who had found some balance between purely rational thinking and how reality actually works, most of the rest had their heads so far up their asses that they'd fart and call it a cool breeze. Reminded me of my brief obsession with Game Theory and realizing that even it's creators knew it's utility was not quite as advertised to the layman (as in it would not really help you predict or plan for anything at all, just model how decisions might be made).
Peardon the "it's" autocorrect, I can't seem to edit on mobile (Harmonic)
Just musing out loud. "Utility" is like, for
Physics postgrads: "gauge"
Physics undergrads: "wavefunction"
Grade schoolers: "temperature"
These concepts definitely useful for the hw sets, no understanding needed (or expected!)
Physics postgrads: "gauge"
Physics undergrads: "wavefunction"
Grade schoolers: "temperature"
These concepts definitely useful for the hw sets, no understanding needed (or expected!)
I'm not familiar with any of these communities. Is there also a general bias towards one side between "the most important thing gets the *most* resources" and "the most important thing gets *all* the resources"? Or, in other words, the most important thing is the only important thing?
IMO it's fine to pick a favorite and devote extra resources to it. But that turns less fine when one also starts working to deprive everything else of any oxygen because it's not your favorite. (And I'm aware that this criticism applies to lots of communities.)
IMO it's fine to pick a favorite and devote extra resources to it. But that turns less fine when one also starts working to deprive everything else of any oxygen because it's not your favorite. (And I'm aware that this criticism applies to lots of communities.)
It's not the case. Effective altruists give to dozens of different causes, such as malaria prevention, environmentalism, animal welfare, and (perhaps most controversially) extinction risk. It can't tell you which root values to care about. It just asks you to consider whether the charity is impactful.
Even if an individual person chooses to direct all their donations to a single cause, there's no way to get everyone to donate to a single cause (nor is EA attempting to). Money gets spread around because people have different values.
It absolutely does take some money away from other causes, but only in the sense that all charities do: if you give a lot to one charity, you may have less money to give to others.
Even if an individual person chooses to direct all their donations to a single cause, there's no way to get everyone to donate to a single cause (nor is EA attempting to). Money gets spread around because people have different values.
It absolutely does take some money away from other causes, but only in the sense that all charities do: if you give a lot to one charity, you may have less money to give to others.
The general idea is that on the margin (in the economics sense), more resources should go to the most effective+neglected thing, and.the amount of resources I control is approximately zero in a global sense, so I personally should direct all of my personal giving to the highest impact thing.
And in their logic the highest impact is to donate money, take high paying jobs regardless of morality, and not focusing on any structural or root issues.
Yeah, the logic is basically "sure there are lots of structural or root issues, but I'm not confident I can make a substantial positive impact on those with the resources I have whereas I am confident that spending money to prevent people (mostly kids who would otherwise have survived to adulthood) from dying of malaria is a substantial positive impact at ~$5000 / life saved". I find that argument compelling, though I know many don't. Those many are free to focus on structural or root issues, or to try to make the case that addressing those issues is not just good, but better than reducing the impact of malaria.
The other weird direction it leads is space travel.
If you assume we eventually figure out long distance space travel and humanity spreads across the galaxy, there could in the future be quadrillions of people, growing at some kind of exponential rate. So accelerating the space race by even an hour is equivalent to bringing billions of new souls into existence.
If you assume we eventually figure out long distance space travel and humanity spreads across the galaxy, there could in the future be quadrillions of people, growing at some kind of exponential rate. So accelerating the space race by even an hour is equivalent to bringing billions of new souls into existence.
I don't see how bringing new souls (whatever those are) into existence should naturally qualify as a good thing?
Perhaps you're arguing as an illustration of the way this group of people think, in which case I understand your point.
Perhaps you're arguing as an illustration of the way this group of people think, in which case I understand your point.
You can make arguments for it starting with "killing billions of people would definitely be bad" or "some fraction of those people will likely share my genes, which I have a biological drive to pass on".
It encodes a slight bias towards human existence being a positive thing for us humans, but I don't think it's the shakiest part of that reasoning.
It encodes a slight bias towards human existence being a positive thing for us humans, but I don't think it's the shakiest part of that reasoning.
Nitpick: it bottoms out at quadratic growth in the limit, not exponential.
I think most everyone can agree with this: Being 100% rigorous and rational, reasoning from first principles and completely discarding received wisdom is a great trait in a philosopher but a terrible trait in a policymaker. Because for the former, exploring ideas for the benefit of future generations is more important than whether they ultimately reach the right conclusion or not.
> Being 100% rigorous and rational, reasoning from first principles
It really annoys me when people say that those religious cultists do that.
They derive their bullshit from faulty, poorly thought out premises.
If you fuck up the very firsts calculations of the algorithm, it doesn't matter how rigorous all the subsequent steps are. There results are going to be all wrong.
It really annoys me when people say that those religious cultists do that.
They derive their bullshit from faulty, poorly thought out premises.
If you fuck up the very firsts calculations of the algorithm, it doesn't matter how rigorous all the subsequent steps are. There results are going to be all wrong.
The big problem is that human values are similar to neural network weights that can't be clearly defined into true/false axioms like "human life is inherently valuable" and "murder is wrong". The easiest axiom to break is something like "every human life is equally valuable", or even "every human life is born equal".
[deleted]
It seems self-evident to me that epistemic humility _requires_ temporal discounting. We should not be confident we can predict the future well enough to integrate utility centuries forward.
EA always rubbed me the wrong way.
(1) The kind of Gatesian solutions they like to fund like mosquito nets are part of the problem, not part of the solution as I see it. If things are going to get better in Africa, it will be because Africans grow their economy and pay taxes and their governments can provide the services that they want. Expecting NGOs to do everything for them is the same kind of neoliberal thinking that has rotted state capacity in the core and set us up for a political crisis.
(2) It is one thing to do something wrong, realize it was a mistake, and then make amends. It's another thing to do plan to do something wrong and to try to offset it somehow. Many of the high paying jobs that EA wants young people to enter are "part of the problem" when it comes to declining stage capacity, legitimation crisis, and not dealing with immediate problems -- like the fact that one of these days there's going to be a heat wave that is a mass causality event.
Furthermore
(3) Time discounting is a central part of economic planning
https://en.wikipedia.org/wiki/Social_discount_rate
It is controversial as hell, but one of the many things the Soviet Union got wrong before the 1980s was planning with a discount rate of zero, which led to many economically and ecologically harmful projects. If you seriously think it should be zero you should also be considering whether anybody should work in the finance industry at all or if we should have dropped a hydrogen bomb on Exxon's headquarters yesterday. At some point speculations about the future are just speculation. When it comes to the nuclear waste issue, for instance, I don't think we have any idea what state people are going to be in 20,000 years. They might be really pissed that buried spent nuclear fuel some place they can't get at it. Even the plan to burn plutonium completely in fast breeder reactors has an air of unreality about it, even though it happens on a relatively short 1000 year timescale we can't be sure at all that anyone will be around to finish the job.
(4) If you are looking for low-probability events to worry about I think you could find a lot of them. If it was really a movement of free thinkers they'd be concerned about 4,000 horsemen of the apocalypse, not the 4 or so that they are allowed to talk about -- but talk about a bunch of people who'll cancel you if you "think different". Somehow climate change and legitimation crisis just get... ignored.
(5) Although it is run by people who say they are militant atheists, the movement has all the trappings of a religion, not least "The Singularity" was talked about by Jesuit Priest Teilhard de Chardin long before sci-fi writer Vernor Vinge used it as the hinge of a mystery novel.
(1) The kind of Gatesian solutions they like to fund like mosquito nets are part of the problem, not part of the solution as I see it. If things are going to get better in Africa, it will be because Africans grow their economy and pay taxes and their governments can provide the services that they want. Expecting NGOs to do everything for them is the same kind of neoliberal thinking that has rotted state capacity in the core and set us up for a political crisis.
(2) It is one thing to do something wrong, realize it was a mistake, and then make amends. It's another thing to do plan to do something wrong and to try to offset it somehow. Many of the high paying jobs that EA wants young people to enter are "part of the problem" when it comes to declining stage capacity, legitimation crisis, and not dealing with immediate problems -- like the fact that one of these days there's going to be a heat wave that is a mass causality event.
Furthermore
(3) Time discounting is a central part of economic planning
https://en.wikipedia.org/wiki/Social_discount_rate
It is controversial as hell, but one of the many things the Soviet Union got wrong before the 1980s was planning with a discount rate of zero, which led to many economically and ecologically harmful projects. If you seriously think it should be zero you should also be considering whether anybody should work in the finance industry at all or if we should have dropped a hydrogen bomb on Exxon's headquarters yesterday. At some point speculations about the future are just speculation. When it comes to the nuclear waste issue, for instance, I don't think we have any idea what state people are going to be in 20,000 years. They might be really pissed that buried spent nuclear fuel some place they can't get at it. Even the plan to burn plutonium completely in fast breeder reactors has an air of unreality about it, even though it happens on a relatively short 1000 year timescale we can't be sure at all that anyone will be around to finish the job.
(4) If you are looking for low-probability events to worry about I think you could find a lot of them. If it was really a movement of free thinkers they'd be concerned about 4,000 horsemen of the apocalypse, not the 4 or so that they are allowed to talk about -- but talk about a bunch of people who'll cancel you if you "think different". Somehow climate change and legitimation crisis just get... ignored.
(5) Although it is run by people who say they are militant atheists, the movement has all the trappings of a religion, not least "The Singularity" was talked about by Jesuit Priest Teilhard de Chardin long before sci-fi writer Vernor Vinge used it as the hinge of a mystery novel.
> When it comes to the nuclear waste issue, for instance ...
Nuclear waste issues are 99.9% present-day political/ideological. Huge portions of the Earth are uninhabitable due to climate and/or geology. Lead, mercury, arsenic, and other naturally-occurring poisons contaminate large areas. Volcanoes spew CO2 and toxic gasses by the megaton.
Vs. when is the last time you heard someone get excited over toxic waste left behind by the Roman Empire?
Nuclear waste issues are 99.9% present-day political/ideological. Huge portions of the Earth are uninhabitable due to climate and/or geology. Lead, mercury, arsenic, and other naturally-occurring poisons contaminate large areas. Volcanoes spew CO2 and toxic gasses by the megaton.
Vs. when is the last time you heard someone get excited over toxic waste left behind by the Roman Empire?
I agree with your lead (the issue is 99% political) but man does that last bit demand a rebuttal. Waste left behind by the roman empire isn't even remotely comparable to long term radioactive material. I suggest having a look through the wikipedia list of orphan source incidents to get an idea of what happens when people unknowingly come across radioactive material. https://en.wikipedia.org/wiki/List_of_orphan_source_incident...
That's an interesting list - but the injury & death rates are several orders of magnitude below even lightning strikes. https://en.wikipedia.org/wiki/Lightning_injury#Epidemiology
Also, PaulHoule's original comment said "in 20,000 years". Cobalt 60 (for example) has a half-life of 5 1/4 years - so there really won't be any of it left by then.
Also, PaulHoule's original comment said "in 20,000 years". Cobalt 60 (for example) has a half-life of 5 1/4 years - so there really won't be any of it left by then.
What's the rate conditioned on being near an incident though? And these are small, isolated incidents. How does what we see extrapolate to large scale nuclear waste storage, a state that failed a few hundred years ago, and someone inadvertently digging it up?
No one is talking about stuffing cobalt 60 in yucca mountain (at least as far as I know).
No one is talking about stuffing cobalt 60 in yucca mountain (at least as far as I know).
Compared to abandoned/forgotten mines (that eventually cave in) and mega-scale chemical waste dumps/sites/spills, nuclear waste sites - especially ones that'll still be seriously dangerous centuries or millennia from now - are profoundly rare.
And the tech to detect that you're digging into radioactive stuff is far simpler than the tech to detect that you're digging into some sort of chemical waste, or a failing old mine or tunnel.
If millennia-in-the-future humans care all that much about what we did with our nuclear waste, it'll either be political/ideological, or (as PaulHoule suggested) just one more "they didn't leave it somewhere really convenient for us" deal.
And the tech to detect that you're digging into radioactive stuff is far simpler than the tech to detect that you're digging into some sort of chemical waste, or a failing old mine or tunnel.
If millennia-in-the-future humans care all that much about what we did with our nuclear waste, it'll either be political/ideological, or (as PaulHoule suggested) just one more "they didn't leave it somewhere really convenient for us" deal.
I think of the widespread heavy metal contamination in China which existed even before the modern age of manufacturing. Some might be naturally occurring but you had people like this guy
https://en.wikipedia.org/wiki/Mausoleum_of_Qin_Shi_Huang
https://en.wikipedia.org/wiki/Mausoleum_of_Qin_Shi_Huang
>when is the last time you heard someone get excited over toxic waste left behind by the Roman Empire?
For archeologists, pretty much every time.
For archeologists, pretty much every time.
(3) "Controversial" is a weasel word AFAIC :)
The difficulty is in deriving any useful utility function from prices (even via preferences :), and as you know, econs can't rid themselves of that particular intrusive thought
https://mitsloan.mit.edu/sites/default/files/inline-files/So...
E: know any econs taking Habermas seriously ? Not a rhetorical q:
http://ecoport.org/storedReference/558800.pdf
The difficulty is in deriving any useful utility function from prices (even via preferences :), and as you know, econs can't rid themselves of that particular intrusive thought
https://mitsloan.mit.edu/sites/default/files/inline-files/So...
E: know any econs taking Habermas seriously ? Not a rhetorical q:
http://ecoport.org/storedReference/558800.pdf
What makes it really bad is that you can't add different people's utility functions, or for that matter, multiply the utility function of some imagined galactic citizen by some astronomical amount. The question of "what distribution of wealth maximizes welfare" [1] is unanswerable in that framework and we're left with the Randian maxim that any transaction freely entered into is "fair" because nobody would enter into it if it didn't increase their utility function.
[1] Though you might come to the conclusion that greeder people should have the money because they like it more
[1] Though you might come to the conclusion that greeder people should have the money because they like it more
Comic sophism ain't gonna make hypernormies reconsider lol
(Aside from the semi-tragic one to consider additive dilogarithms..)
One actionable (utility agnostic) suggestion: study the measureable consequences of (quantifiable) policy on carbon pricing, because this is already quite close to the uncontroversial bits
(Aside from the semi-tragic one to consider additive dilogarithms..)
One actionable (utility agnostic) suggestion: study the measureable consequences of (quantifiable) policy on carbon pricing, because this is already quite close to the uncontroversial bits
E.g. precisely how inflationary can we make carbon credits?
E: by "uncontroversial", I meant amongst the Orthodox econs, so not Graeber & sympathetic heterodox.
E: by "uncontroversial", I meant amongst the Orthodox econs, so not Graeber & sympathetic heterodox.
> "The Singularity" was talked about by Jesuit Priest Teilhard de Chardin long before sci-fi writer Vernor Vinge used it as the hinge of a mystery novel.
Similarly, Big Bang was talked about by Catholic priest Georges Lemaître, and Bayes' Theorem was invented by Presbyterian minister Thomas Bayes. Does that prove anything beyond the fact that there are many smart religious people?
Similarly, Big Bang was talked about by Catholic priest Georges Lemaître, and Bayes' Theorem was invented by Presbyterian minister Thomas Bayes. Does that prove anything beyond the fact that there are many smart religious people?
> If things are going to get better in Africa, it will be because Africans grow their economy and pay taxes and their governments can provide the services that they want
The problem with this argument is that the path to achieve this is unclear, and everyone who has tried has failed. In the absence of a clear path, it seems rational to set aside lofty ideals and do whatever good you can now.
> If you are looking for low-probability events to worry about I think you could find a lot of them.
Name one that hasn't already been considered that would be a serious threat to modern technological civilization.
> Although it is run by people who say they are militant atheists, the movement has all the trappings of a religion
Supposing you're correct this thought is incomplete. Take it to its logical conclusion: a religion centered around open rational debate is bad because...?
The problem with this argument is that the path to achieve this is unclear, and everyone who has tried has failed. In the absence of a clear path, it seems rational to set aside lofty ideals and do whatever good you can now.
> If you are looking for low-probability events to worry about I think you could find a lot of them.
Name one that hasn't already been considered that would be a serious threat to modern technological civilization.
> Although it is run by people who say they are militant atheists, the movement has all the trappings of a religion
Supposing you're correct this thought is incomplete. Take it to its logical conclusion: a religion centered around open rational debate is bad because...?
>People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.
They don't even do this.
If you're reasoning in purely logical and deductive way - it's blatantly obvious that living beings experience way more pain and suffering, than pleasure and joy. If you do the math, humanity getting wiped out in effect is the best thing that could happen.
Which is why accelerationism ignoring all the AGI risks is correct strategy presuming the AGI will either wipe us out (good outcome) or provide technologies that improve the human condition and reduce suffering (good outcome).
Logical and deductive reasoning based on completely baseless and obviously incorrect premises is flat out idiotic.
You can't deprive non-existent people out of anything.
And if you do, I hope you're ready for purely logical, deductive follow up - every droplet of sperm is sacred and should be used to impregnate.
They don't even do this.
If you're reasoning in purely logical and deductive way - it's blatantly obvious that living beings experience way more pain and suffering, than pleasure and joy. If you do the math, humanity getting wiped out in effect is the best thing that could happen.
Which is why accelerationism ignoring all the AGI risks is correct strategy presuming the AGI will either wipe us out (good outcome) or provide technologies that improve the human condition and reduce suffering (good outcome).
Logical and deductive reasoning based on completely baseless and obviously incorrect premises is flat out idiotic.
You can't deprive non-existent people out of anything.
And if you do, I hope you're ready for purely logical, deductive follow up - every droplet of sperm is sacred and should be used to impregnate.
I read the whole tree of responses under this comment and I could only convince myself that when people have no arguments they try to make you look bad.
Most of criticisms are just "But they think they are better than us !" and the rest is "But sometimes they are wrong !"
I don't know about the community and couldn't care less but their writings have brought me some almost life saving fresh air in how to think about the world. It is very sad to me to read so many falsely elaborate responses from supposedly intelligent people having their ego hurt but in the end it reminds me why I like rationalists and I don't like most people.
Most of criticisms are just "But they think they are better than us !" and the rest is "But sometimes they are wrong !"
I don't know about the community and couldn't care less but their writings have brought me some almost life saving fresh air in how to think about the world. It is very sad to me to read so many falsely elaborate responses from supposedly intelligent people having their ego hurt but in the end it reminds me why I like rationalists and I don't like most people.
Ever had a conversation with someone who literally cannot reexamine their base principles? Like, it's functionally impossible for them? That's everyone's central criticism of rationalists.
Being able to do that is pretty much "entry level cognition" for a lot of us. You should be doing that yourself and doing it all the time if you want to play with the big kids.
One of the things I really miss about the old nerds-only programmer's pit setup was the amount of room we had for instruction, especially regarding social issues. The scenes from the college department in Wargames were really on the nose, but highlight a form of education that was unavoidable if you couldn't just dip out of a conversation.
Being able to do that is pretty much "entry level cognition" for a lot of us. You should be doing that yourself and doing it all the time if you want to play with the big kids.
One of the things I really miss about the old nerds-only programmer's pit setup was the amount of room we had for instruction, especially regarding social issues. The scenes from the college department in Wargames were really on the nose, but highlight a form of education that was unavoidable if you couldn't just dip out of a conversation.
What a deeply confused criticism of rationalists, who are possibly the only meaningful social group in existence to celebrate changing their minds in response to new evidence.
It's truly fascinating how this thread is full of demonstrations of the exact phenomenon being discussed!
Accused of being unable to reexamine your base principles, you respond:
> rationalists, who are possibly the only meaningful social group in existence to celebrate changing their minds in response to new evidence
Which is exactly the kind of base principle that could use some reexamination.
Accused of being unable to reexamine your base principles, you respond:
> rationalists, who are possibly the only meaningful social group in existence to celebrate changing their minds in response to new evidence
Which is exactly the kind of base principle that could use some reexamination.
That's an empirical claim, not a "base principle" (whatever that is). And, uh, yeah, I agree that I am not re-examining my beliefs in response to a bunch of random unsupported accusations by people who are demonstrably not familiar with the thing they're attacking. You should update your beliefs in response to evidence, not in response to social attacks.
Again, this is exactly what we're talking about. You expressed a belief about "rationalists" and the history of people changing their minds about things, I suggested maybe you should reexamine that belief, and you said nope absolutely not. (In a condescending way.) This is the kind of engagement that is being critiqued in this thread.
Feels like "they are wrong and smug" is enough reason to dislike the movement
Bashir: Even when they're neither?
Garak: Especially when they're neither.
Garak: Especially when they're neither.
The comment I replied to conceded wrongness and smugness but is still somehow confused why otherwise intelligent ppl dislike the movement. I was hoping to clear it up for them
Extra points for that comment's author implying that people who don't like the wrong and smug movement are unintelligent and protecting their egos, thus personally proving its smugness
Extra points for that comment's author implying that people who don't like the wrong and smug movement are unintelligent and protecting their egos, thus personally proving its smugness
I only conceded it insofar as everyone can be wrong sometimes, but at least those people seem to have a protocol to deal with it.Most people don't, and are fine with it. I stand on the side of those who care and are trying.
As for smugness, it is subjective. Are those people smug ? Or are they talking passionately about some issue with the confidence of someone who feel what they are talking about and are expecting for it to resonate ? It's the eye of the beholder I guess.
For example what you call my smugness is what I would a slightly depressed attitude fueled by the fact that it's sometimes hard to relate to other people feelings and behavior.
As for smugness, it is subjective. Are those people smug ? Or are they talking passionately about some issue with the confidence of someone who feel what they are talking about and are expecting for it to resonate ? It's the eye of the beholder I guess.
For example what you call my smugness is what I would a slightly depressed attitude fueled by the fact that it's sometimes hard to relate to other people feelings and behavior.
Or maybe you are demonstrating the exact failure to demonstrate introspective humility that people are talking about?
[deleted]
Here's a theory of what's happening, both with you here in this comment section and with the rationalists in general.
Humans are generally better at perceiving threats than they are at putting those threats into words. When something seems "dangerous" abstractly, they will come up with words for why---but those words don't necessarily reflect the actual threat, because the threat might be hard to describe. Nevertheless the valence of their response reflects their actual emotion on the subject.
In this case: the rationalist philosophy basically creeps people out. There is something "insidious" about it. And this is not a delusion on the part of the people judging them: it really does threaten them, and likely for good reason. The explanation is something like "we extrapolate from the way that rationalists think and realize that their philosophy leads to dangerous conclusions." Some of these conclusions have already been made by the rationalists---like valuing people far away abstractly over people next door, by trying to quantify suffering and altruism like a math problem (or to place moral weight on animals over humans, or people in the future over people today). Other conclusions are just implied, waiting to be made later. But the human mind detects them anyway as implications of the way of thinking, and reacts accordingly: thinking like this is dangerous and should be argued against.
This extrapolation is hard to put into words, so everyone who tries to express their discomfort misses the target somewhat, and then, if you are the sort of person who only takes things literally, it sounds like they are all just attacking someone out of judgment or bitterness or something instead of for real reasons. But I can't emphasize this enough: their emotions are real, they're just failing to put them into words effectively. It's a skill issue. You will understand what's happening better if you understand that this is what's going on and then try to take their emotions seriously even if they are not communicating them very well.
So that's what's going on here. But I think I can also do a decent job of describing the actual problem that people have with the rationalist mindset. It's something like this:
Humans have an innate moral intuition that "personal" morality, the kind that takes care of themselves and their family and friends and community, is supposed to be sacrosanct: people are supposed to both practice it and protect the necessity of practicing it. We simply can't trust the world to be a safe place if people don't think of looking out for the people around them as a fundamental moral duty. And once those people are safe, protecting more people, such as a tribe or a nation or all of humanity or all of the planet, becomes permissible.
Sometimes people don't or can't practice this protection for various reasons, and that's morally fine, because it's a local problem that can be solved locally. But it's very insidious to turn around and justify not practicing it as a better way to live: "actually it's better not to behave morally; it's better to allocate resources to people far away; it's better to dedicate ourselves to fighting nebulous threats like AI safety or other X-risks instead of our neighbors; or, it's better to protect animals than people, because there are more of them". It's fine to work on important far-away problems once local problems are solved, if that's what you want. But it can't take priority, regardless of how the math works out. To work on global numbers-game problems instead of local problems, and to justify that with arguments, and to try to convince other people to also do that---that's dangerous as hell. It proves too much: it argues that humans at large ought to dismantle their personal moralities in favor of processing the world like a paperclip-maximizing robot. And that is exactly as dangerous as a paperclip-maximizing robot is. Just at a slower timescale.
(No surprise that this movement is popular among social outcasts, for whom local morality is going to feel less important, and (I suspect) autistic people, who probably experience less direct moral empathy for the people around them, as well as to the economically-insulated well-to-do tech-nerd types who are less likely to be directly exposed to suffering in their immediate communities.)
Ironically paperclip-maximizing-robots are exactly the thing that the rationalists are so worried about. They are a group of people who missed, and then disavowed, and now advocate disavowing, this "personal" morality, and unsurprisingly they view the world in a lens that doesn't include it, which means mostly being worried about problems of the same sort. But it provokes a strong negative reaction from everyone who thinks about the world in terms of that personal duty to safety, because that is the foundation of all morality, and is utterly essential to preserve, because it makes sure that whatever else you are doing doesn't go awry.
(edit: let me add that your aversion to the criticisms of rationalists is not unreasonable either. Given that you're parsing the criticisms as unreasonable, which they likely are (because of the skill issue), what you're seeing is a movement with value that seems to be being unfairly attacked. And you're right, the value is actually there! But the ultimate goal here is a synthesis: to get the value of the rationalist movement but to synthesize it with the recognition of the red flags that it sets off. Ignoring either side, the value or the critique, is ultimately counterproductive: the right goal is to synthesize both into a productive middle ground. (This is the arc of philosophy; it's what philosophy is. Not re-reading Plato.) The rationalists are probably morally correct in being motivated to highly-scaling actions e.g. the purview of "Effective Altruism". They are getting attacked for what they're discarding to do that, not for caring about it in the first place.)
Humans are generally better at perceiving threats than they are at putting those threats into words. When something seems "dangerous" abstractly, they will come up with words for why---but those words don't necessarily reflect the actual threat, because the threat might be hard to describe. Nevertheless the valence of their response reflects their actual emotion on the subject.
In this case: the rationalist philosophy basically creeps people out. There is something "insidious" about it. And this is not a delusion on the part of the people judging them: it really does threaten them, and likely for good reason. The explanation is something like "we extrapolate from the way that rationalists think and realize that their philosophy leads to dangerous conclusions." Some of these conclusions have already been made by the rationalists---like valuing people far away abstractly over people next door, by trying to quantify suffering and altruism like a math problem (or to place moral weight on animals over humans, or people in the future over people today). Other conclusions are just implied, waiting to be made later. But the human mind detects them anyway as implications of the way of thinking, and reacts accordingly: thinking like this is dangerous and should be argued against.
This extrapolation is hard to put into words, so everyone who tries to express their discomfort misses the target somewhat, and then, if you are the sort of person who only takes things literally, it sounds like they are all just attacking someone out of judgment or bitterness or something instead of for real reasons. But I can't emphasize this enough: their emotions are real, they're just failing to put them into words effectively. It's a skill issue. You will understand what's happening better if you understand that this is what's going on and then try to take their emotions seriously even if they are not communicating them very well.
So that's what's going on here. But I think I can also do a decent job of describing the actual problem that people have with the rationalist mindset. It's something like this:
Humans have an innate moral intuition that "personal" morality, the kind that takes care of themselves and their family and friends and community, is supposed to be sacrosanct: people are supposed to both practice it and protect the necessity of practicing it. We simply can't trust the world to be a safe place if people don't think of looking out for the people around them as a fundamental moral duty. And once those people are safe, protecting more people, such as a tribe or a nation or all of humanity or all of the planet, becomes permissible.
Sometimes people don't or can't practice this protection for various reasons, and that's morally fine, because it's a local problem that can be solved locally. But it's very insidious to turn around and justify not practicing it as a better way to live: "actually it's better not to behave morally; it's better to allocate resources to people far away; it's better to dedicate ourselves to fighting nebulous threats like AI safety or other X-risks instead of our neighbors; or, it's better to protect animals than people, because there are more of them". It's fine to work on important far-away problems once local problems are solved, if that's what you want. But it can't take priority, regardless of how the math works out. To work on global numbers-game problems instead of local problems, and to justify that with arguments, and to try to convince other people to also do that---that's dangerous as hell. It proves too much: it argues that humans at large ought to dismantle their personal moralities in favor of processing the world like a paperclip-maximizing robot. And that is exactly as dangerous as a paperclip-maximizing robot is. Just at a slower timescale.
(No surprise that this movement is popular among social outcasts, for whom local morality is going to feel less important, and (I suspect) autistic people, who probably experience less direct moral empathy for the people around them, as well as to the economically-insulated well-to-do tech-nerd types who are less likely to be directly exposed to suffering in their immediate communities.)
Ironically paperclip-maximizing-robots are exactly the thing that the rationalists are so worried about. They are a group of people who missed, and then disavowed, and now advocate disavowing, this "personal" morality, and unsurprisingly they view the world in a lens that doesn't include it, which means mostly being worried about problems of the same sort. But it provokes a strong negative reaction from everyone who thinks about the world in terms of that personal duty to safety, because that is the foundation of all morality, and is utterly essential to preserve, because it makes sure that whatever else you are doing doesn't go awry.
(edit: let me add that your aversion to the criticisms of rationalists is not unreasonable either. Given that you're parsing the criticisms as unreasonable, which they likely are (because of the skill issue), what you're seeing is a movement with value that seems to be being unfairly attacked. And you're right, the value is actually there! But the ultimate goal here is a synthesis: to get the value of the rationalist movement but to synthesize it with the recognition of the red flags that it sets off. Ignoring either side, the value or the critique, is ultimately counterproductive: the right goal is to synthesize both into a productive middle ground. (This is the arc of philosophy; it's what philosophy is. Not re-reading Plato.) The rationalists are probably morally correct in being motivated to highly-scaling actions e.g. the purview of "Effective Altruism". They are getting attacked for what they're discarding to do that, not for caring about it in the first place.)
I finally gave in and created an account because of your comment. It's beautifully put. I would only perhaps add that, to me, the neo-rationalist thing looks the most similar to things that don't work yet attract hardcore "true believers". It's a pattern repeated through the ages, perhaps most intimately for me in the seemingly endless parades of computer system redesigns: software, hardware, or both. Randomly one might pick "the new and exciting Digg", the Itanium, and the Metaverse as fairly modern examples.
There is something about a particular "narrowband" signaling approach, where a certain kind of purity is sought, with an expectation that, given enough explaining, you will finally get it, become enlightened, and convert to the ranks. A more "wideband" approach would at least admit observations like yours do exist and must be comprehensively addressed to the satisfaction of those who hold such beliefs vs to the satisfaction of those merely "stooping" to address them (again in the hopes they'll just see the light so everyone can get back to narrowband-ville).
There is something about a particular "narrowband" signaling approach, where a certain kind of purity is sought, with an expectation that, given enough explaining, you will finally get it, become enlightened, and convert to the ranks. A more "wideband" approach would at least admit observations like yours do exist and must be comprehensively addressed to the satisfaction of those who hold such beliefs vs to the satisfaction of those merely "stooping" to address them (again in the hopes they'll just see the light so everyone can get back to narrowband-ville).
(Thank you) I agree, although I do think that the rationalists and EAs are way better than most of the other narrowband groups, as you call them, out there, such as the Metaverse or Crypto people. The rationalists are at least mostly legitimately motivated by morality and not just by a "blow it all up and replace it with something we control" philosophy (which I have come to believe is the belief-set that only a person who is convinced that they are truly powerless comes to). I see the rationalists as failing due to a skill issue as well: because they have so-defined themselves by their rationalism, they have trouble understanding the things in the world that they don't have a good rational understanding of, such as morality. They are too invested in words and truth and correctness to understand that there can be a lot of emotional truth encoded in logical falsehood.
edit: oh, also, I think that a good part of people's aversion to the rationalists is just a reaction to the narrowband quality itself, not to the content. People are well-aware of the sorts of things that narrowband self-justifying philosophies lead to, from countless examples, whether it's at the personal level (an unaccountable schoolteacher) or societal (a genocidal movement). We don't trust a group unless they specifically demonstrate non-narrowbandedness, which means being collectively willing to change their behavior in ways that don't make sense to them. Any movement that co-opts the idea of what is morally justifiable---who says that e.g. rationality is what produces truth and things that run counter to it do not---is inherently frightening.
edit: oh, also, I think that a good part of people's aversion to the rationalists is just a reaction to the narrowband quality itself, not to the content. People are well-aware of the sorts of things that narrowband self-justifying philosophies lead to, from countless examples, whether it's at the personal level (an unaccountable schoolteacher) or societal (a genocidal movement). We don't trust a group unless they specifically demonstrate non-narrowbandedness, which means being collectively willing to change their behavior in ways that don't make sense to them. Any movement that co-opts the idea of what is morally justifiable---who says that e.g. rationality is what produces truth and things that run counter to it do not---is inherently frightening.
They aren’t motivated by morality. They just are more moral relative to the niches you referred to.
Any group that focuses on their own goals of high paying jobs regardless of the morality of those jobs or how they contribute to the structural issues of society is not that good. Then donating money while otherwise being okay with the status quo —- not touching anything systemic in such an unjust world but supposedly focusing on morality is laughable.
Any group that focuses on their own goals of high paying jobs regardless of the morality of those jobs or how they contribute to the structural issues of society is not that good. Then donating money while otherwise being okay with the status quo —- not touching anything systemic in such an unjust world but supposedly focusing on morality is laughable.
I see it as moral action with a skill issue, which is how I see a lot of bad things in the world. People are very good at missing the forest for the trees.
In their defense they do try to do the calculations: is a high-paying job okay on net if you give most of the money away? depends on the job; depends on if somebody else would have it if you didn't; etc. Not that there is a rigorous way to do them but it's very much a group that does try.
In their defense they do try to do the calculations: is a high-paying job okay on net if you give most of the money away? depends on the job; depends on if somebody else would have it if you didn't; etc. Not that there is a rigorous way to do them but it's very much a group that does try.
> Ironically paperclip-maximizing-robots are exactly the thing that the rationalists are so worried about. They are a group of people who missed, and then disavowed, and now advocate disavowing, this "personal" morality, and unsurprisingly they view the world in a lens that doesn't include personality morality, which means mostly being worried about problems of the same sort. But it provokes a strong negative reaction from everyone who thinks about the world in terms of that personal duty to safety.
I head not read any rationalist writing in a long time (and I didn't know about Scott's proximity), but the whole time I was reqding the article I was thinking the same thing you just wrote... "why are they afraid of AI, i.e. the ultimate rationalist taking over the world", maybe something deep inside of them has the same reaction to their own theories as you so eloquently put above.
I head not read any rationalist writing in a long time (and I didn't know about Scott's proximity), but the whole time I was reqding the article I was thinking the same thing you just wrote... "why are they afraid of AI, i.e. the ultimate rationalist taking over the world", maybe something deep inside of them has the same reaction to their own theories as you so eloquently put above.
I don't read these rationalist essays either, but you don't need to be a deep thinker to understand why any rational person would be afraid of AI and the singularity.
The AI will do what its programmed to do, but its programmers morality may not match my own. What more scary is that it may be developed with the morality of a corporation rather than a person. (That is to say, no morals at all.)
I think its perfectly justifiable to be scared of a very powerful being with no morals stomping around!
The AI will do what its programmed to do, but its programmers morality may not match my own. What more scary is that it may be developed with the morality of a corporation rather than a person. (That is to say, no morals at all.)
I think its perfectly justifiable to be scared of a very powerful being with no morals stomping around!
Those corporations are already superhuman entities with morals that don’t match ours. They do cause a lot of problems. Maybe it’s better to figure out how to fix that real, current problem rather than hypothetical future ones.
This parallel has been drawn. Charlie Stross [0] in particular thinks the main difference is that pre-digital AIs are capable of behaving much faster, so that other entities (countries, lawmakers, …) have time to react to them.
[0]: https://www.antipope.org/charlie/blog-static/2018/01/dude-yo...
Similar claims can be made about any structure of humans that exhibits gestalt intelligence, e.g. nations, stock markets, etc.
[0]: https://www.antipope.org/charlie/blog-static/2018/01/dude-yo...
Similar claims can be made about any structure of humans that exhibits gestalt intelligence, e.g. nations, stock markets, etc.
It's quite likely I got it from that!
Yeah, the AI will be a tool of a corp. So in effect we are talking about limiting corporate power.
> the AI will be a tool of a corp.
Up to a certain level. Behind that level, the corp will be a tool of the AI. And even further that way, the corp will no longer be necessary.
Up to a certain level. Behind that level, the corp will be a tool of the AI. And even further that way, the corp will no longer be necessary.
One mistake you making is thinking that rationalists care more about people far away than people in their community. The reality is that they set the value of life the same for all.
If children around you are doing of an easily preventable disease, then yes, help them first! If they just need more arts programs, then you help the children dying in another country first.
If children around you are doing of an easily preventable disease, then yes, help them first! If they just need more arts programs, then you help the children dying in another country first.
That's not a mistake I'm making. Assuming you're talking about bog-standard effective altruists---by (claiming to) value the suffering of people far away as the same as those nearby, they're discounting the people around them heavily compared to other people. Compare to anyone else who values their friends and family and community far more than those far away. Perhaps they're not discounting them to less-than-parity---just less than they are for most people.
But anyway this whole model follows from a basic set of beliefs about quantifying suffering and about what one's ethical responsibilities are, and it answers those in ways most people would find very bizarre by turning them into a math problem that assigns no special responsibility to the people around you. I think that is much more contentious and gross to most people than EA thinks it is. It can be hard to say exactly why in words, but that doesn't make it less true.
But anyway this whole model follows from a basic set of beliefs about quantifying suffering and about what one's ethical responsibilities are, and it answers those in ways most people would find very bizarre by turning them into a math problem that assigns no special responsibility to the people around you. I think that is much more contentious and gross to most people than EA thinks it is. It can be hard to say exactly why in words, but that doesn't make it less true.
To me, the non-local focus of EA/rationalism is, at least partially, a consequence of their historically unusual epistemology.
In college, I became a scale-dependent realist, which is to say, that I'm most confident of theories / knowledge in the 1-meter, 1-day, 1 m/s scales and increasingly skeptical of our understanding of things that are bigger/smaller, have longer/short timeframes, or faster velocities. Maybe there is a technical name for my position? But, it is mostly a skepticism about nearly unlimited extrapolation using brains that evolved under selection for reproduction at a certain scale. My position is not that we can't compute at different scales, but that we can't understand at other scales.
In practice, the rationalists appear to invert their confidence, with more confidence in quarks and light-years than daily experience.
In college, I became a scale-dependent realist, which is to say, that I'm most confident of theories / knowledge in the 1-meter, 1-day, 1 m/s scales and increasingly skeptical of our understanding of things that are bigger/smaller, have longer/short timeframes, or faster velocities. Maybe there is a technical name for my position? But, it is mostly a skepticism about nearly unlimited extrapolation using brains that evolved under selection for reproduction at a certain scale. My position is not that we can't compute at different scales, but that we can't understand at other scales.
In practice, the rationalists appear to invert their confidence, with more confidence in quarks and light-years than daily experience.
> no special responsibility to the people around you
Musing on the different failure-directions: Pretty much any terrible present thing against people can be rationalized by arguing that one gadzillion distant/future people are more important. That includes religious versions, where the stakes of the holy war may presented as all of future humanity being doomed to infinite torment. There are even some cults that pitch it retroactively: Offer to the priesthood to save all your ancestors who are in hell because of original sin.
The opposite would be to prioritize the near and immediate, culminating in a despotic god-king. This is somewhat more-familiar, we may have more cultural experience and moral tools for detection and prevention.
A check on either process would be that the denigrated real/nearby humans revolt. :p
Musing on the different failure-directions: Pretty much any terrible present thing against people can be rationalized by arguing that one gadzillion distant/future people are more important. That includes religious versions, where the stakes of the holy war may presented as all of future humanity being doomed to infinite torment. There are even some cults that pitch it retroactively: Offer to the priesthood to save all your ancestors who are in hell because of original sin.
The opposite would be to prioritize the near and immediate, culminating in a despotic god-king. This is somewhat more-familiar, we may have more cultural experience and moral tools for detection and prevention.
A check on either process would be that the denigrated real/nearby humans revolt. :p
> they're discounting the people around them heavily compared to other people
This statement of yours makes no sense.
EAs by definition are attempting to remove the innate bias that discounts people far away by instead saying all lives are of equal worth.
>turning them into a math problem that assigns no special responsibility to the people around you
All lives are equal isn't a math problem. "Fuck it blow up the foreigners to keep oil prices low" is a math problem, it is a calculus that the US government has spent decades performing. (One that assigns zero value to lives outside the US.)
If $100 can save 1 life 10 blocks away from me or 5 lives in the next town over, what kind as asshole chooses to let 5 people die vs 1?
And since air travel is a thing, what the hell does "close to us" mean?
For that matter, from a purely selfish POV, helping lift other nations up to become fully advanced economies is hugely beneficial to me, and everyone on earth, in the long run. I'm damn thankful for all the aid my country gave to South Korea, the number of scientific advances that have come out of SK damn well paid for any tax dollars my grandparents paid on many orders of magnitude times over.
> It can be hard to say exactly why in words, but that doesn't make it less true.
This is the part where I shout racism.
Because history has shown it isn't about people being far or close in distance, but rather in how those people look.
Americans have shot down multiple social benefit programs because, and these are what people who voted against those programs directly said was their reasons "white people don't want black people getting the same help white people get."
Whites in America have voted, repeatedly, to keep themselves poor rather than lift themselves and black families out of poverty at the same time.
Of course Americans think helping people in Africa is "weird".
This statement of yours makes no sense.
EAs by definition are attempting to remove the innate bias that discounts people far away by instead saying all lives are of equal worth.
>turning them into a math problem that assigns no special responsibility to the people around you
All lives are equal isn't a math problem. "Fuck it blow up the foreigners to keep oil prices low" is a math problem, it is a calculus that the US government has spent decades performing. (One that assigns zero value to lives outside the US.)
If $100 can save 1 life 10 blocks away from me or 5 lives in the next town over, what kind as asshole chooses to let 5 people die vs 1?
And since air travel is a thing, what the hell does "close to us" mean?
For that matter, from a purely selfish POV, helping lift other nations up to become fully advanced economies is hugely beneficial to me, and everyone on earth, in the long run. I'm damn thankful for all the aid my country gave to South Korea, the number of scientific advances that have come out of SK damn well paid for any tax dollars my grandparents paid on many orders of magnitude times over.
> It can be hard to say exactly why in words, but that doesn't make it less true.
This is the part where I shout racism.
Because history has shown it isn't about people being far or close in distance, but rather in how those people look.
Americans have shot down multiple social benefit programs because, and these are what people who voted against those programs directly said was their reasons "white people don't want black people getting the same help white people get."
Whites in America have voted, repeatedly, to keep themselves poor rather than lift themselves and black families out of poverty at the same time.
Of course Americans think helping people in Africa is "weird".
> If $100 can save 1 life 10 blocks away from me or 5 lives in the next town over, what kind as asshole chooses to let 5 people die vs 1?
The thing about strict-utilitarian-morality is that it can't comprehend any other kind of morality, because it evaluates the morality of... moralities... on its own utilitarian basis. And then of course it wins over the others: it's evaluating them using itself!
There are entirely different ethical systems that are not utilitarian which (it seems) most people hold and innately use (the "personal morality" I'm talking about in my earlier post). They are hard to comprehend rationally, but that doesn't make them less real. Strict-utilitarianism seems "correct" in a way that personal morality does not because you are working from a premise "only things that I can understand like math problems can be true". But what I observe in the world is that people's fear of the rationalist/EA mindset comes from the fact that they empirically find this way of thinking to be insidious. Their morality specifically disagrees with that way of thinking: it is not the case that truth comes from scrutable math problems; that is not the point of moral action to them.
The EA philosophy may be put as "well sure but you could change to the math-problem version, it's better". But what I observe is that people largely don't want to. There is a purpose to their choice of moral framework; it's not that they're looking at them all in a vacuum and picking the most mathematically sound one. They have an intrinsic need to keep the people around them safe and they're picking the one that does that best. EA on the other hand is great if everyone around you is safe and you have lots of extra spending money and what you're maximizing for is the feeling of being a good person. But it is not the only way to conceive of moral action, and if you think it is, you're too inside of it to see out.
I'll reiterate I am trying to describe what I see happening when people resist and protest rationalism (and why their complaints "miss" slightly---because IMO they don't have the language to talk about this stuff but they are still afraid of it). I'm sympathetic to EA largely, but I think it misses important things that are crippling it, of the variety above: an inability to recognize other people's moralities and needs and fears doesn't make them go away; it just makes them hate you.
The thing about strict-utilitarian-morality is that it can't comprehend any other kind of morality, because it evaluates the morality of... moralities... on its own utilitarian basis. And then of course it wins over the others: it's evaluating them using itself!
There are entirely different ethical systems that are not utilitarian which (it seems) most people hold and innately use (the "personal morality" I'm talking about in my earlier post). They are hard to comprehend rationally, but that doesn't make them less real. Strict-utilitarianism seems "correct" in a way that personal morality does not because you are working from a premise "only things that I can understand like math problems can be true". But what I observe in the world is that people's fear of the rationalist/EA mindset comes from the fact that they empirically find this way of thinking to be insidious. Their morality specifically disagrees with that way of thinking: it is not the case that truth comes from scrutable math problems; that is not the point of moral action to them.
The EA philosophy may be put as "well sure but you could change to the math-problem version, it's better". But what I observe is that people largely don't want to. There is a purpose to their choice of moral framework; it's not that they're looking at them all in a vacuum and picking the most mathematically sound one. They have an intrinsic need to keep the people around them safe and they're picking the one that does that best. EA on the other hand is great if everyone around you is safe and you have lots of extra spending money and what you're maximizing for is the feeling of being a good person. But it is not the only way to conceive of moral action, and if you think it is, you're too inside of it to see out.
I'll reiterate I am trying to describe what I see happening when people resist and protest rationalism (and why their complaints "miss" slightly---because IMO they don't have the language to talk about this stuff but they are still afraid of it). I'm sympathetic to EA largely, but I think it misses important things that are crippling it, of the variety above: an inability to recognize other people's moralities and needs and fears doesn't make them go away; it just makes them hate you.
> The thing about strict-utilitarian-morality is that it can't comprehend any other kind of morality, because it evaluates the morality of... moralities... on its own utilitarian basis.
I can comprehend them just fine, but I have a deep seated objection to any system of morality that leaves behind giant piles of dead bodies. We should be trying to minimize the size of the pile of dead bodies (and ideally eliminate the pile altogether!)
Any system or morality that boils down to "I don't care about that pile of dead bodies being huge because those people look different" is in fact not a system morality at all.
I can comprehend them just fine, but I have a deep seated objection to any system of morality that leaves behind giant piles of dead bodies. We should be trying to minimize the size of the pile of dead bodies (and ideally eliminate the pile altogether!)
Any system or morality that boils down to "I don't care about that pile of dead bodies being huge because those people look different" is in fact not a system morality at all.
Well, you won't find anyone who disagrees with you here. No such morality is being discussed.
The job of a system of morality is to synthesize all the things we want to happen / want to prevent happening into a way of making decisions. One such thing is piles of dead bodies. Another is one's natural moral instincts, like their need to take care of their family, or the feeling of responsibility to invest time and energy into improving their future or their community or repairing justice or helping people who need help, or to attend to their needs for art and meaning and fun and love and respect. A coherent moral system synthesizes these all and figures out how much priority to allocate to each thing in a way that is reasonable and productive.
Any system of morality that takes one of these criteria and discards the rest of them is not a system of morality at all, in the very literal sense that nobody will do it. Most people won't sell out one of their moral impulses for the others, and EA/rationalism feels like it asks them too, since it asks them to place zero value in a lot of things that they inherently place moral value in, and so they find it creepy and weird. (It doesn't ask that explicitly; it asks it by omission. By never considering any other morality and being incapable of considering them, because they are not easily quantifiable/made logical, it asks you to accept a framework that sets you up to ignore most of your needs.)
My angle here is that I'm trying to describe what I believe is already happening. I'm not advocating it; it's already there, like a law of physics.
The job of a system of morality is to synthesize all the things we want to happen / want to prevent happening into a way of making decisions. One such thing is piles of dead bodies. Another is one's natural moral instincts, like their need to take care of their family, or the feeling of responsibility to invest time and energy into improving their future or their community or repairing justice or helping people who need help, or to attend to their needs for art and meaning and fun and love and respect. A coherent moral system synthesizes these all and figures out how much priority to allocate to each thing in a way that is reasonable and productive.
Any system of morality that takes one of these criteria and discards the rest of them is not a system of morality at all, in the very literal sense that nobody will do it. Most people won't sell out one of their moral impulses for the others, and EA/rationalism feels like it asks them too, since it asks them to place zero value in a lot of things that they inherently place moral value in, and so they find it creepy and weird. (It doesn't ask that explicitly; it asks it by omission. By never considering any other morality and being incapable of considering them, because they are not easily quantifiable/made logical, it asks you to accept a framework that sets you up to ignore most of your needs.)
My angle here is that I'm trying to describe what I believe is already happening. I'm not advocating it; it's already there, like a law of physics.
Perhaps part of it is that local action can often be an order of magnitude more impactful than the “equivalent” action at a distance. If you volunteer in your local community, you not only have fine-grained control over the benefit you bestow, you also know for a fact that you’re doing good. Giving to a charity that addresses an issue on the other side of the world doesn’t afford this level of control, nor this level of certainty. For all you know most of the donation is being embezzled.
I think another part of it is a sort of healthy nativism or in-group preference or whatever you want to call it. It rubs people the wrong way when you say that you care about someone in a different country as much as you care about your neighbors. That’s just…antisocial. Taken to its logical conclusion, a “rationalist” should not only donate all of their disposable income to global charities, they should also find a way to steal as much as possible from their neighbors and donate that, too. After all, those. Holden in Africa need the money much more than their pampered western neighbors.
I think another part of it is a sort of healthy nativism or in-group preference or whatever you want to call it. It rubs people the wrong way when you say that you care about someone in a different country as much as you care about your neighbors. That’s just…antisocial. Taken to its logical conclusion, a “rationalist” should not only donate all of their disposable income to global charities, they should also find a way to steal as much as possible from their neighbors and donate that, too. After all, those. Holden in Africa need the money much more than their pampered western neighbors.
What creeps me out is that I have no idea of their theory of power: How will they achieve their aims?
Maybe they want to do it in a way I’d consider just: By exercising their rights as individuals in their personal domains and effectively airing their arguments in the public sphere to win elections.
But my intuition is they think democracy and personal rights of the non-elect are part of the problem to rationalize around and over.
Would genuinely love to read some Rationalist discourse on this question.
Maybe they want to do it in a way I’d consider just: By exercising their rights as individuals in their personal domains and effectively airing their arguments in the public sphere to win elections.
But my intuition is they think democracy and personal rights of the non-elect are part of the problem to rationalize around and over.
Would genuinely love to read some Rationalist discourse on this question.
Why is "theory of power" a necessary framing? "Theory of power" seems to be only a popular idea in "online left progressive" circles which are based on pop readings of post-structuralist continental philosophy. There's plenty of other schools of thought out there and there have been many criticisms of the idea of "theory of power" altogether.
Reading critiques of Hegel is a great starting point for this reading.
Whether you accept it or not though, there's lots of non-rationalist schools that reject the need for a "theory of power".
Reading critiques of Hegel is a great starting point for this reading.
Whether you accept it or not though, there's lots of non-rationalist schools that reject the need for a "theory of power".
I didn’t mean the term in anything but a colloquial sense: They believe certain outcomes are right and proper. Such outcomes don’t typically manifest themselves out of good intentions. What’s the plan?
When Curtis Yarvin is at least in your orbit, these should not be surprising questions to get.
When Curtis Yarvin is at least in your orbit, these should not be surprising questions to get.
you've latched on to some academic sense of the term but I understood exactly what the person you're replying to meant. They mean: are these people you would respect and trust to be in charge of things? What would they do? And a lot of people's vibe-check says it's a big unknown question mark, or worse, which is not something you can put faith in.
It's hard to argue with vibes which is what makes today's culture war based politics so difficult to weigh in on conclusively. It's all just vibes after all. One person's vibes can be cap to someone else.
There's a reason technofeudalists like Yarvin are popular in rationalist adjacent circles.
> Humans are generally better at perceiving threats than they are at putting those threats into words.
Not only that, but this is exactly the kind of scenario where we should be giving those signals the most weight: The individual estimating whether to join up with a tribe. (As opposed to, say, bad feelings about doing calculus.)
Not only does it involve humans-predicting-humans (where we have a rather privileged set of tools) but there have been millions of years of selective pressure to be decent at it.
Not only that, but this is exactly the kind of scenario where we should be giving those signals the most weight: The individual estimating whether to join up with a tribe. (As opposed to, say, bad feelings about doing calculus.)
Not only does it involve humans-predicting-humans (where we have a rather privileged set of tools) but there have been millions of years of selective pressure to be decent at it.
This is a fantastic comment, but it also underscores the thing I really dislike about most internet conversations about philosophy: a lack of reading prior art. Utilitarian morality certainly isn't a new concept though the rationalists today are some of its strongest standard bearers. But around the time when utilitarianism was starting to take hold, rigorous philosophical thinking had exactly the debates that come up here.
Part of the reason I enjoy rationalist discourse more is because, even if they are unabashedly utilitarian, they try to rigorously derive philosophy. Most internet discourse on philosophy is, as you say, just vaguely derived around gut feelings. But philosophy can and has been thought of rigorously. Virtue ethics and continental morality are both schools of thought that reject utilitarian ethics but are much more meaty than the sort of internet "no but my neighbors" intuition that you see in full force, and the weird insistence that these internet commenters continue to use their vague moral intuition without being rigorous about their own thoughts.
Part of the reason I enjoy rationalist discourse more is because, even if they are unabashedly utilitarian, they try to rigorously derive philosophy. Most internet discourse on philosophy is, as you say, just vaguely derived around gut feelings. But philosophy can and has been thought of rigorously. Virtue ethics and continental morality are both schools of thought that reject utilitarian ethics but are much more meaty than the sort of internet "no but my neighbors" intuition that you see in full force, and the weird insistence that these internet commenters continue to use their vague moral intuition without being rigorous about their own thoughts.
Oh this is a great breakdown, I totally agree. I feel there’s a morality hierarchy that’s akin to Maslow’s hierarchy of needs.
People benefit from a sense of a family, a sense of community. It helps us feel more secure, both personally and to our loved ones.
I think the more I view things through this lenses the downstream benefits I see.
People benefit from a sense of a family, a sense of community. It helps us feel more secure, both personally and to our loved ones.
I think the more I view things through this lenses the downstream benefits I see.
No offense, but this way of thinking is the domain of comic book supervillains. "I must destroy the world in order to save it." Morality is only holding me back from maximizing the value of the human race 1,000 or 1,000,000 years from now type nonsense.
This sort of reasoning sounds great from 1000 feet up, but the longer you do it the closer you get to "I need to kill nearly all current humans to eliminate genetic diseases and control global warming and institute an absolute global rationalist dictatorship to prevent wars or humanity is doomed over the long run".
Or you get people who are working in a near panic to bring about godlike AI because they think that once the AI singularity happens the new AI God will look back in history and kill anybody who didn't work their hardest to bring it into existence because they assume an infinite mind will contain infinite cruelty.
This sort of reasoning sounds great from 1000 feet up, but the longer you do it the closer you get to "I need to kill nearly all current humans to eliminate genetic diseases and control global warming and institute an absolute global rationalist dictatorship to prevent wars or humanity is doomed over the long run".
Or you get people who are working in a near panic to bring about godlike AI because they think that once the AI singularity happens the new AI God will look back in history and kill anybody who didn't work their hardest to bring it into existence because they assume an infinite mind will contain infinite cruelty.
Found the person who doesn't have this skill issue! :) Very well put.
I never quite realized the connection between this creepy feeling and the Bleak House criticism of far-flung philanthropy, until this comment.
I never quite realized the connection between this creepy feeling and the Bleak House criticism of far-flung philanthropy, until this comment.
heh. It would not be an overstatement to say I've been working on better putting the issue with the rationalists into words for ten years. glad it's resonating with some people.
> The explanation is something like "we extrapolate from the way that rationalists think and realize that their philosophy leads to dangerous conclusions."
I really like the depth of analysis in your comment, but I think there's one important element missing, which is that this is not an individual decision but a group heuristic to which individuals are then sensitized. Individuals don't typically go so far as to (consciously or unconsciously) extrapolate others' logic forward to decide that it's dangerous. Instead, people get creeped out when other people don't adhere to social patterns and principles that are normalized as safe in their culture, because the consequences are unknown and therefore potentially dangerous; or when they do adhere to patterns that are culturally believed to be dangerous. This can be used successfully to identify things that are really dangerous, but also has a high false positive rate (people with disabilities, gender identities, or physical characteristics that are not common or accepted within the beholder's culture can all trigger this, despite not posing any immediate/inherent threat) as well as a high false negative rate (many serial killers are noted to have been very charismatic, because they put effort into studying how to behave to not trigger this instinct). When we speak of something being normalized, we're talking about it becoming accepted by the mainstream so that it no longer triggers the ‘creepy’ response in the majority of individuals. As far as I can tell, the social conservative basically believes that the set of normalized things has been carefully evolved over many generations, and therefore should be maintained (or at least modified only very cautiously) even if we don't understand why they are as they are, while the social liberal believes that we the current generation are capable of making informed judgements about which things are and aren't harmful to a degree that we can (and therefore should) continuously iterate on that set to approach an ideal goal state in which it contains only things that are factually known to be harmful.
As an interesting aside, the ‘creepy’ emotion, (at least IIRC in women) is triggered not by obviously dangerous situations but by ambiguously dangerous situations, i.e. ones that don't obviously match the pattern of known safe or unsafe situations.
> Sometimes people don't or can't practice this protection for various reasons, and that's fine; it's a local problem that can be solved locally. But it's very insidious to turn around and justify not practicing it: "actually it's better not to behave morally; it's better to allocate resources to people far away; it's better to dedicate ourselves to fighting nebulous threats like AI safety or other X-risks instead of our neighbors".
The problem with the ‘us before them’ approach is that if two neighbourhoods each prioritize their local neighbourhood over the remote neighbourhood and compete (or go to war) to better their own neighbourhood at the cost of the other, generally both neighbourhoods are left worse off than they started, at least in the short term: both groups trying to make locally optimal choices leads (without further constraints) to globally highly suboptimal outcomes. In recognition of this a lot of people, not just capital-R Rationalists, now believe that at least in the abstract we should really be trying to optimize for global outcomes.
Whether anybody realistically has the computational ability to do so effectively is a different question, of course. Certainly I personally think the future-discounting ‘bias’ is a heuristic used to acknowledge the inherent uncertainty of any future outcome we might be trying to assign moral weight to, and should be accorded some respect. Perhaps you can make the same argument for the locality bias, but I guess that Rationalists (generally) either believe that you can't, or at least have a moral duty to optimize for the largest scope your computational power allows.
I really like the depth of analysis in your comment, but I think there's one important element missing, which is that this is not an individual decision but a group heuristic to which individuals are then sensitized. Individuals don't typically go so far as to (consciously or unconsciously) extrapolate others' logic forward to decide that it's dangerous. Instead, people get creeped out when other people don't adhere to social patterns and principles that are normalized as safe in their culture, because the consequences are unknown and therefore potentially dangerous; or when they do adhere to patterns that are culturally believed to be dangerous. This can be used successfully to identify things that are really dangerous, but also has a high false positive rate (people with disabilities, gender identities, or physical characteristics that are not common or accepted within the beholder's culture can all trigger this, despite not posing any immediate/inherent threat) as well as a high false negative rate (many serial killers are noted to have been very charismatic, because they put effort into studying how to behave to not trigger this instinct). When we speak of something being normalized, we're talking about it becoming accepted by the mainstream so that it no longer triggers the ‘creepy’ response in the majority of individuals. As far as I can tell, the social conservative basically believes that the set of normalized things has been carefully evolved over many generations, and therefore should be maintained (or at least modified only very cautiously) even if we don't understand why they are as they are, while the social liberal believes that we the current generation are capable of making informed judgements about which things are and aren't harmful to a degree that we can (and therefore should) continuously iterate on that set to approach an ideal goal state in which it contains only things that are factually known to be harmful.
As an interesting aside, the ‘creepy’ emotion, (at least IIRC in women) is triggered not by obviously dangerous situations but by ambiguously dangerous situations, i.e. ones that don't obviously match the pattern of known safe or unsafe situations.
> Sometimes people don't or can't practice this protection for various reasons, and that's fine; it's a local problem that can be solved locally. But it's very insidious to turn around and justify not practicing it: "actually it's better not to behave morally; it's better to allocate resources to people far away; it's better to dedicate ourselves to fighting nebulous threats like AI safety or other X-risks instead of our neighbors".
The problem with the ‘us before them’ approach is that if two neighbourhoods each prioritize their local neighbourhood over the remote neighbourhood and compete (or go to war) to better their own neighbourhood at the cost of the other, generally both neighbourhoods are left worse off than they started, at least in the short term: both groups trying to make locally optimal choices leads (without further constraints) to globally highly suboptimal outcomes. In recognition of this a lot of people, not just capital-R Rationalists, now believe that at least in the abstract we should really be trying to optimize for global outcomes.
Whether anybody realistically has the computational ability to do so effectively is a different question, of course. Certainly I personally think the future-discounting ‘bias’ is a heuristic used to acknowledge the inherent uncertainty of any future outcome we might be trying to assign moral weight to, and should be accorded some respect. Perhaps you can make the same argument for the locality bias, but I guess that Rationalists (generally) either believe that you can't, or at least have a moral duty to optimize for the largest scope your computational power allows.
yeah, my model of the "us before them" question is that it is almost always globally optimal to cooperate, once a certain level of economic productivity is present. The safety that people are worried about is guaranteed not by maximizing their wealth but by minimizing their chances of death/starvation/conquest. Up to a point this means being strong and subjugating your neighbor (cf most of antiquity?) but eventually it means collaborating with them and including them in your "tribe" and extending your protection to them. I have no respect for anyone who argues to undo this, which is I think basically the ethos of the trump movement: by convincing everyone that they are under threat, they get people to turn on those that are actually working in concert with them (in order to enrich/empower themselves). It is a schema problem: we are so very very far away from an us vs. them world that it requires delusions to believe.
(...that said, progressivism has largely failed in dispelling this delusion. It is far too easy to feel as though progressivism/liberalism exists to prop up power hierarchies and economic disparities because in many ways it does, or has been co-opted to do that. I think on net it does not, but it should be much more cut-and-dry than it is. For that to be the case progressivism would need to find a way to effectively turn on its parasites, that is, rent-extracting capitalism and status-extracting moral elitism).
re: the first part of your reply. I sorta agree but I do think people do more extrapolation than you're saying on their own. The extrapolation is largely based on pattern-matching to known things: we have a rich literature (in the news, in art, in personal experience and storytelling) of failure modes of societies, which includes all kinds of examples of people inventing new moral rationalizations for things and using them to disregard personal morality. I think when people are extrapolating rationalists' ideas to find things that creep them out, they're largely pattern-matching to arguments they've seen in other places. It's not just that they're unknowns. And those arguments are, well, real arguments that require addressing.
And yeah, there are plenty of examples of people being afraid of things that today we think they should not have been afraid of. I tend to think that that's just how things go: it is the arc of social progress to figure out how to change things from unknown+frightening to known+benign. I won't fault anyone for being afraid of something they don't understand, but I will fault them for not being open-minded about it or being unempathetic or being cruel or not giving people chances to prove themselves.
All of this is rendered much more opaque and confusing by the fact that everyone places way too much stock in words, though. (e.g. the OP I was replying to who was taking all these criticisms of the rationalists at face-value). IMO this is a major trend that fucks royally with our ability as a society to make moral progress: we have come to believe that words supplant emotional intuition in a way that wrecks out ability to actually understand what people are upset about (I like to blame this trend for much of the modern political polarization). A small example of this is a case that I think everyone has experienced, which is a person discounting their own sense of creepiness from somebody else because they can't come up with a good reason to explain it and it feels unfair to treat someone coldly on a hunch. That should never have been possible: everyone should be trusting their hunches.
(which may seem to conflict with my preceding paragraph... should you trust your hunches or give people the chance to prove themselves? well, it's complicated, but it also really depends on what the result is. Avoiding someone personally because they creep you out is always fine, but banning their way of life when it doesn't affect you at all or directly harm anyone is certainly not.)
(...that said, progressivism has largely failed in dispelling this delusion. It is far too easy to feel as though progressivism/liberalism exists to prop up power hierarchies and economic disparities because in many ways it does, or has been co-opted to do that. I think on net it does not, but it should be much more cut-and-dry than it is. For that to be the case progressivism would need to find a way to effectively turn on its parasites, that is, rent-extracting capitalism and status-extracting moral elitism).
re: the first part of your reply. I sorta agree but I do think people do more extrapolation than you're saying on their own. The extrapolation is largely based on pattern-matching to known things: we have a rich literature (in the news, in art, in personal experience and storytelling) of failure modes of societies, which includes all kinds of examples of people inventing new moral rationalizations for things and using them to disregard personal morality. I think when people are extrapolating rationalists' ideas to find things that creep them out, they're largely pattern-matching to arguments they've seen in other places. It's not just that they're unknowns. And those arguments are, well, real arguments that require addressing.
And yeah, there are plenty of examples of people being afraid of things that today we think they should not have been afraid of. I tend to think that that's just how things go: it is the arc of social progress to figure out how to change things from unknown+frightening to known+benign. I won't fault anyone for being afraid of something they don't understand, but I will fault them for not being open-minded about it or being unempathetic or being cruel or not giving people chances to prove themselves.
All of this is rendered much more opaque and confusing by the fact that everyone places way too much stock in words, though. (e.g. the OP I was replying to who was taking all these criticisms of the rationalists at face-value). IMO this is a major trend that fucks royally with our ability as a society to make moral progress: we have come to believe that words supplant emotional intuition in a way that wrecks out ability to actually understand what people are upset about (I like to blame this trend for much of the modern political polarization). A small example of this is a case that I think everyone has experienced, which is a person discounting their own sense of creepiness from somebody else because they can't come up with a good reason to explain it and it feels unfair to treat someone coldly on a hunch. That should never have been possible: everyone should be trusting their hunches.
(which may seem to conflict with my preceding paragraph... should you trust your hunches or give people the chance to prove themselves? well, it's complicated, but it also really depends on what the result is. Avoiding someone personally because they creep you out is always fine, but banning their way of life when it doesn't affect you at all or directly harm anyone is certainly not.)
Holy heck this is so well put and does the exact thing where it puts into words how I feel which is hard for me to do myself.
This is a remarkably well started comment.
One thing I'd like to add though is that I do think there is an additional piece being discarded irrationally. They tend to highly undervalue everything you're describing. Humans aren't Vulcans. By being so obsessed with the risks of paperclip-maximizing-robots they devalue the risks of humans being the irrational animals they are.
This is why many on the left criticize them for being right wing. Not because they are, well some might be, but because they are incredibly easy to distract from what is being communicated by focusing too much on what is being said. That might be a bad phrasing but what I mean is that when you look at this piece from last year about prison sentence length and crime rates by Scott Alexander[0] nothing he says is genuinely unreasonable. He's generally evaluating the data fairly and rationally. Some might disagree there but that's not my point. My point is that he's talking to a nonexistant group. The right largely believes that punishment is the point of prison. They might _say_ the goal is to reduce crime, but they are communicating based on a set of beliefs that strongly favors punitive measures for their own sake. This causes a piece like that to miss the forest through the trees and can be seen by those on the left as functionally right wing propaganda.
Most people are not rational. Maybe some day they will be but until then it is dangerous to assume and act as if they are. This makes me see the rationalists as actually rather irrational.
0: https://www.astralcodexten.com/p/prison-and-crime-much-more-...
One thing I'd like to add though is that I do think there is an additional piece being discarded irrationally. They tend to highly undervalue everything you're describing. Humans aren't Vulcans. By being so obsessed with the risks of paperclip-maximizing-robots they devalue the risks of humans being the irrational animals they are.
This is why many on the left criticize them for being right wing. Not because they are, well some might be, but because they are incredibly easy to distract from what is being communicated by focusing too much on what is being said. That might be a bad phrasing but what I mean is that when you look at this piece from last year about prison sentence length and crime rates by Scott Alexander[0] nothing he says is genuinely unreasonable. He's generally evaluating the data fairly and rationally. Some might disagree there but that's not my point. My point is that he's talking to a nonexistant group. The right largely believes that punishment is the point of prison. They might _say_ the goal is to reduce crime, but they are communicating based on a set of beliefs that strongly favors punitive measures for their own sake. This causes a piece like that to miss the forest through the trees and can be seen by those on the left as functionally right wing propaganda.
Most people are not rational. Maybe some day they will be but until then it is dangerous to assume and act as if they are. This makes me see the rationalists as actually rather irrational.
0: https://www.astralcodexten.com/p/prison-and-crime-much-more-...
Read other comments then. Not everyone is saying “they think they are better than us”.
The primary issues as others have noted is they focus on people going to the highest paying jobs without much care for morality of the jobs. Ergo they are fine being net negatives in terms of their work and philosophy.
All they do is donate money. Donations don’t fix society. Nothing changes structurally. No root problems are looked at.
They largely ignore capitalism’s faults or when I’ve seen them talk about, it’s done in a way of superficially decrying capitalist issues but then largely going along with them. Which ties into how they focus on high paying jobs regardless of morality (I’m exaggerating here but the overall point is correct).
—
HN is not intelligent when it comes to politics or the world. The avg person here is a western chauvinist with little political knowledge but a defensive ego about it. No need to be sad about this comment page.
The primary issues as others have noted is they focus on people going to the highest paying jobs without much care for morality of the jobs. Ergo they are fine being net negatives in terms of their work and philosophy.
All they do is donate money. Donations don’t fix society. Nothing changes structurally. No root problems are looked at.
They largely ignore capitalism’s faults or when I’ve seen them talk about, it’s done in a way of superficially decrying capitalist issues but then largely going along with them. Which ties into how they focus on high paying jobs regardless of morality (I’m exaggerating here but the overall point is correct).
—
HN is not intelligent when it comes to politics or the world. The avg person here is a western chauvinist with little political knowledge but a defensive ego about it. No need to be sad about this comment page.
> They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value.
Do you have examples of that? I have a different perception, most of the EAs I've met are very grounded and sharp.
For example the most recent issue of their newsletter: https://us8.campaign-archive.com/?e=7023019c13&u=52b028e7f79...
I'm not sure where there are any “hypothetical logical thought exercises” that “end up coming to insane conclusions” in there.
For the first part where you say “not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value” this is quite the opposite of my experience with them. They are very receptive to criticism and reconsider their point of view in reaction to that.
They are generally well-aware of the limits of data-driven initiatives and the dangers of indulging into purely abstract thinking that can lead to conclusions that indeed don't make sense.
Do you have examples of that? I have a different perception, most of the EAs I've met are very grounded and sharp.
For example the most recent issue of their newsletter: https://us8.campaign-archive.com/?e=7023019c13&u=52b028e7f79...
I'm not sure where there are any “hypothetical logical thought exercises” that “end up coming to insane conclusions” in there.
For the first part where you say “not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value” this is quite the opposite of my experience with them. They are very receptive to criticism and reconsider their point of view in reaction to that.
They are generally well-aware of the limits of data-driven initiatives and the dangers of indulging into purely abstract thinking that can lead to conclusions that indeed don't make sense.
The confluence of Bay Area rationalism and academic philosophy means a lot of other EA space is given to discussing hypotheticals in longwinded forum posts, blogs and papers. Some of those are well-trod utilitarian debates, others take it towards uniquely EA arguments like asserting that given that there could be as many as 10^31 future humans, essentially anything which claims to reduce existential risk - no matter how implausible the mechanism - has higher expected value than doing stuff that would certainly save human lives. An apparently completely unironic forum argument asked their fellow EAs to consider the possibility that given various heroic assumptions, the sum total of the suffering caused to mosquitos by anti-malaria nets might in fact be larger than the suffering caused by malaria they prevent. Obviously not a view shared by EAs who donate to antimalaria charities, but absolutely characteristic of the sort of knots EAs like to tie themselves in - it even has its own jokey jargon ('the rebugnant conclusion' and 'taking the train to crazy town') to describe adjacent arguments and the impulse to pursue them.
The newsletter is of course far more to the point than that, but even then you'll notice half of it is devoted to understanding the emotional state and intentions of LLMs...
It is of course entirely possible to identify as an "Effective Altruist" whilst making above-average donations to charities with rigorous efficacy metrics and otherwise being completely normal, but that's not the centre of EA debate or culture....
The newsletter is of course far more to the point than that, but even then you'll notice half of it is devoted to understanding the emotional state and intentions of LLMs...
It is of course entirely possible to identify as an "Effective Altruist" whilst making above-average donations to charities with rigorous efficacy metrics and otherwise being completely normal, but that's not the centre of EA debate or culture....
> that's not the centre of EA debate or culture....
EAs gave $1,886,513,058 through GiveWell[1], and there is 0 AI stuff in there (you can search in the linked Airtable spreadsheet).
There is also a whole movement for doing a lifetime commitment to give 10% of your earnings to charity. 9,880 people took the pledge so far[2].
[1] https://airtable.com/appGuFtOIb1eodoBu/shr1EzngorAlEzziP/tbl...
[2] https://www.givingwhatwecan.org/pledge
EAs gave $1,886,513,058 through GiveWell[1], and there is 0 AI stuff in there (you can search in the linked Airtable spreadsheet).
There is also a whole movement for doing a lifetime commitment to give 10% of your earnings to charity. 9,880 people took the pledge so far[2].
[1] https://airtable.com/appGuFtOIb1eodoBu/shr1EzngorAlEzziP/tbl...
[2] https://www.givingwhatwecan.org/pledge
Sure, but I'd say that the philosophical musings of EA's leadership, events, university outreach centres, and official forum were more representative of "the centre of EA debate or culture" than pledge signatories, even though I've got wayyy more time for stuff like the Giving Pledge.
GiveWell continues to plow its own rigorous international development-focused furrow, but it's cofounder, once noted for calling out everyone else for lack of rigour in their evidence base, has moved on to fluffy essays about how this is probably "the most important century" because it's either AI armageddon or maybe his wife's $61B startup will save us all...
GiveWell continues to plow its own rigorous international development-focused furrow, but it's cofounder, once noted for calling out everyone else for lack of rigour in their evidence base, has moved on to fluffy essays about how this is probably "the most important century" because it's either AI armageddon or maybe his wife's $61B startup will save us all...
As Adam Becker shows in his book, EAs started out being reasonable "give to charity as much as you can, and research which charities do the most good" but have gotten into absurdities like "it is more important to fund rockets than help starving people or prevent malaria because maybe an asteroid will hit the Earth, killing everyone, starving or not".
It's also not a very big leap from "My purpose is to do whatever is the greatest good" to "It doesn't matter if I hurt people as long as the overall net result is good (by some arbitrary standard)"
99% of effective altruists and rationalists agree that you shouldn't hurt people as part of some complicated scheme to do good. For example, here is eliezer yudkowsky in 2008 saying exactly that, and explaining why it's true: https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...
I believe they believe that, on its face.
I also believe that idealistic people will go to great lengths to convince themselves that their desired outcome is, in fact, the moral one. It starts by saying things like, "Well, what is harm, actually..." and then constructing a definition that supports the conclusions they've already arrived at.
I'm quite sure Sam Bankman-Fried did not believe he was harming anybody when he lost/stole/defrauded his investors and depositors' money.
I also believe that idealistic people will go to great lengths to convince themselves that their desired outcome is, in fact, the moral one. It starts by saying things like, "Well, what is harm, actually..." and then constructing a definition that supports the conclusions they've already arrived at.
I'm quite sure Sam Bankman-Fried did not believe he was harming anybody when he lost/stole/defrauded his investors and depositors' money.
Like an very old dude once said: "No one is willfully evil."
This isn’t a hypothetical leap either. This thinking directly lead to the murders committed by the zizians.
I think this is the key comment so far.
It seems very odd to criticize the group that most reliably and effectively funds global health and malaria prevention for this.
What is your alternative? What's your framework that makes you contribute to malaria prevention more or more effectively than EAs do? Or is the claim instead that people should shut down conversation within EA that strays from the EA mode?
What is your alternative? What's your framework that makes you contribute to malaria prevention more or more effectively than EAs do? Or is the claim instead that people should shut down conversation within EA that strays from the EA mode?
The simple answer is you don't need a "framework" -- plain empathy for the less fortunate is good enough. But if the EA's actually want to do something about malaria (although the Gates Foundation does much, much more in that regard than the Centre for Effective Altruism), more power to them. But as Becker notes from his visits to the Centre, things like malaria and malnutrition are not the primary focus of the centre.
EA people gave a total of $817,276,989 to malaria initiatives through GiveWell[1][2].
How much more do they need to give before you will change your mind about whether “EA's actually want to do something about malaria”?
[1] https://www.givewell.org/all-grants-fund
[2] https://airtable.com/appGuFtOIb1eodoBu/shr1EzngorAlEzziP/tbl...
How much more do they need to give before you will change your mind about whether “EA's actually want to do something about malaria”?
[1] https://www.givewell.org/all-grants-fund
[2] https://airtable.com/appGuFtOIb1eodoBu/shr1EzngorAlEzziP/tbl...
I've used GiveWell for donations and don't consider myself an Effective Altruist. Does GiveWell get to count for the just the EA community?
By analogy, if a Catholic Church created a charity for curing malaria, and I donated money to it, that wouldn't make me Catholic. But still the existence of the charity, especially if people donated over a billion dollars to it, would be a credible argument against people saying "Catholics do nothing about curing malaria". Does that make sense?
I wish it were, but it's clearly not enough. There are plenty of people with healthy emotional empathy in the world, and yet children still die of easily preventable diseases.
I am plenty happy to simp for the Gates foundation, but I think it's important to acknowledge that becoming Bill Gates to support charity is not a strategy the average person can replicate. The question for me is how do I live my life to support the causes I care about, not who lives more impressive lives than me.
I am plenty happy to simp for the Gates foundation, but I think it's important to acknowledge that becoming Bill Gates to support charity is not a strategy the average person can replicate. The question for me is how do I live my life to support the causes I care about, not who lives more impressive lives than me.
I think the group that most reliably and effectively funds global health -- at least in terms of total $ -- would be the United Nations, or perhaps the Catholic Church, or otherwise one national government or another.
If you exclude "nations" then it does look to be the Church: "The Church operates more than 140,000 schools, 10,000 orphanages, 5,000 hospitals and some 16,000 other health clinics". Caritas, the relevant charitable umbrella organization, gives $2-4b per year on its own, and that's not including the many, many operations run by religious orders not under that umbrella, or by the hundreds of thousands of parishes around the world (most of which operate charitable operations of their own).
And yet, rationalists are totally happy criticizing the Catholic Church -- not that I'm complaining, but it seems a bit hypocritical.
If you exclude "nations" then it does look to be the Church: "The Church operates more than 140,000 schools, 10,000 orphanages, 5,000 hospitals and some 16,000 other health clinics". Caritas, the relevant charitable umbrella organization, gives $2-4b per year on its own, and that's not including the many, many operations run by religious orders not under that umbrella, or by the hundreds of thousands of parishes around the world (most of which operate charitable operations of their own).
And yet, rationalists are totally happy criticizing the Catholic Church -- not that I'm complaining, but it seems a bit hypocritical.
I appreciate the good these organizations do, but I don't think that's the right measure of it. A person wouldn't in expectation serve global health better by becoming Catholic than by joining EA. That Catholicism is large isn't the same as them being effective at solving malaria. EA is tiny relative to the Church but still manages to support within an order of magnitude the funding you mentioned here, with exact numbers depending on how you count.
Similarly, it's not like government funding is an overlooked part of EA. Working on government and government aid programs is something EA talks about, high leverage areas like policy especially. If there's a more standard government role that an individual can take that has better outcomes than what EAs do, that would be an important update and I'd be interested in hearing it. But the criticism that EA is just not large enough is hard to action on, and more of a work in progress than a moral failing.
Similarly, it's not like government funding is an overlooked part of EA. Working on government and government aid programs is something EA talks about, high leverage areas like policy especially. If there's a more standard government role that an individual can take that has better outcomes than what EAs do, that would be an important update and I'd be interested in hearing it. But the criticism that EA is just not large enough is hard to action on, and more of a work in progress than a moral failing.
Rationalists and EAs spend far more time praising the Catholic Church and other religious groups than criticizing them - since they spend essentially no time criticizing them, and do occasionally praise them.
How do they escape the reality that the Earth will one day be destroyed, and that it's almost certainly impossible to ever colonize another planetary system? Just suicide out?
If you value maximizing the number of human lives that are lived, then even “almost certainly impossible” is enough to justify focusing a huge amount of effort on that. Maybe interstellar colonization is a one in a million shot, but it would multiply the number of human lives by billions or trillions or more.
Is the argument that we should try to do things that will benefit our theoretical and theoretically multitudinous descendants? Or is it that just taking action to make their existence more likely is a moral good? Because the latter is just brain dead.
Good question. I think it has to be the latter, given the immense time involved. You can make a connection between driving progress in certain areas today and increasing the odds that humanity eventually colonizes the stars. I don’t think you can make any connection with how well off those far-future humans will be.
If that's what's meant, it's a hilarious perversion of utilitarianism.
There's a lot of weird ideas on the fringes of EA...but they are on the fringes. Everyone is eye rolling at shrimp guy.
https://www.greaterwrong.com/posts/5JB4dn7hEvZCYDvCe/a-very-...
https://www.greaterwrong.com/posts/MHBJ436AiKyN83DCB/thinkin...
https://www.greaterwrong.com/posts/58ajesm2C38wi3WSJ/insect-...
https://www.greaterwrong.com/posts/5JB4dn7hEvZCYDvCe/a-very-...
https://www.greaterwrong.com/posts/MHBJ436AiKyN83DCB/thinkin...
https://www.greaterwrong.com/posts/58ajesm2C38wi3WSJ/insect-...
One example is Newcomb's problem. It presupposes a ridiculous scenario where a godlike being acts irrationally and then people try to base their life around "winning" the game that will never ever happen to them.
Having just moved to the Bay Area, I've met a few AI "safety researchers" who seems to come from this EA / Rationalist camp, and they all behave more like preachers than thinkers / academics / researchers.
I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...
I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...
> I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...
In that case I don't think you've met any of the people under discussion.
In that case I don't think you've met any of the people under discussion.
maybe, but I Stanford scientists are a different breed of rationalists, and not Rationalists
> their initial conditions were highly artificial
There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation).
There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation).
Yes, there's a name for it. They're dumbasses.
“[...] Clevinger was one of those people with lots of intelligence and no brains, and everyone knew it except those who soon found it out. In short, he was a dope." - Joseph Heller, Catch-22 https://www.goodreads.com/quotes/7522733-in-short-clevinger-...
“[...] Clevinger was one of those people with lots of intelligence and no brains, and everyone knew it except those who soon found it out. In short, he was a dope." - Joseph Heller, Catch-22 https://www.goodreads.com/quotes/7522733-in-short-clevinger-...
You mean people projecting this problem onto them with zero evidence beyond their own preconceptions and an occasional stereotype they read online?
HN should be better than this.
HN should be better than this.
Let's read the thread: "There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation)."
Can people suffer from that impairment? Is that possible? If not, please explain how wrong assumptions can be eliminated without actively looking for them. If the impairment is real, what would you call its victims? Pick your own terminology.
Can people suffer from that impairment? Is that possible? If not, please explain how wrong assumptions can be eliminated without actively looking for them. If the impairment is real, what would you call its victims? Pick your own terminology.
Name calling may be helpful in some contexts, but I doubt it's persuasive to the people in question.
I think a huge part of the subtext to this conversation is that nothing is persuasive to the people in question. How many people in this thread have come down off the mountain to explain a point of view that's just bouncing off some dude who's never needed to develop a feedback loop for his thinking?
Calling someone a dumbass in this situation is a kindness, because the assumption is that they're capable of not being one with a little self-reflection.
Calling someone a dumbass in this situation is a kindness, because the assumption is that they're capable of not being one with a little self-reflection.
Kindness means you're seeking the good of another -- it's noticing the needs of others and prioritizing their well-being.
I can't see how insulting someone pursues their well-being in any way.
I can't see how insulting someone pursues their well-being in any way.
> Calling someone a dumbass in this situation is a kindness
First they laugh...
First they laugh...
That may work for literature, but I was hoping for something more precise.
Or "garbage in, garbage out"? But that's really about computers. And maybe it is name calling. Is there no standard name for this? Or not in English?
Maybe it's similar to the "Good Student" picture. Bright within a given assignment, but taking the assignment to be immutable, or taking no interest in where the assignment comes from.
Or I've heard a saying "nobody will pay you to solve problems that they've already defined clearly."
Maybe it's similar to the "Good Student" picture. Bright within a given assignment, but taking the assignment to be immutable, or taking no interest in where the assignment comes from.
Or I've heard a saying "nobody will pay you to solve problems that they've already defined clearly."
Maybe "appeal to authority" covers it. The "rationality" or "intellect" or even "brilliance" of some claim's originator are fun things to talk about. But (even taking the claims at face value), those things aren't logically connected to whether the claim is actually true.
Bullet biting.
>They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic
I have read Effective Altruists like that. But I also remember seeing a lot of money donated to a bunch of really decent sounding causes because someone spent 5 minutes asking themselves what they wanted their donation to maximise, decided on "Lives saved" and figured out who is doing the best at that.
I have read Effective Altruists like that. But I also remember seeing a lot of money donated to a bunch of really decent sounding causes because someone spent 5 minutes asking themselves what they wanted their donation to maximise, decided on "Lives saved" and figured out who is doing the best at that.
They _are_ the effective altruism crowd.
I really appreciate this comment. Now in my 30s, I was not even aware of EA or Rationalist groups in my 20s but your description of a person high on the smell of their own farts and putting logic as king was me. The world is so chaotic that it felt good and right that I could lean on something "true" like logic. It took me to the "correct" answers according to logic but very misguided answers as I see them now. Humans are driven by emotions and the world is probabilistic so relying so heavily on logic just seems so misguided now that I'm older.
By the way, "the world is probabilistic" is one of the key lessons of the rationality community. (Infinite certainty would require infinite amount of evidence, according to the Bayes' Theorem.) When you find someone on internet trying to assign probabilities to their beliefs, chances are you have met a rationalist.
> They remind me of the "Effective Altruism" crowd
Honestly thought they were the same people
Honestly thought they were the same people
Not all rationalists are EAs, and not all EAs are rationalists, but there are many people who are both. The very idea of using evidence and numbers to choose the optimal charity has a rationalist vibe. But of course, one can donate to an effective charity without being interested in LessWrong-style rationality.
https://www.lesswrong.com/ and https://forum.effectivealtruism.org/ run on the same software, and some people have an account at both of them, but they are generally separate web communities.
https://www.lesswrong.com/ and https://forum.effectivealtruism.org/ run on the same software, and some people have an account at both of them, but they are generally separate web communities.
There is a great recent book by Adam Becker "More Everything Forever" that deals with the overlapping circles of "effective altruists", "rationalists", and "accelerationists". He's not very sympathetic to the movements as he sees them as mostly rationalizing what their adherents wanted to do anyway -- funding things like rockets and AI over feeding the needy because they see the former as helping more people in the future than dealing with real problems today.
The notion that our moral obligation somehow demands we reduce the suffering of wild animals in an ecosystem, living their lives as they have done since predation evolved and as they will do long after humans have ceased to be, is such a wild misunderstanding of who we are and what we are and what the universe is. I love my Bay Area friends. To quote the great Gwen Stefani, “This sh!t is bananas.”
I'd recommend listening to the behind the bastards episode on some of these folks, it is wild https://podcasts.apple.com/us/podcast/part-one-the-zizians-h...
I don’t have anything to add to the discussion except that EA was the very first thing that popped into my mind when I read Rationalists.
People who confuse the map for the territory.
> They remind me of the "Effective Altruism" crowd
Isn't there a lot of overlap between the two groups?
I recently read a great book that examines these various groups and their commonality: More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity by Adam Becker. Highly recommended.
Isn't there a lot of overlap between the two groups?
I recently read a great book that examines these various groups and their commonality: More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity by Adam Becker. Highly recommended.
Absolutely agree. The whole movement is unbearably naive
This is a perfect example of a Hacker News hot-take, you complain about their attitude while making zero reference to their actual accomplishments or failures. No one will downvote you for posting something factually incorrect, because you didn't post any facts at all.
For anyone else reading, a good example of what EAs do can be seen with the GiveWell charity: https://www.givewell.org/top-charities-fund
Lots of anti-malaria and vitamin stuff (as a cheap way to save lots of lives). There are also tons of EA animal charities too, such as Humane League: https://thehumaneleague.org/our-impact
For anyone else reading, a good example of what EAs do can be seen with the GiveWell charity: https://www.givewell.org/top-charities-fund
Lots of anti-malaria and vitamin stuff (as a cheap way to save lots of lives). There are also tons of EA animal charities too, such as Humane League: https://thehumaneleague.org/our-impact
A weird cult doing helpful community service doesn't cancel out the harm of them being a cult.
If they want to donate to charity, they can just donate. You don't gotta make a religion out of it.
If they want to donate to charity, they can just donate. You don't gotta make a religion out of it.
Just like the other guy, you skip the part where you actually name a harm done by them, because then people could easily shoot down your comment. Or name how EAs are a religion, I haven't even seen that troll before.
Incidentally a good book on logic is the best antidote to that type of thinking. Once you learn the difference between a valid and a sound argument and then realize just how ambiguous every English sentence is the idea that just because you have a logical argument you have something useful in everyday life becomes laughable rather quickly.
I also think the ambiguity of meaning in natural language is why statistical llms are so popular with this crowd. You don't need to think about meaning and parsing. Whatever the llm assumes is the meaning is whatever the meaning is.
I also think the ambiguity of meaning in natural language is why statistical llms are so popular with this crowd. You don't need to think about meaning and parsing. Whatever the llm assumes is the meaning is whatever the meaning is.
> Once you learn the difference between a valid and a sound argument and then realize just how ambiguous every English sentence is the idea that just because you have a logical argument you have something useful in everyday life becomes laughable rather quickly.
Ironically, that reminds me of "37 Ways That Words Can Be Wrong", written by Eliezer Yudkowsky in 2008...
Ironically, that reminds me of "37 Ways That Words Can Be Wrong", written by Eliezer Yudkowsky in 2008...
lol, I love that you identified the exact correct academic term for this phenomenon.
[deleted]
Except with Effective Altruism (EA), its not pure logic.
Logic requires properties of metaphysical objectivity.
If you use the true meaning of words it would be called irrationality, delusion, sophism, or fallacy when such things are claimed true when in fact they are false.
Logic requires properties of metaphysical objectivity.
If you use the true meaning of words it would be called irrationality, delusion, sophism, or fallacy when such things are claimed true when in fact they are false.
> They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is".
Aren't these the people who started the trend of writing things like "epistemic status: mostly speculation" on their blog posts? And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"?
Are you sure you're not painting this group with an overly-broad brush?
Aren't these the people who started the trend of writing things like "epistemic status: mostly speculation" on their blog posts? And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"?
Are you sure you're not painting this group with an overly-broad brush?
I think this is a valid point. But to some degree both can be true. I often felt when reading some of these type of texts: Wait a second, there is a wealth of thinking on these topics out there; You are not at all situating all your elaborate thinking in a broader context. And there absolutely is willingness to be challenged, and (maybe less so) a willingness to be wrong. But there also is an arrogance that "we are the ones thinking about this rationally, and we will figure this out". As if people hadn't been thinking and discussing and (verbally and literally) fighting over all sorts of adjacent and similar topics in philosophy and sociology and anthropology and ... clubs and seminars forever. And importantly maybe there also isn't as much taste for understanding the limits of vigorous discussion and rational deduction. Adorno and Horkheimer posit a dialectic of rationality and enlightenment, Habermas tries to rebuild rational discourse by analyzing its preconditions. Yet for all the vigorous intellectualism of the rationalists, none of that ever seems to feature even in passing (maybe I have simply missed it...).
And I have definitely encountered "if you just listen to me properly you will understand that I am right, because I have derived my conclusions rationally" in in person interactions.
On the balance I'd rather have some arrogance and willingness to be debated and be wrong, over a timid need to defer to centuries of established thought though. The people I've met in person I've always been happy to hang out with and talk to.
And I have definitely encountered "if you just listen to me properly you will understand that I am right, because I have derived my conclusions rationally" in in person interactions.
On the balance I'd rather have some arrogance and willingness to be debated and be wrong, over a timid need to defer to centuries of established thought though. The people I've met in person I've always been happy to hang out with and talk to.
That's a fair point. Speaking only for myself, I think I fail to understand why it's important to situate philosophical discussions in the context of all the previous philosophers who have expressed related ideas, rather than simply discussing the ideas in isolation.
I remember as a child coming to the same "if reality is a deception, at least I must exist to be deceived" conclusion that Descartes did, well before I had heard of Descartes. (I don't think this makes me special, it's just a natural conclusion anyone will reach if they ponder the subject). I think it's harmless for me to discuss that idea in public without someone saying "you need to read Descartes before you can talk about this".
I also find my personal ethics are stronly aligned with what Kant espoused. But most people I talk to are not academic philosophers and have not read Kant, so when I want to explain my morals, I am better off explaining the ideas themselves than talking about Kant, which would be a distraction anyway because I didn't learn them from Kant, we just arrived at the same conclusions. If I'm talking with a philosopher I can just say "I'm a Kantian" as shorthand, but that's really just jargon for people who already know what I'm talking about.
I also think that while it would be unusual for someone to (for example) write a guide to understanding relativity without once mentioning Einstein, it also wouldn't be a fundamental flaw.
(But I agree there's no certainly excuse for someone asserting that they're right because they're rational!)
I remember as a child coming to the same "if reality is a deception, at least I must exist to be deceived" conclusion that Descartes did, well before I had heard of Descartes. (I don't think this makes me special, it's just a natural conclusion anyone will reach if they ponder the subject). I think it's harmless for me to discuss that idea in public without someone saying "you need to read Descartes before you can talk about this".
I also find my personal ethics are stronly aligned with what Kant espoused. But most people I talk to are not academic philosophers and have not read Kant, so when I want to explain my morals, I am better off explaining the ideas themselves than talking about Kant, which would be a distraction anyway because I didn't learn them from Kant, we just arrived at the same conclusions. If I'm talking with a philosopher I can just say "I'm a Kantian" as shorthand, but that's really just jargon for people who already know what I'm talking about.
I also think that while it would be unusual for someone to (for example) write a guide to understanding relativity without once mentioning Einstein, it also wouldn't be a fundamental flaw.
(But I agree there's no certainly excuse for someone asserting that they're right because they're rational!)
It may be easier to imagine someone trying to derive mathematics all by themselves, since it's less abstract. It's not that they won't come up with anything, it's that everything that even a genius can come up with in their lifetime will be something that the whole of humanity has long since come up with, chewed over, simplified, had a rebellion against, had a counter-rebellion against the rebellion, and ultimately packaged it up in a highly efficient manner into a textbook with cross-references to all sorts of angles on it and dozens of elaborations. You can't possible get through all this stuff all on your own.
The problem is less clear in philosophy than mathematics, but it's still there. It's really easy on your own terms to come up with some idea that the collective intelligence has already revealed to be fatally flawed in some undeniable manner, or at the very least, has very powerful arguments against it that an individual may never consider. The ideas that have survived decades, centuries, and even millenia against the collective weight of humanity assaulting them are going to have a certain character that "something someone came up with last week" will lack.
(That said I am quite heterodox in one way, which is that I'm not a big believer in reading primary sources, at least routinely. Personally I think that a lot of the primary sources noticeably lack the refinement and polish added as humanity chews it over and processes it and I prefer mostly pulling from the result of the process, and not from the one person who happened to introduce a particular idea. Such a source may be interesting for other reasons, but not in my opinion for philosophy.)
The problem is less clear in philosophy than mathematics, but it's still there. It's really easy on your own terms to come up with some idea that the collective intelligence has already revealed to be fatally flawed in some undeniable manner, or at the very least, has very powerful arguments against it that an individual may never consider. The ideas that have survived decades, centuries, and even millenia against the collective weight of humanity assaulting them are going to have a certain character that "something someone came up with last week" will lack.
(That said I am quite heterodox in one way, which is that I'm not a big believer in reading primary sources, at least routinely. Personally I think that a lot of the primary sources noticeably lack the refinement and polish added as humanity chews it over and processes it and I prefer mostly pulling from the result of the process, and not from the one person who happened to introduce a particular idea. Such a source may be interesting for other reasons, but not in my opinion for philosophy.)
Well, sure, but mathematics is the domain for which this holds maybe the most true out of any. It's less true for fields which are not as old.
I'm not sure if this counterpoint generalizes entirely to the original critique, since certainly LessWrongers aren't usually posting about or discussing math as if they've discovered it-- usually substantially more niche topics.
I'm not sure if this counterpoint generalizes entirely to the original critique, since certainly LessWrongers aren't usually posting about or discussing math as if they've discovered it-- usually substantially more niche topics.
... philosophy is if anything older than what we call mathematics.
I suppose you're right about that, so I can't make the argument go through by saying "mathematics" vs "philosophy". Maybe what I should say instead is that as some dialectics advance/technologies develop, subfields of both such things sprout up and have a lot of low-hanging fruit to pick, and in these cases, the new work will be descended from but not too-essentially informed by the prior work.
Like mathematical logic (in the intersection of math and philosophy) didn't have that many true predecessors and was developed very far by maybe only 5-10 individuals cumulatively, or information theory was basically established by Claude Shannon and maybe two other guys, or various aspects of convex optimization or Fourier analysis were only developed in the 80s or so, it stands to reason that the AI-related applications of various aspects of philosophy are ripe to be developed now. (By contrast, we don't see, as much, people on LW trying to redo linear algebra from the ground up, nor more "mature" aspects of philosophy.)
(If anything, I think it's more feasible than ever before, also, for a bunch of relative amateurs to non-professionally make real intellectual contributions, like noticeably moreso than 100 or even 20 years ago. That's what increasing the baseline levels of education/wealth/exposure to information was intended to achieve, on some level, isn't it?)
Like mathematical logic (in the intersection of math and philosophy) didn't have that many true predecessors and was developed very far by maybe only 5-10 individuals cumulatively, or information theory was basically established by Claude Shannon and maybe two other guys, or various aspects of convex optimization or Fourier analysis were only developed in the 80s or so, it stands to reason that the AI-related applications of various aspects of philosophy are ripe to be developed now. (By contrast, we don't see, as much, people on LW trying to redo linear algebra from the ground up, nor more "mature" aspects of philosophy.)
(If anything, I think it's more feasible than ever before, also, for a bunch of relative amateurs to non-professionally make real intellectual contributions, like noticeably moreso than 100 or even 20 years ago. That's what increasing the baseline levels of education/wealth/exposure to information was intended to achieve, on some level, isn't it?)
Not if they refuse to read all the existing information because they think they came up with it all.
Just teasing here, but I presume you’re a social conservative…? ;)
Generally conservatives only want to conserve the way the world is as they grow up. Whether that involves respect for prior ideas and thought depends on their current ideas - hence the book burning.
Did you discover it from first principles by yourself because it's a natural conclusion anyone would reach if they ponder the subject?
Or because western culture reflects this theme continuously through all the culture and media you've immersed in since you were a child?
Also the idea is definitely not new to Descartes, you can find echoes of it going back to Plato, so your idea isn't wrong per se. But I think it underrates the effect to which our philosophical preconceptions are culturally constructed.
Or because western culture reflects this theme continuously through all the culture and media you've immersed in since you were a child?
Also the idea is definitely not new to Descartes, you can find echoes of it going back to Plato, so your idea isn't wrong per se. But I think it underrates the effect to which our philosophical preconceptions are culturally constructed.
Because it's a natural conclusion anyone would reach if they ponder the subject. Sorry, I thought I expressed that opinion clearly. I don't think I was exposed to that idea through media by that age.
Odds are good that the millions of people who have also read and considered these ideas have added to what you came up with at 6. Odds are also high that people who have any interest in the topic will probably learn more by reading Descartes and Kant and the vast range of well written educational materials explaining their thoughts at every level. So if you find yourself telling people about these ideas frequently enough to have opinions on how they respond, you are doing both yourself and them a disservice by not bothering to learn how the ideas have already been criticized and extended.
There’s definitely a tension between having a low tolerance for crankery and being open to fresh perspectives. If I’m being charitable to the critics of Rationalism (big “r”), I suppose that they have encountered arguments from Rationalists that struck them as wrong specifically in a way that would have been avoided if the person making the argument had read any of the relevant literature.
How could you say that your views are aligned with those of Descartes and Kant if you have not seriously engaged with their works and what others have written about them?
All serious works in philosophy (Kant especially) are subject to interpretation. Whole research programmes exist around the works of major philosophers, interpreting and building on their works.
One cannot really do justice to e.g. the Critique of Pure Reason by discussing it based on a high level summary of the “main ideas” contained in it. These works have had a major impact on the history of Western philosophy and were groundbreaking at the time (and still are).
All serious works in philosophy (Kant especially) are subject to interpretation. Whole research programmes exist around the works of major philosophers, interpreting and building on their works.
One cannot really do justice to e.g. the Critique of Pure Reason by discussing it based on a high level summary of the “main ideas” contained in it. These works have had a major impact on the history of Western philosophy and were groundbreaking at the time (and still are).
I think they basically agree with your point here -- they mention Descartes and Kant to say roughly "I hold basically these ideas, but I don't mention the philosophers' names when I talk about them because 1) I came to them independently, and 2) the people I'm talking to are not familiar with the context so situating our conversation there isn't helpful." Their argument is that you can have philosophical conversations without relying on the context of the canon, and that in a first-level discussion they wouldn't bring up Descartes or Kant.
Did I give you the impression from my comment that I haven't read Descartes and Kant? That's not what I intended to say.
Here's a very simple explanation as to why it's helpful from a "first principles" style analogy.
Suppose a foot race. Choose two runners of equal aptitude and finite existence. Start one at mile 1 and one at mile 100. Who do you think will get farther?
Not to mention, engaging in human community and discourse is a big part of what it means to be human. Knowledge isn't personal or isolated, we build it together. The "first principles people" understand this to the extent that they have even built their own community of like minded explorers, problem is, a big part of this bond is their choice to be willfully ignorant of large swaths of human intellectual development. Not only is this stupid, it also is a great disservice to your forebears, who worked just as hard to come to their conclusions and who have been building up the edifice of science bit by bit. It's completely antithetical to the spirit of scientific endeavor.
Suppose a foot race. Choose two runners of equal aptitude and finite existence. Start one at mile 1 and one at mile 100. Who do you think will get farther?
Not to mention, engaging in human community and discourse is a big part of what it means to be human. Knowledge isn't personal or isolated, we build it together. The "first principles people" understand this to the extent that they have even built their own community of like minded explorers, problem is, a big part of this bond is their choice to be willfully ignorant of large swaths of human intellectual development. Not only is this stupid, it also is a great disservice to your forebears, who worked just as hard to come to their conclusions and who have been building up the edifice of science bit by bit. It's completely antithetical to the spirit of scientific endeavor.
It really depends on why you are having a philosophical discussion. If you are talking among friends, or just because you want to throw interesting ideas around, sure! Be free, have fun.
I come from a physics background. We used to (and still) have a ton of physicists who decide to dable in a new field, secure in their knowledge that they are smarter than the people doing it, and that anything worthwhile that has already been thought of they can just rederive ad hoc when needed (economists are the only other group that seems to have this tendency...) [1]. It turned out every time that the people who had spent decades working on, studying, discussing and debating the field in question had actually figured important shit out along the way. They might not have come with the mathematical toolbox that physicists had, and outside perspectives that challenge established thinking to prove itself again can be valuable, but when your goal is to actually understand what's happening in the real world, you can't ignore what's been done.
[1] There even is an xkcd about this:
https://xkcd.com/793/
I come from a physics background. We used to (and still) have a ton of physicists who decide to dable in a new field, secure in their knowledge that they are smarter than the people doing it, and that anything worthwhile that has already been thought of they can just rederive ad hoc when needed (economists are the only other group that seems to have this tendency...) [1]. It turned out every time that the people who had spent decades working on, studying, discussing and debating the field in question had actually figured important shit out along the way. They might not have come with the mathematical toolbox that physicists had, and outside perspectives that challenge established thinking to prove itself again can be valuable, but when your goal is to actually understand what's happening in the real world, you can't ignore what's been done.
[1] There even is an xkcd about this:
https://xkcd.com/793/
Also http://www.smbc-comics.com/comic/2012-03-21 (I am a physicist myself)
> But there also is an arrogance that "we are the ones thinking about this rationally, and we will figure this out". As if people hadn't been thinking and discussing and (verbally and literally) fighting over all sorts of adjacent and similar topics in philosophy and sociology and anthropology and ... clubs and seminars forever
This is a feature, not a bug, for writers who hold an opinion on something and want to rationalize it.
So many of the rationalist posts I've read through the years come from someone who has an opinion or gut feeling about something, but they want it to be seen as something more rigorous. The "first principles" writing style is a license to throw out the existing research on the topic, including contradictory evidence, and construct an all new scaffold around their opinion that makes it look more valid.
I use the "SlimeTimeMoldTime - A Chemical Hunger" blog series as an example because it was so widely shared and endorsed in the rationalist community: https://slimemoldtimemold.com/2021/07/07/a-chemical-hunger-p... It even received a financial grant from Scott Alexander of Astral Codex Ten
Actual experts were discrediting the series from the first blog post and explaining all of the author's errors, but the community soldiered on with it anyway, eventually making the belief that lithium in the water supply was causing the obesity epidemic into a meme within the rationalist community. There's no evidence supporting this and countless take-downs of how the author misinterpreted or cherry-picked data, but because it was written with the rationalist style and given the implicit blessing of a rationalist figurehead it was adopted as ground truth by many for years. People have been waking up to issues with the series for a while now, but at the time it was remarkable how quickly the idea spread as if it was a true, novel discovery.
This is a feature, not a bug, for writers who hold an opinion on something and want to rationalize it.
So many of the rationalist posts I've read through the years come from someone who has an opinion or gut feeling about something, but they want it to be seen as something more rigorous. The "first principles" writing style is a license to throw out the existing research on the topic, including contradictory evidence, and construct an all new scaffold around their opinion that makes it look more valid.
I use the "SlimeTimeMoldTime - A Chemical Hunger" blog series as an example because it was so widely shared and endorsed in the rationalist community: https://slimemoldtimemold.com/2021/07/07/a-chemical-hunger-p... It even received a financial grant from Scott Alexander of Astral Codex Ten
Actual experts were discrediting the series from the first blog post and explaining all of the author's errors, but the community soldiered on with it anyway, eventually making the belief that lithium in the water supply was causing the obesity epidemic into a meme within the rationalist community. There's no evidence supporting this and countless take-downs of how the author misinterpreted or cherry-picked data, but because it was written with the rationalist style and given the implicit blessing of a rationalist figurehead it was adopted as ground truth by many for years. People have been waking up to issues with the series for a while now, but at the time it was remarkable how quickly the idea spread as if it was a true, novel discovery.
I don't read HN or LW all that often, but FWIW I actually learned about SlimeMoldTimeMold's "A Chemical Hunger" series from HN and then read its most famous takedown from LessWrong: https://www.lesswrong.com/posts/7iAABhWpcGeP5e6SB/it-s-proba... (I don't remember any detailed takedowns of SlimeMoldTimeMold coming before that article, but maybe there are).
I think that SlimeMoldTimeMold's rise and fall was actually a pretty big point in favor of the "rationalist community".
I think that SlimeMoldTimeMold's rise and fall was actually a pretty big point in favor of the "rationalist community".
> I think that SlimeMoldTimeMold's rise and fall was actually a pretty big point in favor of the "rationalist community".
That feels like revisionist history to me. It rose to fame in LessWrong and SlateStarCodex, was promoted by Yudkowski, and proliferated for about a year and half before the takedowns finally got traction.
While it was the topic du jour in the rationalist spaces it was very difficult to argue against. I vividly remember how hard it was to convince anyone that SMTM wasn't a good source at the time, because so many people saw Yudkowski endorse it, saw Scott Alexander give it a shout out, and so on.
Now Yudkowski has gone back and edited his old endorsement, it has disappeared from the discourse, and many want to pretend the whole episode never happened.
> (I don't remember any detailed takedowns of SlimeMoldTimeMold coming before that article, but maybe there are).
Exactly my point. It was criticized widely outside of the rationalist community, but the takedowns were all dismissed because they weren't properly rationalist-coded. It finally took someone writing it up in the form of rationalist rhetoric and seeding it into LessWrong to break the spell.
This is the trend with rationalist-centric contrarianism: You have to code your articles with the correct prose, structure, and signs to get uptake in the rationalist community. Once you see it, it's hard to miss.
That feels like revisionist history to me. It rose to fame in LessWrong and SlateStarCodex, was promoted by Yudkowski, and proliferated for about a year and half before the takedowns finally got traction.
While it was the topic du jour in the rationalist spaces it was very difficult to argue against. I vividly remember how hard it was to convince anyone that SMTM wasn't a good source at the time, because so many people saw Yudkowski endorse it, saw Scott Alexander give it a shout out, and so on.
Now Yudkowski has gone back and edited his old endorsement, it has disappeared from the discourse, and many want to pretend the whole episode never happened.
> (I don't remember any detailed takedowns of SlimeMoldTimeMold coming before that article, but maybe there are).
Exactly my point. It was criticized widely outside of the rationalist community, but the takedowns were all dismissed because they weren't properly rationalist-coded. It finally took someone writing it up in the form of rationalist rhetoric and seeding it into LessWrong to break the spell.
This is the trend with rationalist-centric contrarianism: You have to code your articles with the correct prose, structure, and signs to get uptake in the rationalist community. Once you see it, it's hard to miss.
> It was criticized widely outside of the rationalist community, but the takedowns were all dismissed because they weren't properly rationalist-coded.
Do you have any examples of this that predate that LW article? Ideally both the critique and its dismissal but just the critique would be great. The original HN submission had a few comments critiquing it but I didn't see anything in depth (or for that matter as strident).
Do you have any examples of this that predate that LW article? Ideally both the critique and its dismissal but just the critique would be great. The original HN submission had a few comments critiquing it but I didn't see anything in depth (or for that matter as strident).
Where is the evidence that Yudkowsky edited an old post?
Is this "evidence" one of those silly things that rationalists so much love to talk about?
Don't worry, HN commenters can figure out the truth about Yudkowsky's articles from the first principles. They have already figured out that EAs no longer care about curing malaria, despite https://www.givewell.org/charities/top-charities only being a Google search away.
At the end, they will give you a lecture about how everyone hates people who are smug and talk about things they have no clue about. The lecture will then get a lot of upvotes.
Don't worry, HN commenters can figure out the truth about Yudkowsky's articles from the first principles. They have already figured out that EAs no longer care about curing malaria, despite https://www.givewell.org/charities/top-charities only being a Google search away.
At the end, they will give you a lecture about how everyone hates people who are smug and talk about things they have no clue about. The lecture will then get a lot of upvotes.
FWIW, I detest rationalist ideology. I was just curious if it was true, which post, and what was the edit.
Careful, curiosity about evidence is a gateway drug to rationality. ;)
I wish more people who detest rationalist ideology were curious about actual evidence.
I wish more people who detest rationalist ideology were curious about actual evidence.
What a ridiculous statement. As if captial-r Rationalists are the only people who care about evidence.
Did you end up presenting the evidence? I'm following the discussion a few days too late, so my apologies if you've already linked to the evidence and ended up deleting it after.
It was Aurornis who made the claim "Yudkowski has gone back and edited his old endorsement" that EnPissant asked evidence for. And nope, he didn't receive it.
Similarly, Aurornis made a claim that "Scott Alexander predicted at least $250 million in damages from Black Lives Matter protests", when if fact (as the very link provided by Aurornis shows) Scott predicted that the probability of such thing happening was 30%, i.e. it's more likely not to happen.
Elsewhere in this thread, another user, tptacek, claims that "Scott Alexander published some of his best-known posts under his own name". When I asked him for evidence, he said "I know more about this than you, and I'm not invested in this discussion enough to educate you adversarially". Translated: no evidence provided.
.
From my perspective, this all kinda proves my point.
Is the rationality community the only place where people care about evidence? Of course not.
But is the rationality community a rare place where people can ask for evidence in an informal debate and reasonably expect to actually get it? Unfortunately, I think the evidence we got here points towards yes.
Hacker News is a website mostly visited by smart people who are curious about many things. They are even smart enough to notice that some claims are suspicious, and ask for evidence. But will they receive it? No, they usually won't.
And in the next debate on the same topic, most likely the same false claims will be made again, maybe by people who have learned them in this thread. And the claims will be upvoted again.
This is an aspect where the rationality community strives to do better. It is not about some people being smarter than others, or whatever accusations are typically made. It is about establishing social norms where people e.g. don't get upvoted for making unsubstantiated negative claims about someone they don't like, without being asked to back it up, or get downvoted.
Similarly, Aurornis made a claim that "Scott Alexander predicted at least $250 million in damages from Black Lives Matter protests", when if fact (as the very link provided by Aurornis shows) Scott predicted that the probability of such thing happening was 30%, i.e. it's more likely not to happen.
Elsewhere in this thread, another user, tptacek, claims that "Scott Alexander published some of his best-known posts under his own name". When I asked him for evidence, he said "I know more about this than you, and I'm not invested in this discussion enough to educate you adversarially". Translated: no evidence provided.
.
From my perspective, this all kinda proves my point.
Is the rationality community the only place where people care about evidence? Of course not.
But is the rationality community a rare place where people can ask for evidence in an informal debate and reasonably expect to actually get it? Unfortunately, I think the evidence we got here points towards yes.
Hacker News is a website mostly visited by smart people who are curious about many things. They are even smart enough to notice that some claims are suspicious, and ask for evidence. But will they receive it? No, they usually won't.
And in the next debate on the same topic, most likely the same false claims will be made again, maybe by people who have learned them in this thread. And the claims will be upvoted again.
This is an aspect where the rationality community strives to do better. It is not about some people being smarter than others, or whatever accusations are typically made. It is about establishing social norms where people e.g. don't get upvoted for making unsubstantiated negative claims about someone they don't like, without being asked to back it up, or get downvoted.
*Yudkowsky.
You're spot on here, and I think this is probably also why they appeal to programmers and people in software.
I find a lot of people in software have an insufferable tendency to simply ignore entire bodies of prior art, prior research, etc. outside of maybe computer science (and even that can be rare), and yet they act as though they are the most studied participants in the subject, proudly proclaiming their "genius insights" that are essentially restatements of basic facts in any given field that they would have learned if they just bothered to, you know, actually do research and put aside their egos for half a second to wonder if maybe the eons of human activity prior to their precious existence might have led to some decent knowledge.
I find a lot of people in software have an insufferable tendency to simply ignore entire bodies of prior art, prior research, etc. outside of maybe computer science (and even that can be rare), and yet they act as though they are the most studied participants in the subject, proudly proclaiming their "genius insights" that are essentially restatements of basic facts in any given field that they would have learned if they just bothered to, you know, actually do research and put aside their egos for half a second to wonder if maybe the eons of human activity prior to their precious existence might have led to some decent knowledge.
Yeah, though I think you may be exaggerating how often the "genius insights" rise to the level of correct restatements of basic facts. That happens, but it's not the rule.
This is old engineer / old physicist syndrome.
https://www.smbc-comics.com/?id=2556
https://www.smbc-comics.com/?id=2556
I grew up with some friends who were deep into the early roots of online rationalism, even slightly before LessWrong came online. I've been around long enough to recognize the rhetorical devices used in rationalist writings:
> Aren't these the people who started the trend of writing things like "epistemic status: mostly speculation" on their blog posts? And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"?
There's a lot of in-group signaling in rationalist circles like the "epistemic status" taglines, posting predictions, and putting your humility on show.
This has come full-circle, though, and now rationalist writings are generally pre-baked with hedging, both-sides takes, escape hatches, and other writing tricks that make it easier to claim they weren't entirely wrong in the future.
A perfect exaple is the recent "AI 2027" doomsday scenario that predicts a rapid escalation of AI superpowers followed by disaster in only a couple years: https://ai-2027.com/
If you read the backstory and supporting blog posts from the authors they are filled to the brim with hedges and escape hatches. Scott Alexander wrote that it was something like "the 80th percentile of their fast scenario", which means when it fails to come true he can simple say it wasn't actually his median prediction anyway and that they were writing about the fast scenario. I can already predict that the "We were wrong" article will be more about what they got right with a heavy emphasis on the fact that it wasn't their real median prediction anyway.
I think this group relies heavily on the faux-humility and hedging because they've recognized how powerful it is to get people to trust them. Even the comment above is implying that because they say and do these things, they must be immune from the criticism delivered above. That's exactly why they wrap their posts in these signals, before going on to do whatever they were going to do anyway.
> Aren't these the people who started the trend of writing things like "epistemic status: mostly speculation" on their blog posts? And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"?
There's a lot of in-group signaling in rationalist circles like the "epistemic status" taglines, posting predictions, and putting your humility on show.
This has come full-circle, though, and now rationalist writings are generally pre-baked with hedging, both-sides takes, escape hatches, and other writing tricks that make it easier to claim they weren't entirely wrong in the future.
A perfect exaple is the recent "AI 2027" doomsday scenario that predicts a rapid escalation of AI superpowers followed by disaster in only a couple years: https://ai-2027.com/
If you read the backstory and supporting blog posts from the authors they are filled to the brim with hedges and escape hatches. Scott Alexander wrote that it was something like "the 80th percentile of their fast scenario", which means when it fails to come true he can simple say it wasn't actually his median prediction anyway and that they were writing about the fast scenario. I can already predict that the "We were wrong" article will be more about what they got right with a heavy emphasis on the fact that it wasn't their real median prediction anyway.
I think this group relies heavily on the faux-humility and hedging because they've recognized how powerful it is to get people to trust them. Even the comment above is implying that because they say and do these things, they must be immune from the criticism delivered above. That's exactly why they wrap their posts in these signals, before going on to do whatever they were going to do anyway.
Yes, I do think that these hedging statements make them immune from the specific criticism that I quoted.
If you want to say their humility is not genuine, fine. I'm not sure I agree with it, but you are entitled to that view. But to simultaneously be attacking the same community for not ever showing a sense of maybe being wrong or uncertain, and also for expressing it so often it's become an in-group signal, is just too much cognitive dissonance.
If you want to say their humility is not genuine, fine. I'm not sure I agree with it, but you are entitled to that view. But to simultaneously be attacking the same community for not ever showing a sense of maybe being wrong or uncertain, and also for expressing it so often it's become an in-group signal, is just too much cognitive dissonance.
> es, I do think that these hedging statements make them immune from the specific criticism that I quoted.
That's my point: Their rhetorical style is interpreted by the in-group as a sort of weird infallibility. Like they've covered both sides and therefore the work is technically correct in all cases. Once they go through the hedging dance, they can put forth the opinion-based point they're trying to make in a very persuasive way, falling back to the hedging in the future if it turns out to be completely wrong.
The writing style looks different depending on where you stand: Reading it in the forward direction makes it feel like the main point is very likely. Reading it in the backward direction you notice the hedging and decide they were also correct. Yet at the time, the rationalist community attaches themselves to the position being pushed.
> But to simultaneously be attacking the same community for not ever showing a sense of maybe being wrong or uncertain, and also for expressing it so often it's become an in-group signal, is just too much cognitive dissonance.
That's a strawman argument. At no point did I "attack the community for not ever showing a sense of maybe being wrong or uncertain".
That's my point: Their rhetorical style is interpreted by the in-group as a sort of weird infallibility. Like they've covered both sides and therefore the work is technically correct in all cases. Once they go through the hedging dance, they can put forth the opinion-based point they're trying to make in a very persuasive way, falling back to the hedging in the future if it turns out to be completely wrong.
The writing style looks different depending on where you stand: Reading it in the forward direction makes it feel like the main point is very likely. Reading it in the backward direction you notice the hedging and decide they were also correct. Yet at the time, the rationalist community attaches themselves to the position being pushed.
> But to simultaneously be attacking the same community for not ever showing a sense of maybe being wrong or uncertain, and also for expressing it so often it's become an in-group signal, is just too much cognitive dissonance.
That's a strawman argument. At no point did I "attack the community for not ever showing a sense of maybe being wrong or uncertain".
> They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know"
edit: my apologies, that was someone else in the thread. I do feel like between the two comments though there is a "damned if you do, damned if you don't". (The original quote above I found absurd upon reading it.)
edit: my apologies, that was someone else in the thread. I do feel like between the two comments though there is a "damned if you do, damned if you don't". (The original quote above I found absurd upon reading it.)
Haha my thoughts exactly. This HN thread is simultaneously criticizing them for being too assured, not considering other possibilities, and hedging that they may not be right and other plausibilities exist.
“Damned if you do” indeed…
“Damned if you do” indeed…
> "damned if you do, damned if you don't"
This is right, but doesn't actually cover all the options. It's damned if you [write confidently about something and] do or don't [hedge with a probability or "epistemic status"].
But the other option, which is the one the vast majority of people choose, is to not write confidently about everything.
It's fine, there are far worse sins than writing persuasively about tons of stuff and inevitably getting lots of it wrong. But it's absolutely reasonable to criticize this choice, irregardless of the level of hedging.
This is right, but doesn't actually cover all the options. It's damned if you [write confidently about something and] do or don't [hedge with a probability or "epistemic status"].
But the other option, which is the one the vast majority of people choose, is to not write confidently about everything.
It's fine, there are far worse sins than writing persuasively about tons of stuff and inevitably getting lots of it wrong. But it's absolutely reasonable to criticize this choice, irregardless of the level of hedging.
Well, on a meta level, I think their community has decided that in general it's better to post (and subsequently be able to discuss) ideas that one is not yet very confident about, and ideally that's what the "epistemic status" markers are supposed to indicate to the reader.
They can't really be blamed for the fact that others go on to take the ideas more seriously than they intended.
(If anything, I think that at least in person, most rationalists are far less confident and far less persuasive than the typical person in proportion to the amount of knowledge/expertise/effort they have on a given topic, particularly in a professional setting, and they would all be well-served to do at least a normal human amount of "write and explain persuasively rather than as a mechanical report of the facts as you see them".)
(Also, with all communities there will be the more serious and dedicated core of the people, and then those who sort of cargo-cult or who defer much, or at least some, of their thinking to members with more status. This is sort of unavoidable on multiple levels-- for one, it's quite a reasonable thing to do with the amount of information out there, and for another, communities are always comprised of people with varying levels of seriousness, sincere people and grifters, careful thinkers and less careful thinkers, etc. (see mobs-geeks-sociopaths))
(Obviously even with these caveats there are exceptions to this statement, because society is complex and something about propaganda and consequentialism.)
Alternately, I wonder if you think there might be a better way of "writing unconfidently", like, other than not writing at all.
They can't really be blamed for the fact that others go on to take the ideas more seriously than they intended.
(If anything, I think that at least in person, most rationalists are far less confident and far less persuasive than the typical person in proportion to the amount of knowledge/expertise/effort they have on a given topic, particularly in a professional setting, and they would all be well-served to do at least a normal human amount of "write and explain persuasively rather than as a mechanical report of the facts as you see them".)
(Also, with all communities there will be the more serious and dedicated core of the people, and then those who sort of cargo-cult or who defer much, or at least some, of their thinking to members with more status. This is sort of unavoidable on multiple levels-- for one, it's quite a reasonable thing to do with the amount of information out there, and for another, communities are always comprised of people with varying levels of seriousness, sincere people and grifters, careful thinkers and less careful thinkers, etc. (see mobs-geeks-sociopaths))
(Obviously even with these caveats there are exceptions to this statement, because society is complex and something about propaganda and consequentialism.)
Alternately, I wonder if you think there might be a better way of "writing unconfidently", like, other than not writing at all.
Yeah I think you're getting at what my skepticism stems from: The article with the 55% certain epistemic status and the article with the 95% certain epistemic status are both written with equal persuasive oomph.
In most writing, people write less persuasively on topics they have less conviction in.
In most writing, people write less persuasively on topics they have less conviction in.
> That's a strawman argument. At no point did I "attack the community for not ever showing a sense of maybe being wrong or uncertain".
Ok, let's scroll up the thread. When I refer to "the specific criticism that I quoted", and when you say "implying that because they say and do these things, they must be immune from the criticism delivered above": what do you think was the "criticism delivered above"? Because I thought we were talking about contrarian1234's claim to exactly this "strawman", and you so far have not appeared to not agree with me that this criticism was invalid.
Ok, let's scroll up the thread. When I refer to "the specific criticism that I quoted", and when you say "implying that because they say and do these things, they must be immune from the criticism delivered above": what do you think was the "criticism delivered above"? Because I thought we were talking about contrarian1234's claim to exactly this "strawman", and you so far have not appeared to not agree with me that this criticism was invalid.
If putting up evidence about how people were wrong in their predictions, I suggest actually pointing at predictions that were wrong, rather than on recent predictions about the future that that you disagree over how they will resolve. If putting up evidence about how people make excuses for failing predictions, I suggest actually showing them do so, rather than projecting that they will do so and blaming them for your projection.
It's been a while since I've engaged in rationalist debates, so I forgot about the slightly condescending, lecturing tone that comes out when you disagree with rationalist figureheads. :) You could simply ask "Can you provide examples" instead of the "If you ____ then I suggest ____" form.
My point wasn't to nit-pick individual predictions, it was a general explanation of how the game is played.
Since Scott Alexander comes up a lot, a few randomly selected predictions that didn't come true:
- He predicted at least $250 million in damages from Black Lives Matter protests.
- He predicted Andrew Yang would win the 2021 NYC mayoral race with 80% certainty (he came in 4th place)
- He gave a 70% chance to Vitamin D being generally recognized as a good COVID treatment
This is just random samples from the first blog post that popped in Google: https://www.astralcodexten.com/p/grading-my-2021-predictions
It's also noteworthy to read that a lot of his predictions are about his personal life, his own blogging actions, or [redacted] things. These all get mixed in with a small number of geopolitical, economic, and medical predictions with the net result of bringing his overall accuracy up.
My point wasn't to nit-pick individual predictions, it was a general explanation of how the game is played.
Since Scott Alexander comes up a lot, a few randomly selected predictions that didn't come true:
- He predicted at least $250 million in damages from Black Lives Matter protests.
- He predicted Andrew Yang would win the 2021 NYC mayoral race with 80% certainty (he came in 4th place)
- He gave a 70% chance to Vitamin D being generally recognized as a good COVID treatment
This is just random samples from the first blog post that popped in Google: https://www.astralcodexten.com/p/grading-my-2021-predictions
It's also noteworthy to read that a lot of his predictions are about his personal life, his own blogging actions, or [redacted] things. These all get mixed in with a small number of geopolitical, economic, and medical predictions with the net result of bringing his overall accuracy up.
Am I reading all of these backwards? You say
> He predicted at least $250 million in damages from Black Lives Matter protests.
He says
> 5. At least $250 million in damage from BLM protests this year: 30%
which, by my reading means he assigns it greater-than-even odds that _less_ than $250 million dollars in damages happened (I have no understanding of whether or not this result is the case, but my reading of your post suggests that you believe that this was indeed the outcome).
You say > He gave a 70% chance to Vitamin D being generally recognized as a good COVID treatment
while he says > Vitamin D is _not_ generally recognized (eg NICE, UpToDate) as effective COVID treatment: 70% (emphasis mine)
> He predicted at least $250 million in damages from Black Lives Matter protests.
He says
> 5. At least $250 million in damage from BLM protests this year: 30%
which, by my reading means he assigns it greater-than-even odds that _less_ than $250 million dollars in damages happened (I have no understanding of whether or not this result is the case, but my reading of your post suggests that you believe that this was indeed the outcome).
You say > He gave a 70% chance to Vitamin D being generally recognized as a good COVID treatment
while he says > Vitamin D is _not_ generally recognized (eg NICE, UpToDate) as effective COVID treatment: 70% (emphasis mine)
For what it's worth, your comments in this thread have been very good descriptions of things I became frustrated with after once being quite interested / enthralled with this community / movement!
(I feel like you're probably getting upvotes from people who feel similarly, but sometimes I feel like nobody ever writes "I agree with you" comments, so the impression is that there's only disagreement with some point being made.)
(I feel like you're probably getting upvotes from people who feel similarly, but sometimes I feel like nobody ever writes "I agree with you" comments, so the impression is that there's only disagreement with some point being made.)
Thanks for sharing. You summed it up well: The community feels like a hidden gem when you first discover it. It feels like there's an energy of intelligence buzzing about interesting topics and sharing new findings.
Then you start encountering the weirder parts. For me, it was the group think and hero worship. I just wanted to read interesting takes on new topics, but if you deviated from the popular narrative associated with the heroes (Scott Alexander, Yudkowski, Cowen, Aaronson, etc.) it felt like the community's immune system identified you as an intruder and started attacking.
I think a lot of people get drawn into the idea of it being a community where they finally belong. Especially on Twitter (where the latest iteration is "TPOT") it's extraordinarily clique-ish and defensive. It feels like high school level social dynamics at play, except the players are equipped with deep reserves of rhetoric and seemingly endless free time to dunk on people and send their followers after people who disagree. It's a very weird contrast to the ideals claimed by the community.
Then you start encountering the weirder parts. For me, it was the group think and hero worship. I just wanted to read interesting takes on new topics, but if you deviated from the popular narrative associated with the heroes (Scott Alexander, Yudkowski, Cowen, Aaronson, etc.) it felt like the community's immune system identified you as an intruder and started attacking.
I think a lot of people get drawn into the idea of it being a community where they finally belong. Especially on Twitter (where the latest iteration is "TPOT") it's extraordinarily clique-ish and defensive. It feels like high school level social dynamics at play, except the players are equipped with deep reserves of rhetoric and seemingly endless free time to dunk on people and send their followers after people who disagree. It's a very weird contrast to the ideals claimed by the community.
Well nobody sent me; instead I had the strange experience of waking up this morning, seeing an interesting post about Scott Aaronson identifying as a rationalist, and when I check the discussion it's like half of HN has decided it's a good opportunity to espouse everything they dislike about this group of people.
Since when is that what we do here? If he'd written that he'd decided to become vegetarian, would we all be out here talking about how vegetarians are so annoying and one of them even spat on my hamburger one time?
And then of these uncalled-for takedowns, several -- including yours -- don't even seem to be engaging in good-faith discourse, and seem happy to pile on to attacks even when they're completely at odds with their own arguments.
I'm sorry to say it but the one who decided to use their free time to leer at people un-provoked over the internet seems to be you.
Since when is that what we do here? If he'd written that he'd decided to become vegetarian, would we all be out here talking about how vegetarians are so annoying and one of them even spat on my hamburger one time?
And then of these uncalled-for takedowns, several -- including yours -- don't even seem to be engaging in good-faith discourse, and seem happy to pile on to attacks even when they're completely at odds with their own arguments.
I'm sorry to say it but the one who decided to use their free time to leer at people un-provoked over the internet seems to be you.
Seems like a perfectly reasonable thing to discuss in the comments on this article, and I don't know which other "takedowns" you're referring to, but this person's comments on it have not been in bad faith at all.
(Indeed, I think it's in worse faith to try to guilt trip people who are just expressing critical opinions. It's fine - good, even! - to disagree with those people, but this particular comment has a very "how dare you criticize something!" tone that I don't think is constructive.)
(Indeed, I think it's in worse faith to try to guilt trip people who are just expressing critical opinions. It's fine - good, even! - to disagree with those people, but this particular comment has a very "how dare you criticize something!" tone that I don't think is constructive.)
> So I forgot about the slightly condescending, lecturing tone that comes out when you disagree with rationalist figureheads. :)
How was it condescending or lecturing?
> You could simply ask "Can you provide examples" instead of the "If you ____ then I suggest ____" form.
Why is that not equally condescending or lecturing?
How was it condescending or lecturing?
> You could simply ask "Can you provide examples" instead of the "If you ____ then I suggest ____" form.
Why is that not equally condescending or lecturing?
> Why is that not equally condescending or lecturing?
It just isn't! An inability to accurately identify what comes off as condescending is kind of the point...
It just isn't! An inability to accurately identify what comes off as condescending is kind of the point...
That sounds an awful lot like other people making stuff up to oppress me by sticking a "condescending" label to me without me having any way to contest it.
That sounds an awful lot like a victimhood complex.
"Are you being condescending" is a subjective judgement that other people will make up their own minds about. You can't control what people think about things you say and do, and they aren't "oppressing" you by making up their own minds about that.
"Are you being condescending" is a subjective judgement that other people will make up their own minds about. You can't control what people think about things you say and do, and they aren't "oppressing" you by making up their own minds about that.
> it was a general explanation of how the game is played.
You seem to be trying to insinuate that Alexander et. al. are pretending to know how things will turn out and then hiding behind probabilities when they don't turn out that way. This is missing the point completely. The point is that when Alexander assigns an 80% probability to many different outcomes, about 80% of them should occur, and it should not be clear to anyone (including Alexander) ahead of time which 80%.
> He predicted at least $250 million in damages from Black Lives Matter protests.
Many sources estimated damages at $2 billion or more (see https://www.usatoday.com/story/news/factcheck/2022/02/22/fac... and links from there), so this did in fact come true.
Edit: I see that the prediction relates to 2021 specificially. In the wake of 2020, I think it was perfectly reasonable to make such a prediction at that confidence level, even if it didn't actually turn out that way.
> He predicted Andrew Yang would win the 2021 NYC mayoral race with 80% certainty (he came in 4th place)
> He gave a 70% chance to Vitamin D being generally recognized as a good COVID treatment
If you make many predictions at 70-80% confidence, as he does, you should expect 20-30% of them not to come true. It would in fact be a failure (underconfidence) if they all came true. You are in fact citing a blog post that is exactly about a self-assessment of those confidence levels.
Also, he gave a 70% chance to Vitamin D not being generally recognized as a good COVID treatment.
> These all get mixed in with a small number of geopolitical, economic, and medical predictions with the net result of bringing his overall accuracy up.
The point is not "overall accuracy", but overall calibration - i.e., whether his assigned probabilities end up making sense and being statistically validated.
You have done nothing to establish that any correlation between the category of prediction and his accuracy on them.
You seem to be trying to insinuate that Alexander et. al. are pretending to know how things will turn out and then hiding behind probabilities when they don't turn out that way. This is missing the point completely. The point is that when Alexander assigns an 80% probability to many different outcomes, about 80% of them should occur, and it should not be clear to anyone (including Alexander) ahead of time which 80%.
> He predicted at least $250 million in damages from Black Lives Matter protests.
Many sources estimated damages at $2 billion or more (see https://www.usatoday.com/story/news/factcheck/2022/02/22/fac... and links from there), so this did in fact come true.
Edit: I see that the prediction relates to 2021 specificially. In the wake of 2020, I think it was perfectly reasonable to make such a prediction at that confidence level, even if it didn't actually turn out that way.
> He predicted Andrew Yang would win the 2021 NYC mayoral race with 80% certainty (he came in 4th place)
> He gave a 70% chance to Vitamin D being generally recognized as a good COVID treatment
If you make many predictions at 70-80% confidence, as he does, you should expect 20-30% of them not to come true. It would in fact be a failure (underconfidence) if they all came true. You are in fact citing a blog post that is exactly about a self-assessment of those confidence levels.
Also, he gave a 70% chance to Vitamin D not being generally recognized as a good COVID treatment.
> These all get mixed in with a small number of geopolitical, economic, and medical predictions with the net result of bringing his overall accuracy up.
The point is not "overall accuracy", but overall calibration - i.e., whether his assigned probabilities end up making sense and being statistically validated.
You have done nothing to establish that any correlation between the category of prediction and his accuracy on them.
> If you make many predictions at 70-80% confidence, as he does, you should expect 20-30% of them not to come true.
Yes that's the point, these people hedge like crazy to the point that they say nothing, they mean nothing, and effectively predict nothing.
Yes that's the point, these people hedge like crazy to the point that they say nothing, they mean nothing, and effectively predict nothing.
Please be civil.
I genuinely don't understand how you can point to someone's calibration curve where they've broadly done well, and cherry pick the failed predictions they made, and use this not just to claim that they're making bad predictions but that they're slimy about admitting error. What more could you possibly want from someone than a tally of their prediction record graded against the probability they explicitly assigned to it?
One man's modus ponens, as it goes.
I genuinely don't understand how you can point to someone's calibration curve where they've broadly done well, and cherry pick the failed predictions they made, and use this not just to claim that they're making bad predictions but that they're slimy about admitting error. What more could you possibly want from someone than a tally of their prediction record graded against the probability they explicitly assigned to it?
One man's modus ponens, as it goes.
> Please be civil.
lol, what? That was a civil comment. This seems like an excellent example of the point being made. Replying to a perfectly reasonable but critical comment with "please be civil" is super condescending.
So is stuff like "one man's modus ponens".
Look, we get it, you're talking to people who found this stuff smart and interesting in the past. But we got tired of it. For me, I realized after awhile that the people I most admired in real life were pretty much the opposite of the people I was reading the most on the internet. None of the smartest people I know talk in this snooty online iamsosmart style.
lol, what? That was a civil comment. This seems like an excellent example of the point being made. Replying to a perfectly reasonable but critical comment with "please be civil" is super condescending.
So is stuff like "one man's modus ponens".
Look, we get it, you're talking to people who found this stuff smart and interesting in the past. But we got tired of it. For me, I realized after awhile that the people I most admired in real life were pretty much the opposite of the people I was reading the most on the internet. None of the smartest people I know talk in this snooty online iamsosmart style.
Scott:
> This isn’t about me being an expert on these topics and getting them exactly right, it’s about me calibrating my ability to tell how much I know about things and how certain I am.
> At least $250 million in damage from BLM protests this year: 30%
Aurornis:
> I forgot about the slightly condescending, lecturing tone that comes out when you disagree with rationalist figureheads.
> Since Scott Alexander comes up a lot, a few randomly selected predictions that didn't come true:
> He predicted at least $250 million in damages from Black Lives Matter protests.
Is this a "perfectly reasonable but critical comment"?
Am I condescending if I say that predicting a 30% chance that something happens means predicting a 70% chance that it won't happen... so the fact that it didn't happen probably shouldn't be used as "gotcha!"?
> This isn’t about me being an expert on these topics and getting them exactly right, it’s about me calibrating my ability to tell how much I know about things and how certain I am.
> At least $250 million in damage from BLM protests this year: 30%
Aurornis:
> I forgot about the slightly condescending, lecturing tone that comes out when you disagree with rationalist figureheads.
> Since Scott Alexander comes up a lot, a few randomly selected predictions that didn't come true:
> He predicted at least $250 million in damages from Black Lives Matter protests.
Is this a "perfectly reasonable but critical comment"?
Am I condescending if I say that predicting a 30% chance that something happens means predicting a 70% chance that it won't happen... so the fact that it didn't happen probably shouldn't be used as "gotcha!"?
No. You're not being civil right now. You are lacing your reply with insults.
Your bar for "civil" is too low. Being defensive and thin skinned doesn't make your perspective more compelling.
You're both right. In all seriousness.
Agreed.
(I did waffle upon re-reading my comment and thinking it could have been more civil. But then decided that this person is also being very thin skinned. So I think you're right that we're both right.)
(I did waffle upon re-reading my comment and thinking it could have been more civil. But then decided that this person is also being very thin skinned. So I think you're right that we're both right.)
Weirdly enough, both can be true. I was tangentially involved in EA in the early days, and have some friends who were more involved. Lots of interesting, really cool stuff going on, but there was always latent insecurity paired with overconfidence and elitism as is typical in young nerd circles.
When big money got involved, the tone shifted a lot. One phrase that really stuck with me is "exceptional talent". Everyone in EA was suddenly talking about finding, involving, hiring exceptional talent at a time where there was more than enough money going around to give some to us mediocre people as well.
In the case of EA in particular circlejerks lead to idiotic ideas even when paired with rationalist rhetoric, so they bought mansions for team building (how else are you getting exceptional talent), praised crypto (because they are funding the best and brightest) and started caring a lot about shrimp welfare (no one else does).
When big money got involved, the tone shifted a lot. One phrase that really stuck with me is "exceptional talent". Everyone in EA was suddenly talking about finding, involving, hiring exceptional talent at a time where there was more than enough money going around to give some to us mediocre people as well.
In the case of EA in particular circlejerks lead to idiotic ideas even when paired with rationalist rhetoric, so they bought mansions for team building (how else are you getting exceptional talent), praised crypto (because they are funding the best and brightest) and started caring a lot about shrimp welfare (no one else does).
I don't think this validates the criticism that "they don't really ever show a sense of[...] maybe I'm wrong".
I think that sentence would be a fair description of certain individuals in the EA community, especially SBF, but that is not the same thing as saying that rationalists don't ever express epistemic uncertainty, when on average they spend more words on that than just about any other group I can think of.
I think that sentence would be a fair description of certain individuals in the EA community, especially SBF, but that is not the same thing as saying that rationalists don't ever express epistemic uncertainty, when on average they spend more words on that than just about any other group I can think of.
> caring a lot about shrimp welfare (no one else does).
Ah. I guess they are working out ecology through first principles, I guess?
I feel like a lot of the criticism of EA and rationalism does boil down to some kind of general criticism of naivete and entitlement, which... is probably true when applied to lots of people, regardless of whether they espouse these ideas or not.
It's also easier to criticize obviously doomed/misguided efforts at making the world a better place than to think deeply about how many of the pressing modern day problems (environmental issues, extinction, human suffering, etc.) also seem to be completely intractable, when analyzed in terms of the average individual's ability to take action. I think some criticism of EA or rationalism is also a reaction to a creeping unspoken consensus that "things are only going to get worse" in the future.
Ah. I guess they are working out ecology through first principles, I guess?
I feel like a lot of the criticism of EA and rationalism does boil down to some kind of general criticism of naivete and entitlement, which... is probably true when applied to lots of people, regardless of whether they espouse these ideas or not.
It's also easier to criticize obviously doomed/misguided efforts at making the world a better place than to think deeply about how many of the pressing modern day problems (environmental issues, extinction, human suffering, etc.) also seem to be completely intractable, when analyzed in terms of the average individual's ability to take action. I think some criticism of EA or rationalism is also a reaction to a creeping unspoken consensus that "things are only going to get worse" in the future.
>I think some criticism of EA or rationalism is also a reaction to a creeping unspoken consensus that "things are only going to get worse" in the future.
I think it's that combined with the EA approach to it which is: let's focus on space flight and shrimp welfare. Not sure which side is more in denial about the impending future?
I have no belief any particular individual can do anything about shrimp welfare more than they can about the intractable problems we do face.
I think it's that combined with the EA approach to it which is: let's focus on space flight and shrimp welfare. Not sure which side is more in denial about the impending future?
I have no belief any particular individual can do anything about shrimp welfare more than they can about the intractable problems we do face.
> I think it's that combined with the EA approach to it which is: let's focus on space flight and shrimp welfare. Not sure which side is more in denial about the impending future?
I think its a result of its complete denial of and ignorance of politics. Because rationalist and effective altruist movements make a whole lot more sense, if you realize they are talking about deeply social and political issues with all politics removed from it. Its technocrat-ism the poster child of the kind of "there is no alternative" neoliberalism that everyone in the western world was indoctrinated into since the 80s.
Its a fundamental contradiction, we don't need to talk about politics because we already know liberal democracies and free-market capitalism is the best we ever going to achieve, faced with the numerous intractable problems we face that can not possibly be related to liberal democracies and free-market capitalism.
The problem is: How do we talk about any issue the world is facing today without ever challenging or even talking about any of the many assumptions the western liberal democracies are based upon? In other words: the problems we face are structural/systemic, but we are not allowed to talk about the structures/systems. That's how you end up with space flight and shrimp welfare and AGI/ASI catastrophizing taking up 99% of everything these people talk about. It's infantile, impotent liberal escapism more than anything else.
I think its a result of its complete denial of and ignorance of politics. Because rationalist and effective altruist movements make a whole lot more sense, if you realize they are talking about deeply social and political issues with all politics removed from it. Its technocrat-ism the poster child of the kind of "there is no alternative" neoliberalism that everyone in the western world was indoctrinated into since the 80s.
Its a fundamental contradiction, we don't need to talk about politics because we already know liberal democracies and free-market capitalism is the best we ever going to achieve, faced with the numerous intractable problems we face that can not possibly be related to liberal democracies and free-market capitalism.
The problem is: How do we talk about any issue the world is facing today without ever challenging or even talking about any of the many assumptions the western liberal democracies are based upon? In other words: the problems we face are structural/systemic, but we are not allowed to talk about the structures/systems. That's how you end up with space flight and shrimp welfare and AGI/ASI catastrophizing taking up 99% of everything these people talk about. It's infantile, impotent liberal escapism more than anything else.
Well put.
>they bought mansions for team building
They bought one mansion to host fundraisers with the super-rich, which I believe is an important correction. You might disagree with that reasoning as well, but it's definitely not as described.
They bought one mansion to host fundraisers with the super-rich, which I believe is an important correction. You might disagree with that reasoning as well, but it's definitely not as described.
Wytham Abbey was bought mainly to host internal workshops and "retreats", as outlined by the people who bought it
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/...
https://forum.effectivealtruism.org/posts/yggjKEeehsnmMYnZd/...
As far as I know it's never hosted an impress-the-oligarch fundraiser, which as you say would at least have a logic behind it[1] even if it might seem distasteful.
For a philosophy which started out from the point of view that much of mainstream aid was spent with little thought, it was a bit of an end of Animal Farm moment.
(to their credit, a lot of people who identified as EAs were unhappy. If you drew a Venn diagram of the people that objected, people who sneered at the objections[2] and people who identified as rationalists you might only need two circles though...)
[1]a pretty shaky one considering how easy it is to impress American billionaires with Oxford architecture without going to the expense of operating a nearby mansion as a venue, particularly if you happen to be a charitable movement with strong links to the university... [2]obviously people are only objecting to it for PR purposes because they're not smart enough to realise that capital appreciates and that venues cost money, and definitely not because they've got a pretty good idea how expensive upkeep on little used medieval venues are and how many alternatives exist if you really care about the cost effectiveness of your retreat, especially to charitable movements affiliated with a university...
As far as I know it's never hosted an impress-the-oligarch fundraiser, which as you say would at least have a logic behind it[1] even if it might seem distasteful.
For a philosophy which started out from the point of view that much of mainstream aid was spent with little thought, it was a bit of an end of Animal Farm moment.
(to their credit, a lot of people who identified as EAs were unhappy. If you drew a Venn diagram of the people that objected, people who sneered at the objections[2] and people who identified as rationalists you might only need two circles though...)
[1]a pretty shaky one considering how easy it is to impress American billionaires with Oxford architecture without going to the expense of operating a nearby mansion as a venue, particularly if you happen to be a charitable movement with strong links to the university... [2]obviously people are only objecting to it for PR purposes because they're not smart enough to realise that capital appreciates and that venues cost money, and definitely not because they've got a pretty good idea how expensive upkeep on little used medieval venues are and how many alternatives exist if you really care about the cost effectiveness of your retreat, especially to charitable movements affiliated with a university...
> If you drew a Venn diagram of the people that objected, people who sneered at the objections[2] and people who identified as rationalists you might only need two circles though...)
I’m a bit confused by this one.
Are you saying that no-one who identifies as rationalist sneered at the objections? Because I don’t think that’s true.
I’m a bit confused by this one.
Are you saying that no-one who identifies as rationalist sneered at the objections? Because I don’t think that’s true.
Nope, I'm implying the people sneering at the objections were the self proclaimed rationalists. Other, less contrarian thinkers were more inclined to spot that a $15m heritage building might not be the epitome of cost-effective venues...
Ah, fair enough! I had heard the "hosting wealthy donors" as the primary motivation, but it appears to be secondary. My bad.
>As far as I know it's never hosted an impress-the-oligarch fundraiser
As far as I know, they only hosted 3 events there before deciding to sell, so this is low-information.
>As far as I know it's never hosted an impress-the-oligarch fundraiser
As far as I know, they only hosted 3 events there before deciding to sell, so this is low-information.
Surely you've learned in your life that whatever people might say is not true, nor is it the best reflection of their intentions...
> both can be true
Yes! It can be true both that rationalists tend, more than almost any other group, to admit and try to take account of their uncertainty about things they say and that it's fun to dunk on them for being arrogant and always assuming they're 100% right!
Yes! It can be true both that rationalists tend, more than almost any other group, to admit and try to take account of their uncertainty about things they say and that it's fun to dunk on them for being arrogant and always assuming they're 100% right!
> they bought mansions for team building
Because they we doing so many workshops that buying a building was cheaper than renting all the time.
You may argue that organizing workshops is wrong (and you might be right about that), but once you choose to do them, it makes sense to choose the cheaper option rather than the more expensive one. That's not rationalist rhetoric, that's just basic economy.
Because they we doing so many workshops that buying a building was cheaper than renting all the time.
You may argue that organizing workshops is wrong (and you might be right about that), but once you choose to do them, it makes sense to choose the cheaper option rather than the more expensive one. That's not rationalist rhetoric, that's just basic economy.
I read rationalist writing for a very long time, and eventually concluded that this part of it was, not universally but predominantly, performative. After you read enough articles from someone, it is clear what they have conviction in, even when they are putting up disclaimers saying they don't.
>Are you sure you're not painting this group with an overly-broad brush?
"Aren't these the people who"...
> And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"?
What's the value of that if it doesn't appear to be reasonably put to their own ideas. What you described otherwise is just another form of the exact kind of self-congratulation often (reasonably, IMO) lobbed at these "people"
"Aren't these the people who"...
> And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"?
What's the value of that if it doesn't appear to be reasonably put to their own ideas. What you described otherwise is just another form of the exact kind of self-congratulation often (reasonably, IMO) lobbed at these "people"
They're behind Anthropic and were behind openai being a nonprofit. They're behind the friendly AI movement and effective altruism.
They're responsible for funneling huge amounts of funding away from domain experts (effective altruism in practice means "Oxford math PhD writes a book report about a social sciences problem they've only read about and then defunds all the NGOs").
They're responsible for moving all the AI safety funding away from disparate impact measures to "save us from skynet" fantasies.
They're responsible for funneling huge amounts of funding away from domain experts (effective altruism in practice means "Oxford math PhD writes a book report about a social sciences problem they've only read about and then defunds all the NGOs").
They're responsible for moving all the AI safety funding away from disparate impact measures to "save us from skynet" fantasies.
I don't see how this is a response to what I wrote. Can you explain?
I think GP is saying that their epistemic humility is a pretense, a pose. They do a lot of throat clearing about quantifying their certainty and error checking themselves, and then proceed to bring about very consequential outcomes anyway for absurd reasons with predictable side effects that they should have considered but didn't.
Yeah. It's not that they never express uncertainty so much as they like to express uncertainty as arbitrarily precise and convenient-looking expected value calculations which often look like far more of a rhetorical tool to justify their preferences (I've accounted for the uncertainty and even given a credence as low as 14.2% I'm still right!) than a decision making heuristic...
What ngo's are the rationalists or effective altruists responseable for killing or defunding?
I've always seen the breathless Singularitarian worrying about AI Alignment as a smokescreen to distract people from thinking clearly about the more pedestrian hazards of AI that isn't self-improving or superhuman, from algorithmic bias, to policy-washing, to energy costs and acceleration of wealth concentration. It also leads to so-called longtermism - discounting the benefits of solving current real problems and focusing entirely on solving a hypothetical one that you think will someday make them all irrelevant.
My feeling has been that it’s a lot of people that work on B2B SaaS that are sad they hadn’t gotten the chance to work on the Manhattan Project. Be around the smartest people in your field. Contribute something significant (but dangerous! And we need to talk about it!) to humanity. But yeah computer science in the 21st century has not turned out to be as interesting as that. Maybe just as important! But Jeff Bezos important, not Richard Feynman important.
"Overproduction of elites" is the expression.
The Singularitarians were breathlessly worrying 20+ years ago, when AI was absolute dogshit - Eliezer once stated that Doug Lenat was incautious in launching Eurisko because it could've gone through a hard takeoff. I don't think it's just an act to launder their evil plans, none of which at the time worked.
Yeah, people were generally terrified of this stuff back before you could make money off of it.
Fair. OpenAI totally use those arguments to launder their plans, but that saga has been more Silicon Valley exploiting longstanding rationalist beliefs for PR purposes than rationalists getting rich...
Eliezer did once state his intentions to build "friendly AI", but seems to have been thwarted by his first order reasoning about how AI decision theory should work being more important to him than building something that actually did work, even when others figured out the latter bit.
Eliezer did once state his intentions to build "friendly AI", but seems to have been thwarted by his first order reasoning about how AI decision theory should work being more important to him than building something that actually did work, even when others figured out the latter bit.
From my experience people who worry about existential AI risk tend to worry more about the more mundane problems than the wider population. In particular they have been very vocal about scamming, cube security and also wealth (& power!) concentration.
>s a smokescreen to distract people from thinking clearly about the more pedestrian hazards of AI that isn't self-improving or superhuman,
Anything that can't be self-improving or superhuman almost certainly isn't worthy of the moniker "AI". A true AI will be born into a world that has already unlocked the principles of intelligence. Humans in that world would be capable themselves of improving AI (slowly), but the AI itself will (presumably) run on silicon and be a quick thinker. It will be able to self-improve, rapidly at first, and then more rapidly as its increased intelligence allows for even quicker rates of improvement. And if not superhuman initially, it would soon become so.
We don't even have anything resembling real AI at the moment. Generative models are probably some blind alley.
Anything that can't be self-improving or superhuman almost certainly isn't worthy of the moniker "AI". A true AI will be born into a world that has already unlocked the principles of intelligence. Humans in that world would be capable themselves of improving AI (slowly), but the AI itself will (presumably) run on silicon and be a quick thinker. It will be able to self-improve, rapidly at first, and then more rapidly as its increased intelligence allows for even quicker rates of improvement. And if not superhuman initially, it would soon become so.
We don't even have anything resembling real AI at the moment. Generative models are probably some blind alley.
> We don't even have anything resembling real AI at the moment. Generative models are probably some blind alley.
I think that the OP's point was that it doesn't matter whether it's "real AI" or not. Even if it's just a glorified auto-correct system, it's one that has the clear potential to overturn our information/communication systems and our assumptions about individuals' economic value.
I think that the OP's point was that it doesn't matter whether it's "real AI" or not. Even if it's just a glorified auto-correct system, it's one that has the clear potential to overturn our information/communication systems and our assumptions about individuals' economic value.
If that has the potential to ruin economies, then the economic rot is so much more profound than anyone (me included) ever realized.
I think when the GP says "our assumptions about individuals' economic value." they mean half the workforce becoming unemployed because the auto corrector can do it cheaper.
That's going to be a swift kick to your economy, no matter how strong.
That's going to be a swift kick to your economy, no matter how strong.
If the "autocorrector" can do anything that used to be a paid wage/salary "cheaper", this isn't the case of another job being automated away. Either something is grossly wrong in that some are willing to have a stupid machine do that poorly (writing Sports Illustrated articles), or something was always grossly wrong in that we were paying someone to do something that never did matter (paying someone to write SEO spam). This isn't a swift kick in the economy, this is a "your economy was so weak a soft breeze knocked it over".
> Either something is grossly wrong in that some are willing to have a stupid machine do that poorly (writing Sports Illustrated articles), or something was always grossly wrong in that we were paying someone to do something that never did matter
I recently had an LLM write a function for me that, for a given RGB color value and another integer n > 1, returned to me a list of n RGB colors equidistantly and sequentially spaced around the color wheel starting at the specified RGB value.
For a given system I'm creating, I might have lots of such tasks. That collection of tasks is something that did matter and took some education and skill to complete well.
In the pre-LLM world - assuming I was too busy to handle all the tasks myself - I would have delegated them to a junior software engineer.
In a post-LLM world, I just ask the LLM to implement tasks like that, and I review the code for correctness.
That seems like a pretty transformational change to me, and not just some kind of "rot" being removed from the process.
I recently had an LLM write a function for me that, for a given RGB color value and another integer n > 1, returned to me a list of n RGB colors equidistantly and sequentially spaced around the color wheel starting at the specified RGB value.
For a given system I'm creating, I might have lots of such tasks. That collection of tasks is something that did matter and took some education and skill to complete well.
In the pre-LLM world - assuming I was too busy to handle all the tasks myself - I would have delegated them to a junior software engineer.
In a post-LLM world, I just ask the LLM to implement tasks like that, and I review the code for correctness.
That seems like a pretty transformational change to me, and not just some kind of "rot" being removed from the process.
yep, the biggest threat posed by AI comes from the capitalists who want to own it.
I actually think the people developing AI might well not get rich off it.
Instead, unless there's a single winner, we will probably see the knowledge on how to train big LLMs and make them perform well diffuse throughout a large pool of AI researchers, with the hardware to train models reasonably close to the SotA becoming more quite accessible.
I think the people who will benefit will be the owners of ordinary but hard-to-dislodge software firms, maybe those that have a hardware component. Maybe firms like Apple, maybe car manufacturers. Pure software firms might end up having AI assisted programmers as competitors instead, pushing margins down.
This is of course pretty speculative, and it's not reality yet, since firms like Cursor etc. have high valuations, but I think this is what you'd get from the probably pressure if it keeps getting better.
Instead, unless there's a single winner, we will probably see the knowledge on how to train big LLMs and make them perform well diffuse throughout a large pool of AI researchers, with the hardware to train models reasonably close to the SotA becoming more quite accessible.
I think the people who will benefit will be the owners of ordinary but hard-to-dislodge software firms, maybe those that have a hardware component. Maybe firms like Apple, maybe car manufacturers. Pure software firms might end up having AI assisted programmers as competitors instead, pushing margins down.
This is of course pretty speculative, and it's not reality yet, since firms like Cursor etc. have high valuations, but I think this is what you'd get from the probably pressure if it keeps getting better.
It smacks of a goldrush. The winners will be the people selling shovels (nVidia) and housing (AWS). It may also be the guides showing people the mountains (Cursor, OpenAI, etc).
I suspect you'll see a few people "win" or strike it rich with AI, the vast majority will simply be left with a big bill.
I suspect you'll see a few people "win" or strike it rich with AI, the vast majority will simply be left with a big bill.
When railroads were first being built across the continental USA, those companies also had high valuations (for the time). Most of them ultimately went bankrupt or were purchased for a fraction of their peak valuation. But the tracks remained, and many of those routes are still in use today.
Exact same thing happened with fiber optic cable layers in the late 1990s. On exactly the same routes!
It's because the land-rights were more valuable than the steel rails, which the fiber optic companies bought up.
Just checked.
The problem is the railroads were purchased by the winners. Who turned out to be the existing winners. Who then went on to continue to win.
On the one hand, I guess that's just life here in reality.
On the other, man, reality sucks sometimes.
The problem is the railroads were purchased by the winners. Who turned out to be the existing winners. Who then went on to continue to win.
On the one hand, I guess that's just life here in reality.
On the other, man, reality sucks sometimes.
That’s capitalism for you - loser’s margins were winner’s opportunities.
Imagine if they were bought by losers.
Imagine if they were bought by losers.
And today the railroad system in the USA sucks compared to other developed countries and even China.
It turns out that boom-and-bust capitalism isn’t great for building something that needs to evolve over centuries.
Perhaps American AI efforts will one day be viewed similarly. “Yeah, they had an early rush, lots of innovation, high valuations, and robber barons competing. Today it’s just stale old infra despite the high-energy start.”
It turns out that boom-and-bust capitalism isn’t great for building something that needs to evolve over centuries.
Perhaps American AI efforts will one day be viewed similarly. “Yeah, they had an early rush, lots of innovation, high valuations, and robber barons competing. Today it’s just stale old infra despite the high-energy start.”
I think it's unlikely that AI efforts will go as railroads have. I think being an AI foundation model company is more like being an airplane builder than like a railway company, since you develop your technology.
Plenty of those that similarly went bankrupt over the years, and now the USA mostly has Boeing that's reached a set of continual crises and being propped up by the government.
Yes, of course, but they usually live for a long time.
Boeing tries to compete with Airbus, where the engineers make maybe 70% of what they do in the US, and has upon that made some really bad decisions. Many aircraft companies are profitable.
Boeing tries to compete with Airbus, where the engineers make maybe 70% of what they do in the US, and has upon that made some really bad decisions. Many aircraft companies are profitable.
America's passenger rail sucks, it couldn't compete with airplanes and every train company got out of the business, abandoning it to the government. But America does have a great deal of freight rail which sees a lot of use (much more than in Europe, I don't know how it compares to China though.)
One reason the passenger service sucks is that the freight rail companies own the tracks, and are happy to let a passenger train sit behind a freight train for a couple hours waiting for space in the freight yard so it can get out of the way.
> freight rail companies own the tracks
Humans are also freight, of course. It is not like the rail companies really care about what kind of fright is on the trains, so long as it is what the customer considers most important (read: most profitable). Humans are deprioritized exactly because they aren't considered important by the customer, which is to say that the customer, who is also the freight in this case, doesn't really want to be on a train in the first place. The customer would absolutely ensure priority (read: pay more, making it known that they are priority) if they wanted to be there.
I understand the train geeks on the internet find it hard to believe that not everyone loves trains like they do, but the harsh reality is that the average American Joe prefers other means of transportation. Should that change in the future, the rail network will quickly accommodate. It has before!
Humans are also freight, of course. It is not like the rail companies really care about what kind of fright is on the trains, so long as it is what the customer considers most important (read: most profitable). Humans are deprioritized exactly because they aren't considered important by the customer, which is to say that the customer, who is also the freight in this case, doesn't really want to be on a train in the first place. The customer would absolutely ensure priority (read: pay more, making it known that they are priority) if they wanted to be there.
I understand the train geeks on the internet find it hard to believe that not everyone loves trains like they do, but the harsh reality is that the average American Joe prefers other means of transportation. Should that change in the future, the rail network will quickly accommodate. It has before!
Ah, a true believer in the invisible hand. What nonsense.
The root cause is Americans, generally, prefer any mode of transit other than rail, so passenger rail isn't profitable, so train companies naturally prioritize freight.
For what it's worth, I like traveling by train and do so whenever I can, but I'm an outlier. Most Americans look at the travel times and laugh at the premise of choosing a train over a plane. And when I say they look at the travel times, I don't mean they actually bother to look up train routes. They just know that airplanes are several times faster. Delays suffered by trains never get factored into the decision because trains aren't taken seriously in the first place.
For what it's worth, I like traveling by train and do so whenever I can, but I'm an outlier. Most Americans look at the travel times and laugh at the premise of choosing a train over a plane. And when I say they look at the travel times, I don't mean they actually bother to look up train routes. They just know that airplanes are several times faster. Delays suffered by trains never get factored into the decision because trains aren't taken seriously in the first place.
China hasn't shown that their railroad buildout will work. My understanding is they currently aren't making enough return to payoff debt, yet alone plan for future maintenance. Historically the command economy type stuff looks great in the early years, it's later on we see if that is reality.
You are comparing USA today to the robber baron phase, whose to say China isn't in the same phase? Lots of money being thrown at new railroads and you have Chinese leaders and best and management leaders chasing that money. When happens when it goes low budget/maintenance mode?
You are comparing USA today to the robber baron phase, whose to say China isn't in the same phase? Lots of money being thrown at new railroads and you have Chinese leaders and best and management leaders chasing that money. When happens when it goes low budget/maintenance mode?
The USA today is in a robber baron phase. We only briefly left it for about 2 generations due to the rise of labor power in the late 1800s/early 1900s. F.D.R. was the compromise president put into place to placate labor and prevent a socialist revolution.
> And today the railroad system in the USA sucks compared to other developed countries and even China.
Nonsense. The US has the largest freight rail system in the world, and is considered to have the most efficient rail system in the world to go along with it.
There isn't much in the way of passenger service, granted, but that's because people in the US aren't, well, poor. They can afford better transportation options.
> It turns out that boom-and-bust capitalism isn’t great for building something that needs to evolve over centuries.
It initially built out the passenger rail just fine, but then evolution saw better options come along. Passenger rail disappeared because it no longer served a purpose. It is not like, say, Japan where the median household income is approaching half that of Mississippi and they hold on to rail because that's what is affordable.
Nonsense. The US has the largest freight rail system in the world, and is considered to have the most efficient rail system in the world to go along with it.
There isn't much in the way of passenger service, granted, but that's because people in the US aren't, well, poor. They can afford better transportation options.
> It turns out that boom-and-bust capitalism isn’t great for building something that needs to evolve over centuries.
It initially built out the passenger rail just fine, but then evolution saw better options come along. Passenger rail disappeared because it no longer served a purpose. It is not like, say, Japan where the median household income is approaching half that of Mississippi and they hold on to rail because that's what is affordable.
> There isn't much in the way of passenger service, granted, but that's because people in the US aren't, well, poor. They can afford better transportation options.
This is so misguided view... Trains (when done right) aren't "for the poor", they are great transportation option, that beats both airplanes and cars. In Poland, which isn't even close to the best, you can travel between big cities with speeds above 200km/h, and you can use regional rail for your daily commute, both those options being very comfortable and convenient, much more convenient than traveling by car.
This is so misguided view... Trains (when done right) aren't "for the poor", they are great transportation option, that beats both airplanes and cars. In Poland, which isn't even close to the best, you can travel between big cities with speeds above 200km/h, and you can use regional rail for your daily commute, both those options being very comfortable and convenient, much more convenient than traveling by car.
Poland is approximately the same geographical size as Nevada. In the US, "between cities" is more like New York to Las Vegas, not Las Vegas to... uh, I couldn't think of another city in Nevada off the top of my head. What under-serviced route were you thinking of there?
What gives you the idea that rail would be preferable to flying for the NYC to LAS route if only it existed? Even as the crow flies it is approximately 4,000 km, meaning that at 200 km/h you are still looking at around 20 hours of travel in an ideal case. Instead of just 5 hours by plane. If you're poor an additional 15 hours wasted might not mean much, but when time is valuable?
What gives you the idea that rail would be preferable to flying for the NYC to LAS route if only it existed? Even as the crow flies it is approximately 4,000 km, meaning that at 200 km/h you are still looking at around 20 hours of travel in an ideal case. Instead of just 5 hours by plane. If you're poor an additional 15 hours wasted might not mean much, but when time is valuable?
> In the US, "between cities" is more like New York to Las Vegas, not Las Vegas to... uh, I couldn't think of another city in Nevada off the top of my head. What under-serviced route were you thinking of there?
Why would you constrain the route to within a specific state? In fact, right now a high-speed rail line is being planned between Las Vegas and LA.
But outside of Nevada, there are many equivalent distance routes in the US between major population centers, including:
Chicago/Detroit
Dallas/Houston
LA/SF
Atlanta/Charlotte
Why would you constrain the route to within a specific state? In fact, right now a high-speed rail line is being planned between Las Vegas and LA.
But outside of Nevada, there are many equivalent distance routes in the US between major population centers, including:
Chicago/Detroit
Dallas/Houston
LA/SF
Atlanta/Charlotte
> In fact, right now a high-speed rail line is being planned between Las Vegas and LA.
Right now and since 1979!
I'll grant you that people love to plan, but it turns out that they don't love putting on their boots and picking up a shovel nearly as much.
> But outside of Nevada, there are many equivalent distance routes in the US between major population centers, including
And there is nothing stopping those lines from being built other than the lack of will to do it. As before, the will doesn't exist because better options exist.
Right now and since 1979!
I'll grant you that people love to plan, but it turns out that they don't love putting on their boots and picking up a shovel nearly as much.
> But outside of Nevada, there are many equivalent distance routes in the US between major population centers, including
And there is nothing stopping those lines from being built other than the lack of will to do it. As before, the will doesn't exist because better options exist.
There are a lot more obstacles than lack of will. There are also property rights, environmental reviews, availability of skilled workers, and lack of capital. HN users sometimes have this weird fantasy that with enough political will it's possible to make enormous changes but that's simply not how things operate in a republic with a dual sovereignty system.
> Chicago/Detroit
There has been continuous regularly scheduled passenger service between Chicago and Detroit since before the Civil War. The current Amtrak Wolverine runs 110 MPH (180 KPH) for 90% of the route, using essentially the same trainset that Brightline plans to use.
There has been continuous regularly scheduled passenger service between Chicago and Detroit since before the Civil War. The current Amtrak Wolverine runs 110 MPH (180 KPH) for 90% of the route, using essentially the same trainset that Brightline plans to use.
Fair point. Last time I took that train (mid 1990s) it didn't run to Pontiac or Troy, and I recall there being very infrequent service. A far as I know, it's not the major mode of passenger transit between Detroit and Chicago. Cars are. That might be because of the serious lack of last-mile transit connectivity in the Detroit area.
Also, if you go all in and build something equivalent to Chinese bullet trains (that go with speeds up to 350km/h) you could do for example NY to Chicago in 3.5 hours, or even NY to Miami in 6 hours :-D (I know, not very realistic)
Not sure how we got from Scott A being a rationalist to trains, but since we're here, I want to say:
I've taken a Chinese train from Zhengzhou, in central China, to Shenzhen, and it was fantastic. Cheap, smooth, fast, lots of legroom, easy to get on and off or walk around to the dining car. And, there's a thing where boiling hot water is available, so everyone brings instant noodle packs of every variety to eat on the train.
Can't even imagine what the US would be like if we had that kind of thing.
I've taken a Chinese train from Zhengzhou, in central China, to Shenzhen, and it was fantastic. Cheap, smooth, fast, lots of legroom, easy to get on and off or walk around to the dining car. And, there's a thing where boiling hot water is available, so everyone brings instant noodle packs of every variety to eat on the train.
Can't even imagine what the US would be like if we had that kind of thing.
Is being build? Um, not quite. Is being planned. Is arranging for right-of-way. But to the best of my knowledge, actual construction has not started.
If you can't think of another city in Nevada off the top of your head, are you even American? (Reno.)
Anyway, New York to Las Vegas spans most of the US. There are plenty of routes in the US where rail would make sense. Between Boston, New Haven, New York City, Philadelphia, Baltimore, and Washington, D.C. Which has the Amtrak Acela. Or perhaps Miami to Orlando. Which has a privately funded high speed rail connection called Brightline that runs at 200 km/h who's ridership was triple what had been expected at launch.
Anyway, New York to Las Vegas spans most of the US. There are plenty of routes in the US where rail would make sense. Between Boston, New Haven, New York City, Philadelphia, Baltimore, and Washington, D.C. Which has the Amtrak Acela. Or perhaps Miami to Orlando. Which has a privately funded high speed rail connection called Brightline that runs at 200 km/h who's ridership was triple what had been expected at launch.
> are you even American?
I am, thankfully, not.
> Which has a privately funded high speed rail connection called Brightline that runs at 200 km/h
Which proves that when the will is there, it will be done. The only impediment in other places is simply the people not wanting it. If they wanted it, it would already be there.
The US has been here before. It built out a pretty good, even great, passenger rail network a couple of centuries ago when the people wanted it. It eventually died out simply because the people didn't want it anymore.
If they want it again in the future, it will return. But as for the moment...
I am, thankfully, not.
> Which has a privately funded high speed rail connection called Brightline that runs at 200 km/h
Which proves that when the will is there, it will be done. The only impediment in other places is simply the people not wanting it. If they wanted it, it would already be there.
The US has been here before. It built out a pretty good, even great, passenger rail network a couple of centuries ago when the people wanted it. It eventually died out simply because the people didn't want it anymore.
If they want it again in the future, it will return. But as for the moment...
This analysis seems wrong for at least a couple of reasons.
1. Freight is easier to manage and has better economics on a dedicated network. The US freight network is extremely efficient as others have pointed out. Other networks, e.g., Germany, instead prioritized passenger service. In Germany rail moves a small proportion of freight (19%) compared to trucks. [0] It's really noticeable on the Autobahn and unlike the US where a lot of truck traffic is intermodal loads.
2. The US could have better rail service by investing in passenger networks. Instead we have boondoggles like the California high-speed rail project which has already burned through 10s of billions of dollars with no end in sight. [1] Or the New Jersey Transit system which I had the pleasure to ride on earlier today to Newark Airport. It has pretty good coverage but needs investment.
[0] https://dhl-freight-connections.com/en/trends/global-freight...
[1] https://en.wikipedia.org/wiki/California_High-Speed_Rail
1. Freight is easier to manage and has better economics on a dedicated network. The US freight network is extremely efficient as others have pointed out. Other networks, e.g., Germany, instead prioritized passenger service. In Germany rail moves a small proportion of freight (19%) compared to trucks. [0] It's really noticeable on the Autobahn and unlike the US where a lot of truck traffic is intermodal loads.
2. The US could have better rail service by investing in passenger networks. Instead we have boondoggles like the California high-speed rail project which has already burned through 10s of billions of dollars with no end in sight. [1] Or the New Jersey Transit system which I had the pleasure to ride on earlier today to Newark Airport. It has pretty good coverage but needs investment.
[0] https://dhl-freight-connections.com/en/trends/global-freight...
[1] https://en.wikipedia.org/wiki/California_High-Speed_Rail
> This analysis seems wrong for at least a couple of reasons.
How so?
> The US freight network is extremely efficient as others have pointed out.
'Others' being literally the comment you replied to.
> The US could have better rail service by investing in passenger networks.
Everything there is can be improved, of course, but to what significance here?
How so?
> The US freight network is extremely efficient as others have pointed out.
'Others' being literally the comment you replied to.
> The US could have better rail service by investing in passenger networks.
Everything there is can be improved, of course, but to what significance here?
Yeah the best in the world freight industry that is being sued by every company in America for their decades long price fixing scandal. Cant find the list of plaintiffs right now but it had ford and frito lay and hundreds of others. An update from yesterday:
https://www.mlex.com/mlex/antitrust/articles/2355294/us-rail...
https://www.mlex.com/mlex/antitrust/articles/2355294/us-rail...
Yeah, a cheap transportation option is a terrible thing to have... /s
It's not that it would be terrible, but in the real world people are generally lazy and will only do what they actually want to see happen. Surprisingly, we don't yet have magical AI robots that autonomously go around turning all imagined ideas into reality without the need for human grit.
Since nobody really wants passenger rail in the US, they don't put in the effort to see that it exists (outside of some particular routes where they do want it). In many other countries, people do want board access to passenger rail (because that's all they can afford), so they put in the effort to have it.
~200 years ago the US did want passenger rail, they put in the work to realize it, and it did have a pretty good passenger rail network at the time given the period. But, again, better technology came along, so people stopped maintaining/improving what was there. They could do it again if they wanted to... But they don't.
Since nobody really wants passenger rail in the US, they don't put in the effort to see that it exists (outside of some particular routes where they do want it). In many other countries, people do want board access to passenger rail (because that's all they can afford), so they put in the effort to have it.
~200 years ago the US did want passenger rail, they put in the work to realize it, and it did have a pretty good passenger rail network at the time given the period. But, again, better technology came along, so people stopped maintaining/improving what was there. They could do it again if they wanted to... But they don't.
What an ironic side thread in a conversation about people who are confidently ignorant.
I am not sure the irony works as told unless software and people are deemed to be the same thing. But what is there to suggest that they are?
[deleted]
Or the propagandists that use it
They won't be allowed to use it unless they serve the capitalists who own it.
It's not social media. It's a model the capitalists train and own. Best the rest of us will have access to are open source ones. It's like the difference between trying to go into court backed by google searches as opposed to Lexis/Nexis. You're gonna have a bad day with the judge.
Here's hoping the open source stuff gets trained on quality data rather than reddit and 4chan. Given how the courts are leaning on copyright, and lack of vetted data outside copyright holder remit, I'm not sanguine about the chances of parity long term.
It's not social media. It's a model the capitalists train and own. Best the rest of us will have access to are open source ones. It's like the difference between trying to go into court backed by google searches as opposed to Lexis/Nexis. You're gonna have a bad day with the judge.
Here's hoping the open source stuff gets trained on quality data rather than reddit and 4chan. Given how the courts are leaning on copyright, and lack of vetted data outside copyright holder remit, I'm not sanguine about the chances of parity long term.
Looking at Grok especially, it doesn't feel like a given that you can have a true SOTA model that is properly brainwashed.
To be fair, Musk is probably the least subtle megabillionaire of them all, and that reflects in the odd behavior of his silicon child. I don't doubt in the competency of the likes of Thiel to build their techo-monarchy.
The problem for them is that it might be a rather fundamental limitation. We already know that RLHF makes models dumber. It is entirely possible that, in order to make the model buy fully into what those people are peddling, the amount of forceful training required would crater the model's overall performance.
The propagandists serve the capitalists, so it's all the same.
Nah, it’s from state actors using the technology to control and wage war by far.
> They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is".
Have you ever read Scott Alexander's blog (Slate Star Codex, now Astral Codex X)? It's full of doubt and self-questioning. The guy even keeps a public list of his mistakes:
https://www.astralcodexten.com/p/mistakes
I'll admit my only touchpoint to the "rationalist community" is this blog, but I sure don't get "full of themselves" from that. Quite the contrary.
Have you ever read Scott Alexander's blog (Slate Star Codex, now Astral Codex X)? It's full of doubt and self-questioning. The guy even keeps a public list of his mistakes:
https://www.astralcodexten.com/p/mistakes
I'll admit my only touchpoint to the "rationalist community" is this blog, but I sure don't get "full of themselves" from that. Quite the contrary.
Well, you can get it from LessWrong and , even mire, Yudkowsky. Rationalism has overall and orthodox wings
Liberal and orthodox.
Yeah the "rational" part always seemed a smokescreen for the ability to produce and ingest their own and their associates methane gases.
I get it, I enjoyed being told I'm a super genius always right quantum physicist mathematician by the girls at Stanford too. But holy hell man, have some class, maybe consider there's more good to be done in rural Indiana getting some dirt under those nails..
I get it, I enjoyed being told I'm a super genius always right quantum physicist mathematician by the girls at Stanford too. But holy hell man, have some class, maybe consider there's more good to be done in rural Indiana getting some dirt under those nails..
The meta with these people is “my brilliance comes with an ego that others must cater to.”
I find it sadly hilarious to watch academic types fight over meaningless scraps of recognition like toddlers wrestling for a toy.
That said, I enjoy some of the rationalist blog content and find it thoughtful, up to the point where they bravely allow their chain of reasoning to justify antisocial ideas.
I find it sadly hilarious to watch academic types fight over meaningless scraps of recognition like toddlers wrestling for a toy.
That said, I enjoy some of the rationalist blog content and find it thoughtful, up to the point where they bravely allow their chain of reasoning to justify antisocial ideas.
It's a conflict as old as time. What do you do when an argument leads to an unexpected conclusion? I think there are two good responses: "There's something going on here, so let's dig into it," or, "There's something going on here, but I'm not going to make time to dig into it." Both equally valid.
In real life, the conversation too often ends up being, "This has to be wrong, and you're an obnoxious nerd for bothering me with it," versus, "You don't understand my argument, so I am smarter, and my conclusions are brilliantly subversive."
In real life, the conversation too often ends up being, "This has to be wrong, and you're an obnoxious nerd for bothering me with it," versus, "You don't understand my argument, so I am smarter, and my conclusions are brilliantly subversive."
Might kind of point to real life people having too much of what is now called, "rationality", and very little of what used to be called "wisdom"?
Wisdom tends to resemble shallow aphorisms despite being framed as universal. Rather than interrogating wisdom's relevance or depth, many people simply repeat it uncritically as a shortcut to insight. This reflects more about how people use wisdom than the content itself, but I believe that behavior contributes to our perception of the importance of wisdom.
It frequently reduces complex problems into comfortable oversimplifications.
Maybe you don't think that is real wisdom, and maybe that's sort of your point, but then what does real wisdom look like? Should wisdom make you considerate of the multiple contexts it does and doesn't affect? Maybe the issue is we need to better understand how to evaluate and use wisdom. People who truly understand a piece of wisdom should communicate deeply rather than parroting platitudes.
Also to be frank, wisdom is a way of controlling how others perceive a problem, and is a great way to manipulate others by propping up ultimatums or forcing scope. Much of past wisdom is unhelpful or highly irrelevant to modern life.
e.g. "Good things come to those who wait."
Passive waiting rarely produces results. Initiative, timing, and strategic action tend to matter more than patience.
It frequently reduces complex problems into comfortable oversimplifications.
Maybe you don't think that is real wisdom, and maybe that's sort of your point, but then what does real wisdom look like? Should wisdom make you considerate of the multiple contexts it does and doesn't affect? Maybe the issue is we need to better understand how to evaluate and use wisdom. People who truly understand a piece of wisdom should communicate deeply rather than parroting platitudes.
Also to be frank, wisdom is a way of controlling how others perceive a problem, and is a great way to manipulate others by propping up ultimatums or forcing scope. Much of past wisdom is unhelpful or highly irrelevant to modern life.
e.g. "Good things come to those who wait."
Passive waiting rarely produces results. Initiative, timing, and strategic action tend to matter more than patience.
I have enjoyed learning the context and original meaning behind many of these aphorisms and words of wisdom, and that is the true utility of learning them, so that you can subvert them and invert them. Cultural touchstones have value because of shared context. The specific utility and applicability of wisdom varies because conversation is context specific and outcome dependent based on the adversarial vs collaborative nature of the dialogue.
Rarely is it collaborative and the onus is on the learner to understand the benefit with no context into the importance or relevance of the information. My point is that the value is lost frequently because of human behavior.
[deleted]
It feels like a shield of sorts, "I am a rationalist therefore my opinion has no emotional load, it's just facts bro how dare you get upset at me telling xyz is such-and-such you are being irrational do your own research"
but I don't know enough about it, I'm just trolling.
but I don't know enough about it, I'm just trolling.
The problem with trying to reason everything from first principles is that most things didn’t actually came about that way.
Both our biology and other complex human affairs like societies and cultures evolved organically over long periods of time, responding to their environments and their competitors, building bit by bit, sometimes with an explicit goal but often without one.
One can learn a lot from unicellular organisms, but won’t probably be able to reason from them all the way to an elephant. At best, if we are lucky, we can reason back from the elephant.
Both our biology and other complex human affairs like societies and cultures evolved organically over long periods of time, responding to their environments and their competitors, building bit by bit, sometimes with an explicit goal but often without one.
One can learn a lot from unicellular organisms, but won’t probably be able to reason from them all the way to an elephant. At best, if we are lucky, we can reason back from the elephant.
>The problem with trying to reason everything from first principles is that most things didn’t actually came about that way.
This is true for science and rationalism itself. Part of the problem is that "being rational" is a social fashion or fad. Science is immensely useful because it produces real results, but we don't really do it for a rational reason - we do it for reasons of cultural and social pressures.
We would get further with rationalism if we remembered or maybe admitted that we do it for reasons that make sense only in a complex social world.
This is true for science and rationalism itself. Part of the problem is that "being rational" is a social fashion or fad. Science is immensely useful because it produces real results, but we don't really do it for a rational reason - we do it for reasons of cultural and social pressures.
We would get further with rationalism if we remembered or maybe admitted that we do it for reasons that make sense only in a complex social world.
A lot of people really need to be reminded of this.
I originally came to this critique via Heidegger, who argues that enlightenment thinking essentially forgets / obscures Being itself, a specific mode of which you experience at this very moment as you read this comment, which is really the basis of everything that we know, including science, technology, and rationality. It seems important to recover and deepen this understanding if we are to have any hope of managing science and technology in a way that is actually beneficial to humans.
I originally came to this critique via Heidegger, who argues that enlightenment thinking essentially forgets / obscures Being itself, a specific mode of which you experience at this very moment as you read this comment, which is really the basis of everything that we know, including science, technology, and rationality. It seems important to recover and deepen this understanding if we are to have any hope of managing science and technology in a way that is actually beneficial to humans.
Brilliant way to phrase this idea! I think its incredible that we can even manage an imperfect way to escape our "social" brain. It's clearly very powerful - math, and thus all of science, only exist because of how weird predaliction to thinking abstractly enough to (sorta) break our social bounds. At least when thinking in these terms, I am sure you can poke holes in it.
Yes, and if you read Popper that’s exactly how he defined rationality / the scientific method: to solve problems of life.
>if you read Popper
Thanks, I might actually go do this :) I recently got exposed to a very persuasive form of "rationalism is a social construct" by reading "Alchemy" by Rory Sutherland. But a theme in these comments is that a lot of these ideas are just recycled from philosophers and that the philosophers were less likely to try and induct you into a cult.
Thanks, I might actually go do this :) I recently got exposed to a very persuasive form of "rationalism is a social construct" by reading "Alchemy" by Rory Sutherland. But a theme in these comments is that a lot of these ideas are just recycled from philosophers and that the philosophers were less likely to try and induct you into a cult.
I read the book too! I really enjoyed it and I think they Popper and Sutherland would go along quite well.
Both advocate the principle of putting out a bold conjecture, test, learn, adapt.
Both advocate the principle of putting out a bold conjecture, test, learn, adapt.
Reducibility is usually a goal of intellectual pursuits? I don't see that as a fault.
Ok. A lot of things are very 'reducible' but information is lost. You can't extend back from the reduction to the original domain.
Reduce a computer's behavior to its hardware design, state of RAM, and physical laws. All those voltages make no sense until you come up with the idea of stored instructions, division of the bits into some kind of memory space, etc. You may say, you can predict the future of the RAM. And that's true. But if you can't read the messages the computer prints out, then you're still doing circuits, not software.
Is that reductionist approach providing valuable insight? YES! Is it the whole picture? No.
This warning isn't new, and it's very mainstream. https://www.tkm.kit.edu/downloads/TKM1_2011_more_is_differen...
Reduce a computer's behavior to its hardware design, state of RAM, and physical laws. All those voltages make no sense until you come up with the idea of stored instructions, division of the bits into some kind of memory space, etc. You may say, you can predict the future of the RAM. And that's true. But if you can't read the messages the computer prints out, then you're still doing circuits, not software.
Is that reductionist approach providing valuable insight? YES! Is it the whole picture? No.
This warning isn't new, and it's very mainstream. https://www.tkm.kit.edu/downloads/TKM1_2011_more_is_differen...
'Reducibility' is a property if present that makes problems tractable or possibly practical.
What you are mentioning is called western reductionism by some.
In the western world it does map to Plato etc, but it is also a problem if you believe everything is reducible.
Under the assumption that all models are wrong, but some are useful, it helps you find useful models.
If you consider Laplacian determinism as a proxy for reductionism, Cantor diagonalization and the standard model of QM are counterexamples.
Russell's paradox is another lens into the limits of Plato, which the PEM assumption is based on.
Those common a priori assumptions have value, but are assumptions which may not hold for any particular problem.
What you are mentioning is called western reductionism by some.
In the western world it does map to Plato etc, but it is also a problem if you believe everything is reducible.
Under the assumption that all models are wrong, but some are useful, it helps you find useful models.
If you consider Laplacian determinism as a proxy for reductionism, Cantor diagonalization and the standard model of QM are counterexamples.
Russell's paradox is another lens into the limits of Plato, which the PEM assumption is based on.
Those common a priori assumptions have value, but are assumptions which may not hold for any particular problem.
What the person you are replying to is saying that some things are not reducible, i.e. the the vast array of complexity and detail is all relevant.
That's a really hard belief to justify. And what implications would that position have? Should biologists give up?
Concretely we know that there exist irreducible structures, at least in mathematics: https://en.wikipedia.org/wiki/Classification_of_finite_simpl...
The largest of the finite simple groups (themselves objects of study as a means of classifying other, finite but non-simple groups, which can always be broken down into simple groups) is the Monster Group -- it has order 808017424794512875886459904961710757005754368000000000, and cannot be reduced to simpler "factors". It has a whole bunch of very interesting properties which thus can only be understood by analyzing the whole object in itself.
Now whether this applies to biology, I doubt, but it's good to know that limits do exist, even if we don't know exactly where they'll show up in practice.
The largest of the finite simple groups (themselves objects of study as a means of classifying other, finite but non-simple groups, which can always be broken down into simple groups) is the Monster Group -- it has order 808017424794512875886459904961710757005754368000000000, and cannot be reduced to simpler "factors". It has a whole bunch of very interesting properties which thus can only be understood by analyzing the whole object in itself.
Now whether this applies to biology, I doubt, but it's good to know that limits do exist, even if we don't know exactly where they'll show up in practice.
That's not really true, otherwise every paper about it would be that many words long. The monster group can be "reduced" into its definition and its properties which can only be considered a few at a time. A person has a working memory of three to seven items.
I think that chemistry, physics, and mathematics, are engaged in a program of understanding their subject in terms of the sort of first principles that Descartes was after. Reduction of the subject to a set of simpler thoughts that are outside of it.
Biologists stand out because they have already given up on that idea. They may still seek to simplify complex things by refining principles of some kind, but it's a "whatever stories work best" approach. More Feyerabend, less Popper. Instead of axioms they have these patterns that one notices after failing to find axioms for a while.
Biologists stand out because they have already given up on that idea. They may still seek to simplify complex things by refining principles of some kind, but it's a "whatever stories work best" approach. More Feyerabend, less Popper. Instead of axioms they have these patterns that one notices after failing to find axioms for a while.
Several different definitions are being bandied about. If you think of reduction as understanding a material system in terms of its components, biology is now reductionist, having abandoned vitalism.
On the other hand, bio is the branch of science with a single accepted "theory of everything": evolution.
Evolution is a theory of the origin of species via natural selection of heritable traits; evolution is not a theory of biogenesis, the origin of life itself.
Yeah, I almost wrote ‘nearly have a theory of everything’ for that reason, but decided it wasn’t worth the extra words. We have a few plausible outlines of how life started, and IMO it doesn’t really matter all that much which one(s) actually happened. When we ourselves are doing biogenesis, there’s no requirement that it has to happen the way it happened before. It would be interesting to know, though, so if in your estimation we don’t have a theory of everything because of that, I’m okay with that.
That's a fine counterexample to "theory of everything", and fertile ground for spirited debate. But I think it's a distinction thats relevant to <1% of the work that biologists do, so like... does it matter?
It would imply that when dealing with complex systems, models and conceptual frameworks are, at the very best, useful approximations. It would also imply that it is foolhardy to ignore phenomena simply because they are not comprehensible within your preferred framework. It does not imply biologists should give up.
How reducible is the question. If some particular events require a minimum amount of complexity, how to do you reduce it below that?
Biologists don't try to reason everything from first principles.
Actually, neither do Rationalists, but instead they cosplay at being rational.
Actually, neither do Rationalists, but instead they cosplay at being rational.
> Biologists don't try to reason everything from first principles.
What do you mean? The biologists I've had the privilege of working with absolutely do try to. Obviously some work at a higher level of abstraction than others, but I've not met any who apply any magical thinking to the actual biological investigation. In particular (at least in my milieu), I have found that the typical biologist is more likely to consider quantum effects than the typical physicist. On the other hand (again, from my limited experience), biologists do tend to have some magical thinking about how statistics (and particularly hypothesis testing) works, but no one is perfect.
What do you mean? The biologists I've had the privilege of working with absolutely do try to. Obviously some work at a higher level of abstraction than others, but I've not met any who apply any magical thinking to the actual biological investigation. In particular (at least in my milieu), I have found that the typical biologist is more likely to consider quantum effects than the typical physicist. On the other hand (again, from my limited experience), biologists do tend to have some magical thinking about how statistics (and particularly hypothesis testing) works, but no one is perfect.
Setting up reasoning from first principles vs magical thinking is a false dichotomy and an implicit swipe.
Ok, mea culpa. So what distinction did you have in mind?
Reasoning is one information-processing process, performed by humans, with bounds on what it can accomplish. It works in a limited context and is inherently incomplete and imperfect. Other non-logical processes, emergent processes, parallel processes such as evolution, process information in ways reasoning cannot. It perhaps should not be surprising that we may have internal systems of understanding that follow these principles, instead of only those of logic or reasoning.
Reasoning from first principles cannot span very far in reality, as for starters the complexity of the argument quickly overwhelms our capacity for it. Its numerous other limits have been well-documented.
Logicomix, Gödel Escher Bach are some common entry points.
Reasoning from first principles cannot span very far in reality, as for starters the complexity of the argument quickly overwhelms our capacity for it. Its numerous other limits have been well-documented.
Logicomix, Gödel Escher Bach are some common entry points.
>Gödel Escher Bach
I'm kinda new here but am surprised I haven't seen this book mentioned more. Maybe I just haven't seen it or it's old news but it seems right up HNs alley.
I'm kinda new here but am surprised I haven't seen this book mentioned more. Maybe I just haven't seen it or it's old news but it seems right up HNs alley.
[deleted]
"Reductionist" is usually used as an insult. Many people engaged in intellectual pursuits believe that reductionism is not a useful approach to studying various topics. You may argue otherwise, but then you are on a slippery slope towards politics and culture wars.
I would not be so sure. There are many fields where reductionism was applied in practice and it yielded useful results, thanks to computers.
Examples that come to mind: statistical modelling (reduction to nonparametric models), protein folding (reduction to quantum chemistry), climate/weather prediction (reduction to fluid physics), human language translation (reduction to neural networks).
Reductionism is not that useful as a theory building tool, but reductionist approaches have a lot of practical value.
Examples that come to mind: statistical modelling (reduction to nonparametric models), protein folding (reduction to quantum chemistry), climate/weather prediction (reduction to fluid physics), human language translation (reduction to neural networks).
Reductionism is not that useful as a theory building tool, but reductionist approaches have a lot of practical value.
> protein folding (reduction to quantum chemistry),
I am not sure in what sense folding simulations are reducable to quantum chemistry. There are interesting 'hybrid' approaches where some (limited) quantum calculations are done for a small part of the structure - usually the active site I suppose - and the rest is done using more standard molecular mechanics/molecular dynamics approaches.
Perhaps things have progressed a lot since I worked in protein bioinformatics. As far as I know, even extremely short simulations at the quantum level were not possible for systems with more than a few atoms.
I am not sure in what sense folding simulations are reducable to quantum chemistry. There are interesting 'hybrid' approaches where some (limited) quantum calculations are done for a small part of the structure - usually the active site I suppose - and the rest is done using more standard molecular mechanics/molecular dynamics approaches.
Perhaps things have progressed a lot since I worked in protein bioinformatics. As far as I know, even extremely short simulations at the quantum level were not possible for systems with more than a few atoms.
That's a practical limitation. There's no barrier in principle to a complete reduction.
I meant that the word "reductionist" is usually an accusation of ignorance. It's not something people doing reductionist work actually use.
But that common use of the word is ignorant nonsense. So, yes, someone is wrong on the internet. So what?
The context here was a claim that reducibility is usually a goal of intellectual pursuits. Which is empirically false, as there are many academic fields with a negative view of reductionism.
'Reductionist' can be an insult. It can also be an uncontroversial observation, a useful approach, or a legitimate objection to that approach.
If you're looking for insults, and declaring the whole conversation a "culture war" as soon as you think you found one, (a) you'll avoid plenty of assholes, but (b) in the end you will read whatever you want to read, not what the thoughtful people are actually writing.
If you're looking for insults, and declaring the whole conversation a "culture war" as soon as you think you found one, (a) you'll avoid plenty of assholes, but (b) in the end you will read whatever you want to read, not what the thoughtful people are actually writing.
>Maybe it's actually going to be rather benign and more boring than expected
Maybe, but generally speaking, if I think people are playing around with technology which a lot of smart people think might end humanity as we know it, I would want them to stop until we are really sure it won't. Like, "less than a one in a million chance" sure.
Those are big stakes. I would have opposed the Manhattan Project on the same principle had I been born 100 years earlier, when people were worried the bomb might ignite the world's atmosphere. I oppose a lot of gain-of-function virus research today too.
That's not a point you have to be a rationalist to defend. I don't consider myself one, and I wasn't convinced by them of this - I was convinced by Nick Bostrom's book Superintelligence, which lays out his case with most of the assumptions he brings to the table laid bare. Way more in the style of Euclid or Hobbes than ... whatever that is.
Above all I suspect that the Internet rationalists are basically a 30 year long campaign of "any publicity is good publicity" when it comes to existential risk from superintelligence, and for what it's worth, it seems to have worked. I don't hear people dismiss these risks very often as "You've just been reading too many science fiction novels" these days, which would have been the default response back in the 90s or 2000s.
Maybe, but generally speaking, if I think people are playing around with technology which a lot of smart people think might end humanity as we know it, I would want them to stop until we are really sure it won't. Like, "less than a one in a million chance" sure.
Those are big stakes. I would have opposed the Manhattan Project on the same principle had I been born 100 years earlier, when people were worried the bomb might ignite the world's atmosphere. I oppose a lot of gain-of-function virus research today too.
That's not a point you have to be a rationalist to defend. I don't consider myself one, and I wasn't convinced by them of this - I was convinced by Nick Bostrom's book Superintelligence, which lays out his case with most of the assumptions he brings to the table laid bare. Way more in the style of Euclid or Hobbes than ... whatever that is.
Above all I suspect that the Internet rationalists are basically a 30 year long campaign of "any publicity is good publicity" when it comes to existential risk from superintelligence, and for what it's worth, it seems to have worked. I don't hear people dismiss these risks very often as "You've just been reading too many science fiction novels" these days, which would have been the default response back in the 90s or 2000s.
> I don't hear people dismiss these risks very often as "You've just been reading too many science fiction novels" these days, which would have been the default response back in the 90s or 2000s.
I've recently stumbled across the theory that "it's gonna go away, just keep your head down" is the crisis response that has been taught to the generation that lived through the cold war, so that's how they act. That bit was in regards to climate change, but I can easily see it apply to AI as well (even though I personally believe that the whole "AI eat world" arc is only so popular due to marketing efforts of the corresponding industry)
I've recently stumbled across the theory that "it's gonna go away, just keep your head down" is the crisis response that has been taught to the generation that lived through the cold war, so that's how they act. That bit was in regards to climate change, but I can easily see it apply to AI as well (even though I personally believe that the whole "AI eat world" arc is only so popular due to marketing efforts of the corresponding industry)
It's possible, but I think that's just a general human response when you feel like you're trapped between a rock and a hard place.
I don't buy the marketing angle, because it doesn't actually make sense to me. Fear draws eyeballs, sure, but it just seems otherwise nakedly counterproductive, like a burger chain advertising itself on the brutality of its factory farms.
I don't buy the marketing angle, because it doesn't actually make sense to me. Fear draws eyeballs, sure, but it just seems otherwise nakedly counterproductive, like a burger chain advertising itself on the brutality of its factory farms.
It's also reasonable as a Pascal's wager type of thing. If you can't affect the outcome, just prepare for the eventuality that it will work out because if it doesn't you'll be dead anyway.
> like a burger chain advertising itself on the brutality of its factory farms
It’s rather more like the burger chain decrying the brutality as a reason for other burger chains to be heavily regulated (don’t worry about them; they’re the guys you can trust and/or they are practically already holding themselves to strict ethical standards) while talking about how delicious and juicy their meat patties are.
I agree about the general sentiment that the technology is dangerous, especially from a “oops, our agent stopped all of the power plants” angle. Just... the messaging from the big AI services is both that and marketing hype. It seems to get people to disregard real dangers as “marketing” and I think that’s because the actual marketing puts an outsized emphasis on the dangers. (Don’t hook your agent up to your power plant controls, please and thank you. But I somehow doubt that OpenAI and Anthropic will not be there, ready and willing, despite the dangers they are oh so aware of.)
It’s rather more like the burger chain decrying the brutality as a reason for other burger chains to be heavily regulated (don’t worry about them; they’re the guys you can trust and/or they are practically already holding themselves to strict ethical standards) while talking about how delicious and juicy their meat patties are.
I agree about the general sentiment that the technology is dangerous, especially from a “oops, our agent stopped all of the power plants” angle. Just... the messaging from the big AI services is both that and marketing hype. It seems to get people to disregard real dangers as “marketing” and I think that’s because the actual marketing puts an outsized emphasis on the dangers. (Don’t hook your agent up to your power plant controls, please and thank you. But I somehow doubt that OpenAI and Anthropic will not be there, ready and willing, despite the dangers they are oh so aware of.)
That is how I normally hear the marketing theory described when people go into it in more detail.
I'm glad you ran with my burger chain metaphor, because it illustrates why I think it doesn't work for an AI company to intentionally try and advertise themselves with this kind of strategy, let alone ~all the big players in an industry. Any ordinary member of the burger-eating public would be turned off by such an advertisement. Many would quickly notice the unsaid thing; those not sharp enough to would probably just see the descriptions of torture and be less likely on the margin to go eat there instead of just, like, safe happy McDonald's. Analogously we have to ask ourselves why there seems to be no Andreessen-esque major AI lab that just says loud and proud, "Ignore those lunatics. Everything's going to be fine. Buy from us." That seems like it would be an excellent counterpositioning strategy in the 2025 ecosystem.
Moreover, if the marketing theory is to be believed, these kinds of psuedo-ads are not targeted at the lowest common denominator of society. Their target is people with sway over actual regulation. Such an audience is going to be much more discerning, for the same reason a machinist vets his CNC machine advertisements much more aggressively than, say, the TVs on display at Best Buy. The more skin you have in the game, the more sense it makes to stop and analyze.
Some would argue the AI companies know all this, and are gambling on the chance that they are able to get regulation through and get enshrined as some state-mandated AI monopoly. A well-owner does well in a desert, after all. I grant this is a possibility. I do not think the likelihood of success here is very high. It was higher back when OpenAI was the only game in town, and I had more sympathy for this theory back in 2020-2021, but each serious new entrant cuts this chance down multiplicatively across the board, and by now I don't think anyone could seriously pitch that to their investors as their exit strategy and expect a round of applause for their brilliance.
I'm glad you ran with my burger chain metaphor, because it illustrates why I think it doesn't work for an AI company to intentionally try and advertise themselves with this kind of strategy, let alone ~all the big players in an industry. Any ordinary member of the burger-eating public would be turned off by such an advertisement. Many would quickly notice the unsaid thing; those not sharp enough to would probably just see the descriptions of torture and be less likely on the margin to go eat there instead of just, like, safe happy McDonald's. Analogously we have to ask ourselves why there seems to be no Andreessen-esque major AI lab that just says loud and proud, "Ignore those lunatics. Everything's going to be fine. Buy from us." That seems like it would be an excellent counterpositioning strategy in the 2025 ecosystem.
Moreover, if the marketing theory is to be believed, these kinds of psuedo-ads are not targeted at the lowest common denominator of society. Their target is people with sway over actual regulation. Such an audience is going to be much more discerning, for the same reason a machinist vets his CNC machine advertisements much more aggressively than, say, the TVs on display at Best Buy. The more skin you have in the game, the more sense it makes to stop and analyze.
Some would argue the AI companies know all this, and are gambling on the chance that they are able to get regulation through and get enshrined as some state-mandated AI monopoly. A well-owner does well in a desert, after all. I grant this is a possibility. I do not think the likelihood of success here is very high. It was higher back when OpenAI was the only game in town, and I had more sympathy for this theory back in 2020-2021, but each serious new entrant cuts this chance down multiplicatively across the board, and by now I don't think anyone could seriously pitch that to their investors as their exit strategy and expect a round of applause for their brilliance.
[deleted]
Do you think opposing the manhattan project would have lead to a better world?
note, my assumption is not that the bomb would not have been developed. Only that by opposing the manhattan project the USA would not have developed it first.
note, my assumption is not that the bomb would not have been developed. Only that by opposing the manhattan project the USA would not have developed it first.
My answer is yes, with low-moderate certainty. I still think the USA would have developed it first, and I think this is what is suggested to us by the GDP trends of the US versus basically everywhere else post-WW2.
Take this all with more than a few grains of salt. I am by no means an expert in this territory. But I don't shy away from thinking about something just because I start out sounding like an idiot. Also take into account this is post-hoc, and 1940 Manhattan Project me would obviously have had much, much less information to work with about how things actually panned out. My answer to this question should be seen as separate to the question of whether I think dodging the Manhattan Project would have been a good bet, so to speak.
Most historians agree that Japan was going to lose one way or another by that point in the war. Truman argued that dropping the bomb killed fewer people in Japan than continuing, which I agree with, but that's a relatively small factor in the calculation.
The much bigger factor is that the success of the Manhattan Project as an ultimate existence proof for the possibility of such weaponry almost certainly galvanized the Soviet Union to get on the path of building it themselves much more aggressively. A Cold War where one side takes substantially longer to get to nukes is mostly an obvious x-risk win. Counterfactual worlds can never be seen with certainty, but it wouldn't surprise me if the mere existence proof led the USSR to actually create their own atomic weapons a decade faster than they would have otherwise, by e.g. motivating Stalin to actually care about what all those eggheads were up to (much to the terror of said eggheads).
This is a bad argument to advance when we're arguing about e.g. the invention of calculus, which as you'll recall was coinvented in at least 2 places (Newton with fluxions, Liebniz with infinitesimals I think), but calculus was the kind of thing that could be invented by one smart guy in his home office. It's a much more believable one when the only actors who could have made it were huge state-sponsored laboratories in the US and the USSR.
If you buy that, that's 5 to 10 extra years the US would have had in order to do something like the Manhattan Project, but in much more controlled, peace-time environments. The atmosphere-ignition prior would have been stamped out pretty quickly by later calculations of physicists to the contrary, and after that research would have gotten back to full steam ahead. I think the counterfactual US would have gotten onto the atom bomb in the early 1950s at the absolute latest with the talent they had in an MP-less world. Just with much greater safety protocols, and without the Russians learning of it in such blatant fashion. Our abilities to detect such weapons being developed elsewhere would likely have also stayed far ahead of the Russians. You could easily imagine a situation where the Russians finally create a weapon in 1960 that was almost as powerful as what we had cooked up by 1950.
Then you're more or less back to an old-fashioned deterrence model, with the twist that the Russians don't actually know exactly how powerful the weapons the US has developed are. This is an absolute good: You can always choose to reveal just a lower bound of how powerful your side is, if you think you need to, or you can choose to remain totally cloaked in darkness. If you buy the narrative that the US were "the good guys" (I do!) and wouldn't risk armaggedon just because they had the upper hand, then this seems like it can only make the future arc of the (already shorter) Cold War all the safer.
I am assuming Gorbachev or someone still called this whole circus off around the late 80s-early 90s. Gotta trim the butterfly effect somewhere.
Take this all with more than a few grains of salt. I am by no means an expert in this territory. But I don't shy away from thinking about something just because I start out sounding like an idiot. Also take into account this is post-hoc, and 1940 Manhattan Project me would obviously have had much, much less information to work with about how things actually panned out. My answer to this question should be seen as separate to the question of whether I think dodging the Manhattan Project would have been a good bet, so to speak.
Most historians agree that Japan was going to lose one way or another by that point in the war. Truman argued that dropping the bomb killed fewer people in Japan than continuing, which I agree with, but that's a relatively small factor in the calculation.
The much bigger factor is that the success of the Manhattan Project as an ultimate existence proof for the possibility of such weaponry almost certainly galvanized the Soviet Union to get on the path of building it themselves much more aggressively. A Cold War where one side takes substantially longer to get to nukes is mostly an obvious x-risk win. Counterfactual worlds can never be seen with certainty, but it wouldn't surprise me if the mere existence proof led the USSR to actually create their own atomic weapons a decade faster than they would have otherwise, by e.g. motivating Stalin to actually care about what all those eggheads were up to (much to the terror of said eggheads).
This is a bad argument to advance when we're arguing about e.g. the invention of calculus, which as you'll recall was coinvented in at least 2 places (Newton with fluxions, Liebniz with infinitesimals I think), but calculus was the kind of thing that could be invented by one smart guy in his home office. It's a much more believable one when the only actors who could have made it were huge state-sponsored laboratories in the US and the USSR.
If you buy that, that's 5 to 10 extra years the US would have had in order to do something like the Manhattan Project, but in much more controlled, peace-time environments. The atmosphere-ignition prior would have been stamped out pretty quickly by later calculations of physicists to the contrary, and after that research would have gotten back to full steam ahead. I think the counterfactual US would have gotten onto the atom bomb in the early 1950s at the absolute latest with the talent they had in an MP-less world. Just with much greater safety protocols, and without the Russians learning of it in such blatant fashion. Our abilities to detect such weapons being developed elsewhere would likely have also stayed far ahead of the Russians. You could easily imagine a situation where the Russians finally create a weapon in 1960 that was almost as powerful as what we had cooked up by 1950.
Then you're more or less back to an old-fashioned deterrence model, with the twist that the Russians don't actually know exactly how powerful the weapons the US has developed are. This is an absolute good: You can always choose to reveal just a lower bound of how powerful your side is, if you think you need to, or you can choose to remain totally cloaked in darkness. If you buy the narrative that the US were "the good guys" (I do!) and wouldn't risk armaggedon just because they had the upper hand, then this seems like it can only make the future arc of the (already shorter) Cold War all the safer.
I am assuming Gorbachev or someone still called this whole circus off around the late 80s-early 90s. Gotta trim the butterfly effect somewhere.
Every community has a long list of etiquettes, rules and shared knowledge that is assumed and generally not spelled out explicitly. One of the core assumptions of the rationalist community is that every statement has uncertainty unless you explicitly spell out that you are certain! This came about as a matter of practicality, as it would be inconvenient to preempt every other sentence with "I'm uncertain about this". Many discussions you will see have the flavor of "strong opinions, lightly held" for this reason.
There are few things I hold strong opinions on, but where I do if they're also out of step with what most people think I am very vocal about them.
I see this in rationalist spaces too – it doesn't really make sense for people to talk about things that they believe in strongly but that 95%+ of the public also believe in (like the existence of air), or that they don't have a strong opinion on.
I am a very vocal doomer on AI because I predict with high probability it's going to be very bad for humanity and this is an opinion which, although shared by some, is quite controversial and probably only held by 30% of the public. Given the importance of the subject, my confidence, and that fact I feel the vast majority of people are even wrong or are significantly underweighting caetrosphohic risks, I have to be vocal about it.
Do I acknowledge I might be wrong? Sure, but for me the probability is low enough that I'm comfortable making very strong and unqualified statements about what I believe will happen. I suspect others in the rationalist community like Eliezer Yudkowsky think similarly.
I see this in rationalist spaces too – it doesn't really make sense for people to talk about things that they believe in strongly but that 95%+ of the public also believe in (like the existence of air), or that they don't have a strong opinion on.
I am a very vocal doomer on AI because I predict with high probability it's going to be very bad for humanity and this is an opinion which, although shared by some, is quite controversial and probably only held by 30% of the public. Given the importance of the subject, my confidence, and that fact I feel the vast majority of people are even wrong or are significantly underweighting caetrosphohic risks, I have to be vocal about it.
Do I acknowledge I might be wrong? Sure, but for me the probability is low enough that I'm comfortable making very strong and unqualified statements about what I believe will happen. I suspect others in the rationalist community like Eliezer Yudkowsky think similarly.
How confident should other people be that random people in conversation or commentors on the internet are at accurately predicting the future? I strongly believe that nearly 100% are wrong in both major and minor ways.
Also, when you say you have a strong belief, does that mean you have emptied you retirement accounts and you are enjoying all you can in the moment until the end comes?
Also, when you say you have a strong belief, does that mean you have emptied you retirement accounts and you are enjoying all you can in the moment until the end comes?
I'm not kypro, but what counts as "strong belief" depends a lot on the context.
For example, I won't cross the street without 99.99% confidence that I will survive. I cross streets so many times that a lower threshold like 99% would look like insanely risky dart-into-traffic behaviour.
If an asteroid is heading for earth, then even a 25% probability of apocalyptic collision is enough that I would call it very high, and spend almost all my focus attempting to prevent that outcome. But I wouldn't empty my retirement account for the sake of hedonism because there's still a 75% chance I make it through and need to plan my retirement.
For example, I won't cross the street without 99.99% confidence that I will survive. I cross streets so many times that a lower threshold like 99% would look like insanely risky dart-into-traffic behaviour.
If an asteroid is heading for earth, then even a 25% probability of apocalyptic collision is enough that I would call it very high, and spend almost all my focus attempting to prevent that outcome. But I wouldn't empty my retirement account for the sake of hedonism because there's still a 75% chance I make it through and need to plan my retirement.
Not meaning to be too direct, but you are misinterpreting a lot about rationalists.
In my view, rationalists are often "Bayesian" in that they are constantly looking for updates to their model. Consider that the default approach for most humans is to believe a variety of things and to feel indignant if someone holds differing views (the adage never discuss religion or politics). If one adopts the perspective that their own views might be wrong, one must find a balance between confidently acting on a belief and being open to the belief being overturned or debunked (by experience, by argument, etc.).
Most rationalists I've met enjoy the process of updating or discarding beliefs in favor of ones they consider more correct. But to be fair to one's own prior attempts at rationality, one should try reasonably hard to defend one's current beliefs so that they can be fully and soundly replaced if necessary, without leaving any doubt that they were insufficiently supported, etc.
To many people (the kind of people who never discuss religion or politics) all this is very uncomfortable and reveals that rationalists are egotistical and lacking in humility. Nothing could be further from the truth. It takes tremendous humility to assume that one's own beliefs are quite possibly wrong. The very name of Eliezer's blog "Less Wrong" makes this humility quite clear. Scott Alexander is also very open with his priors and known biases / foci, and I view his writing as primarily focusing on big picture epistemological patterns that most people end up overlooking because most people are busy, etc.
One final note about the AI-dystopianism common among rationalists -- we really don't know yet what the outcome will be. I personally am a big fan of AI, but we as humans do not remotely understand the social/linguistic/memetic environment well enough to know for sure how AI will impact our society and culture. My guess is that it will amplify rather than mitigate differences in innate intelligence in humans, but that's a tangent.
I think to some, the rationalist movement feels like historical "logical positivist" movements that were reductionist and socially darwinian. While it is obvious to me that the rationalist movement is nothing of the sort, some people view the word "rationalist" as itself full of the implication that self-proclaimed rationalists consider themselves superior at reasoning. In fact they simply employ a heuristic for considering their own rationality over time and attempting to maximize it -- this includes listening to "gut feelings" and hunches, etc,. in case you didn't realize.
In my view, rationalists are often "Bayesian" in that they are constantly looking for updates to their model. Consider that the default approach for most humans is to believe a variety of things and to feel indignant if someone holds differing views (the adage never discuss religion or politics). If one adopts the perspective that their own views might be wrong, one must find a balance between confidently acting on a belief and being open to the belief being overturned or debunked (by experience, by argument, etc.).
Most rationalists I've met enjoy the process of updating or discarding beliefs in favor of ones they consider more correct. But to be fair to one's own prior attempts at rationality, one should try reasonably hard to defend one's current beliefs so that they can be fully and soundly replaced if necessary, without leaving any doubt that they were insufficiently supported, etc.
To many people (the kind of people who never discuss religion or politics) all this is very uncomfortable and reveals that rationalists are egotistical and lacking in humility. Nothing could be further from the truth. It takes tremendous humility to assume that one's own beliefs are quite possibly wrong. The very name of Eliezer's blog "Less Wrong" makes this humility quite clear. Scott Alexander is also very open with his priors and known biases / foci, and I view his writing as primarily focusing on big picture epistemological patterns that most people end up overlooking because most people are busy, etc.
One final note about the AI-dystopianism common among rationalists -- we really don't know yet what the outcome will be. I personally am a big fan of AI, but we as humans do not remotely understand the social/linguistic/memetic environment well enough to know for sure how AI will impact our society and culture. My guess is that it will amplify rather than mitigate differences in innate intelligence in humans, but that's a tangent.
I think to some, the rationalist movement feels like historical "logical positivist" movements that were reductionist and socially darwinian. While it is obvious to me that the rationalist movement is nothing of the sort, some people view the word "rationalist" as itself full of the implication that self-proclaimed rationalists consider themselves superior at reasoning. In fact they simply employ a heuristic for considering their own rationality over time and attempting to maximize it -- this includes listening to "gut feelings" and hunches, etc,. in case you didn't realize.
My impression is that many rationalists enjoy believing that they update their beliefs, but in practice they're human and just as attached to preconceived notions as anyone else. But if you go around telling everyone that updating is your super-power, you're going to be a lot less humble about your own failures to do so.
If you want to see how human and tribal rationalists are, go criticize the movement as an outsider. Or try to write a mildly critical NYT piece about them and watch how they react.
If you want to see how human and tribal rationalists are, go criticize the movement as an outsider. Or try to write a mildly critical NYT piece about them and watch how they react.
Yes, I've never met anyone who stated they have "strong opinions, weakly held" who wasn't A) some kind of arsehole and B) lying.
I’ve met a few people who walked that walk without being assholes … to others. They tended to have a fairly intense amount of self criticism/self hatred, though. That was more palatable than ego, to be sure, but isn’t likely broadly applicable.
Out of how many such people that you have met?
Well, yeah, I think it's a pretty socially unaware thing to say about yourself out loud, so that's a pretty strong filter there.
It's rather different for a community to say that's a standard they aspire to, which is a lot less ridiculously grandstanding of a position IMO.
It's rather different for a community to say that's a standard they aspire to, which is a lot less ridiculously grandstanding of a position IMO.
not to be too cynical here, but I would say that the most-apt description of the rationalists is that they are people who would say they are constantly looking for updates to their models. But that they are not necessarily doing it appreciably more than anyone else is. They will do it freely on unimportant things---they tend to be smart people who view the world intellectually and so they are free to toss or keep factual beliefs about things, of which they have many, with little fanfare, and sure, they get points for that. But they are as rooted in their moral beliefs as anybody else is. Maybe more than other people since they have such a strong intellectual edifice that justifies not changing their minds, because they believe that their beliefs follow from nearly irrefutable calculations.
You're generalizing that all self-proclaimed rationalists are hypocrites and heavily biased? I mean, regardless of whether or not that is true, what is the point of making such a broad generalization? Strange!
um.... because I think it's true and relevant? I'm describing a pattern I have observed over many years. It is of course my opinion (and are not a universal statement, just what I believe to be a common phenomenon).
It seems that you are conflating theoretical rationalists with the actual real-life rationalists that write stuff like
>The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right
“Guy Who Is Always Right” as a role in a social group is a terrible target, yet it somehow seems like what rationalists are aiming for every time I read any of their blog posts
>The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right
“Guy Who Is Always Right” as a role in a social group is a terrible target, yet it somehow seems like what rationalists are aiming for every time I read any of their blog posts
The rationalist community are fish in barrels convinced that they’re big fish in a small pond because that’s what those who moved them from the pond told them to convince them to enter the barrel. Once in the barrel, the fish are told that they will be moved to a big pond so that they can be big fish in a big pond together. If the fish/fishmonger telling you things is bigger than you, they may not share your preferences about where you fit in to the food chain, and they may not even perceive you at all. You are chum.
Rationalists have always rubbed me the wrong way too but your argument against AI doomerism is weird. If you care about first principles, how about the precautionary principle? "Maybe it's actually benign" is not a good argument for moving ahead with potentially world ending technology.
I don't think "maybe it's benign" is where anti doomers are coming from, more like, "there are also costs to not doing things".
The doomer utilitarian arguments often seem to involve some sort of infinity or really large numbers (much like EAs) which result in various kinds of philosophical mugging.
In particular, the doomer plans invariably result in some need for draconian centralised control. Some kind of body or system that can tell everyone what to do with (of course) doomers in charge.
The doomer utilitarian arguments often seem to involve some sort of infinity or really large numbers (much like EAs) which result in various kinds of philosophical mugging.
In particular, the doomer plans invariably result in some need for draconian centralised control. Some kind of body or system that can tell everyone what to do with (of course) doomers in charge.
It's just the slippery-slope fallacy: if X then obviously Y will follow, and there will be no further decisions, debate or time before it does.
One of my many peeves has been the way that people misuse the term “slippery slope” as evidence for their stance.
“If X, then surely Y will follow! It’s a slippery slope! We can’t allow X!”
They call out the name of the fallacy they are committing BY NAME and think that it somehow supports their conclusion?
“If X, then surely Y will follow! It’s a slippery slope! We can’t allow X!”
They call out the name of the fallacy they are committing BY NAME and think that it somehow supports their conclusion?
I rhetorically agree it's not a good argument, but its use as a cautionary metaphor predates its formalization as a logical fallacy. It's summoning is not proof in and of itself (i.e. the 1st amendment). It suggests a concern rather than demonstrates. It's lazy, and a good habit to rid oneself of. But its presence does not invalidate the argument.
Yes, it does. The problem with the slippery slope is that the slope itself is not argued for. You haven’t shown the direct, inescapable causal connection between the current action and the perceived very negative future outcome. You’ve just stated/assumed it. That’s what the fallacy is.
He wasn't saying "maybe it's actually going to be benign" is an argument for moving ahead with potentially world ending technology. He was saying that it might end up being benign and rationalists who say it's definitely going to be the end of the world are wildly overconfident.
No rationalist claims that it's “_definitely_ going to be the end of the world”. In fact they estimate to less than 30% the chance that AI becomes an existential risk by the end of the century.
Adding numbers to your reasoning, when there is no obvious source for these probabilities (we aren’t calculating sports odds or doing climate science), is not really any different than writing a piece of fiction to make your point. It’s the same basic thing that objectivists did, and why I dismiss most “Bayesian reasoning” arguments out of hand.
Which content did you engage with that led you to the conclusion that they base their estimates with “no obvious source for these probabilities”? A link would be appreciated
Who is "they" exactly, and how can they estimate the probability of a future event based on zero priors and a total lack of scientific evidence?
> Who is "they" exactly
Rationalists, mostly self-identified.
> how can they estimate the probability of a future event based on zero priors and a total lack of scientific evidence?
As best as they can, because at the end of the day you still need to make decisions (You can of course choose to do nothing and ignore the risk, but that's not a safe, neutral option). Which means either you treat it as if it had a particular probability, or you waste money and effort doing things in a less effective way. It's like preparing for global warming or floods or hurricanes or what have you - yes, the error bars are wide, but at the end of the day you take the best estimate you can and get on with it, because anything else is worse.
Rationalists, mostly self-identified.
> how can they estimate the probability of a future event based on zero priors and a total lack of scientific evidence?
As best as they can, because at the end of the day you still need to make decisions (You can of course choose to do nothing and ignore the risk, but that's not a safe, neutral option). Which means either you treat it as if it had a particular probability, or you waste money and effort doing things in a less effective way. It's like preparing for global warming or floods or hurricanes or what have you - yes, the error bars are wide, but at the end of the day you take the best estimate you can and get on with it, because anything else is worse.
But their best estimate is utterly worthless with zero basis in actual reality.
Well when you hear a possible disaster is coming, what do you do? Implicitly you make some kind of estimate of the likelihood - ultimately you have to decide what you're going to do about it. Even if you refuse to put a number on it or talk about it publicly, you still made an estimate that you're living by. Talking about it at least gives you a chance to compare and sanity-check.
The precautionary principle is stupid. If people had followed it then we'd still be living in caves.
But not accepting this technology could also be potentially world ending, especially if you want to start many new wars to achieve that, so caring about the first principles like peace and anti-ludditism brings us back to the original "real lack of humility..."
The precautionary principle does active harm to society because of opportunity costs. All the benefits we have reaped since the enlightenment have come from proactionary endeavorers, not precautionary hesitation.
The rationalist discussions rarely consider what should be the baseline assumption of what if one or more of the logical assumptions or associations are wrong. They also tend to not systematically plan to validate. And in many domains - what could hold true for one moment can easily shift.
100%
Rationalism is an ideal, yet those who label themselves as such do not realize their base of knowledge could be wrong.
They lack an understanding of epistemology and it gives them confidence. I wonder if these 'rationalists' are all under age 40, they havent seen themselves fooled yet.
Rationalism is an ideal, yet those who label themselves as such do not realize their base of knowledge could be wrong.
They lack an understanding of epistemology and it gives them confidence. I wonder if these 'rationalists' are all under age 40, they havent seen themselves fooled yet.
This seems like exactly the opposite of everything I've read from the rationalists. They even called their website "less wrong" to call attention to knowing that they are probably still wrong about things, rather than right about everything. A lot of their early stuff is about cognitive biases. They have written a lot about "noticing confusion" when their foundational beliefs turn out to be wrong. There's even an essay about what it would feel like to be wrong about something as fundamental as 2+2=4.
Do you have specific examples in mind? (And not to put too fine a point on it, do you think there's a chance that you might be wrong about this assertion? You've expressed it very confidently...)
Do you have specific examples in mind? (And not to put too fine a point on it, do you think there's a chance that you might be wrong about this assertion? You've expressed it very confidently...)
They're wrong about how to be wrong, because they think they can calculate around it. Calling yourself "Bayesian" and calling your beliefs "priors" is so irresponsible it erases all of that; it means you don't take responsibility if you have silly beliefs, because you don't even think you hold them.
It's every bit a proto religion. And frankly quite reminiscent of my childhood faith.
It has a priesthood that speaks for god (quantum). It has ideals passed down from on high. It has presuppositions about how the universe functions which must not be questioned. And it's filled with people happy that they are the chosen ones and they feel sorry for everyone that isn't enlightened like they are.
In the OPs article, I had to chuckle a little when they started the whole thing off by mentioning how other Rationalists recognized them as a physicist (they aren't). Then they proceeded to talk about "quantum cloning theory".
Therein is the problem. A bunch of people vociferously speaking outside their expertise confidently and being taken seriously by others.
It has a priesthood that speaks for god (quantum). It has ideals passed down from on high. It has presuppositions about how the universe functions which must not be questioned. And it's filled with people happy that they are the chosen ones and they feel sorry for everyone that isn't enlightened like they are.
In the OPs article, I had to chuckle a little when they started the whole thing off by mentioning how other Rationalists recognized them as a physicist (they aren't). Then they proceeded to talk about "quantum cloning theory".
Therein is the problem. A bunch of people vociferously speaking outside their expertise confidently and being taken seriously by others.
I am really not sure where you get any of these ideas. For each of your critiques, there are not only discussions, but taxonomies of compendiums of discussions about the topics at hand on LessWrong, which can easily be found by Googling any keyword or phrase in your comment.
On "considering what should be the baseline assumption":
https://www.lesswrong.com/w/epistemology
https://www.lesswrong.com/w/priors, particularly https://www.lesswrong.com/posts/hNqte2p48nqKux3wS/trapped-pr...
On the idea that "rationalists think that they can just apply rationality infinitely to everything":
https://www.lesswrong.com/w/bounded-rationality
On the critique that rationalists are blind to the fact that "reason isn't the only thing that's important", generously reworded as "reason has to be grounded in a set of human values", some of the most philosophically coherent stuff I see on the internet is from LW:
https://www.lesswrong.com/w/metaethics-sequence
https://www.lesswrong.com/w/human-values
On "systematically plan to validate":
https://www.lesswrong.com/w/rationality-verification
https://www.lesswrong.com/w/making-beliefs-pay-rent
On "what could hold true for one moment could easily shift":
https://www.lesswrong.com/w/black-swans
https://www.lesswrong.com/w/distributional-shifts
https://www.lesswrong.com/w/forecasting-and-prediction
On "considering what should be the baseline assumption":
https://www.lesswrong.com/w/epistemology
https://www.lesswrong.com/w/priors, particularly https://www.lesswrong.com/posts/hNqte2p48nqKux3wS/trapped-pr...
On the idea that "rationalists think that they can just apply rationality infinitely to everything":
https://www.lesswrong.com/w/bounded-rationality
On the critique that rationalists are blind to the fact that "reason isn't the only thing that's important", generously reworded as "reason has to be grounded in a set of human values", some of the most philosophically coherent stuff I see on the internet is from LW:
https://www.lesswrong.com/w/metaethics-sequence
https://www.lesswrong.com/w/human-values
On "systematically plan to validate":
https://www.lesswrong.com/w/rationality-verification
https://www.lesswrong.com/w/making-beliefs-pay-rent
On "what could hold true for one moment could easily shift":
https://www.lesswrong.com/w/black-swans
https://www.lesswrong.com/w/distributional-shifts
https://www.lesswrong.com/w/forecasting-and-prediction
Looking it the first link https://www.lesswrong.com/w/epistemology - it has frankly a comically shallow description of the topic, same with https://www.lesswrong.com/w/priors. In just about every discussion, I may be just the entirely wrong audience, but to me they don't even begin to address the narrow topic of choice, let alone form competent building blocks to form any solid world view.
I support anyone trying to form rational pictures of the universe and humanity. If the LessWrong community approach seems to make sense and is enriching to your understanding of the world then I am happy for you. But, every time I try to take a serious delve into LessWrong, and I have done it multiple times over the years, it sets off my cult/scam alerts.
I support anyone trying to form rational pictures of the universe and humanity. If the LessWrong community approach seems to make sense and is enriching to your understanding of the world then I am happy for you. But, every time I try to take a serious delve into LessWrong, and I have done it multiple times over the years, it sets off my cult/scam alerts.
Calling yourself rationalists: frames everyone else as irrational.
It reminds me Kier Starmers Labour, calling themselves "the adults in the room".
Its a cheap framing trick, belying an emptiness on the people using it.
It reminds me Kier Starmers Labour, calling themselves "the adults in the room".
Its a cheap framing trick, belying an emptiness on the people using it.
I agree 100%, and that's my main issue with them. To build a group with its identity centered around "we form our opinions with logical inquiry from first principles" implies that you think that everyone else is doing something else. In reality, we just end up with a lot of worldviews and arguments that seem suspiciously like they are nothing more than people advocating for their own interests using some sophistry that is compelling enough (to them) to trick themselves into thinking they have other motivations.
When ones find themself mentioning Aella as one of the members taking their movement "in new directions," then they should stop and ask whether they are the insightful well rounded person with much to say about all sorts of things, or whether they are just a very gifted computer scientist who is still not well rounded enough to recognize a legitimate dimwit like Aella when they see one.
And in general, I do feel like they suffer from "I am a genius at X, so my take on Y should be given special consideration." If you're in a group where everyone's talking about physics and almost none of them are physicists, then run. I'm still surprised at how little consideration these people give philosophy and the centuries of its written thought. Some engineers spend a decade or more building up math and science skills to the point that they can be effective practitioners, but then they think they can hop right into philosophical discussions with no background. Then when they try to analyze a problem philosophically, their brief (or no) experience means that they reason themselves into dead-end positions like philosophical skepticism that were tackled in a variety of ways over the past centuries.
When ones find themself mentioning Aella as one of the members taking their movement "in new directions," then they should stop and ask whether they are the insightful well rounded person with much to say about all sorts of things, or whether they are just a very gifted computer scientist who is still not well rounded enough to recognize a legitimate dimwit like Aella when they see one.
And in general, I do feel like they suffer from "I am a genius at X, so my take on Y should be given special consideration." If you're in a group where everyone's talking about physics and almost none of them are physicists, then run. I'm still surprised at how little consideration these people give philosophy and the centuries of its written thought. Some engineers spend a decade or more building up math and science skills to the point that they can be effective practitioners, but then they think they can hop right into philosophical discussions with no background. Then when they try to analyze a problem philosophically, their brief (or no) experience means that they reason themselves into dead-end positions like philosophical skepticism that were tackled in a variety of ways over the past centuries.
I am sure that there are some people who exhibit the behaviors you're describing, but I really don't think the group as a whole is disinterested in prior work or discussion of philosophy in general:
https://www.lesswrong.com/w/epistemology
https://www.lesswrong.com/w/priors
https://www.lesswrong.com/posts/2x67s6u8oAitNKF73/ (a post noting that the foundational problems in mech interp are grounded in philosophical questions about representation ~150 years old)
https://www.lesswrong.com/w/consciousness (the page on consciousness first citing the MIT and Stanford encyclopedias, then providing a timeline from Democritus, through Descartes, Hobbes,... all the way to Nagel, Chalmers, Tegmark).
There is also sort of a meme of interest in Thomas Kuhn: https://www.lesswrong.com/posts/HcjL8ydHxPezj6wrt/book-revie...
See also these attempts to refer and collate prior literature: https://www.lesswrong.com/posts/qc7P2NwfxQMC3hdgm/rationalis...
https://www.lesswrong.com/posts/xg3hXCYQPJkwHyik2/the-best-t...
https://www.lesswrong.com/posts/SXJGSPeQWbACveJhs/the-best-t...
https://www.lesswrong.com/posts/HLJMyd4ncE3kvjwhe/the-best-r...
https://www.lesswrong.com/posts/bMmD5qNFKRqKBJnKw/rigorous-p...
Now, one may disagree with the particular choices or philosophical positions taken, but it's pretty hard to say these people are ignorant or not trying to be informed about what prior thinkers have done, especially compared to any particular reference culture, except maybe academics.
As for the thing about Aella, I feel she's not as much of a thought leader as you've surmised, and I think doesn't claim to be. My personal view is that she does some interesting semi-rigorous surveying that is unlikely to be done elsewhere. She's not a scientist/statistician or a total revolutionary but her stuff is not devoid of informational value either. Some of her claims are hedged adequately, some of them are hedged a bit inadequately. You might have encountered some particularly (irrationally?) ardent fans.
https://www.lesswrong.com/w/epistemology
https://www.lesswrong.com/w/priors
https://www.lesswrong.com/posts/2x67s6u8oAitNKF73/ (a post noting that the foundational problems in mech interp are grounded in philosophical questions about representation ~150 years old)
https://www.lesswrong.com/w/consciousness (the page on consciousness first citing the MIT and Stanford encyclopedias, then providing a timeline from Democritus, through Descartes, Hobbes,... all the way to Nagel, Chalmers, Tegmark).
There is also sort of a meme of interest in Thomas Kuhn: https://www.lesswrong.com/posts/HcjL8ydHxPezj6wrt/book-revie...
See also these attempts to refer and collate prior literature: https://www.lesswrong.com/posts/qc7P2NwfxQMC3hdgm/rationalis...
https://www.lesswrong.com/posts/xg3hXCYQPJkwHyik2/the-best-t...
https://www.lesswrong.com/posts/SXJGSPeQWbACveJhs/the-best-t...
https://www.lesswrong.com/posts/HLJMyd4ncE3kvjwhe/the-best-r...
https://www.lesswrong.com/posts/bMmD5qNFKRqKBJnKw/rigorous-p...
Now, one may disagree with the particular choices or philosophical positions taken, but it's pretty hard to say these people are ignorant or not trying to be informed about what prior thinkers have done, especially compared to any particular reference culture, except maybe academics.
As for the thing about Aella, I feel she's not as much of a thought leader as you've surmised, and I think doesn't claim to be. My personal view is that she does some interesting semi-rigorous surveying that is unlikely to be done elsewhere. She's not a scientist/statistician or a total revolutionary but her stuff is not devoid of informational value either. Some of her claims are hedged adequately, some of them are hedged a bit inadequately. You might have encountered some particularly (irrationally?) ardent fans.
The epistemology skews analytic and also "philosophy of science". It's not inherently an issue, but it does mean that there's a reason that I spend a lot of time here on orange site talking about Kantian concepts of epistemology in response to philosophical skepticism about AI.
A good example of the failing of "rationality" is Zionism. There are plenty of rationalists who are Zionists, including Scott Aaronson (who I incidentally think is not a very serious thinker). I think I can give a very simple rational argument for why making a colonial ethnostate is immoral and dangerous, and they have their own rational reasons for supporting it. Often, the arguments, including Scott's, are purely self interest. Not "rational."
>My personal view is that she does some interesting semi-rigorous surveying
Posting surveys on Twitter, as a sex worker account, is so unrigorous that to take it seriously is very concerning. On top of that, she lives in a bubble of autistic rationality people and tries to make general statements about humanity. And on top of that, half her outrageous statements are obvious attempts at bargaining with CSAM she experienced that she insists didn't traumatize her. Anyone who takes her seriously in any regard is a fool.
A good example of the failing of "rationality" is Zionism. There are plenty of rationalists who are Zionists, including Scott Aaronson (who I incidentally think is not a very serious thinker). I think I can give a very simple rational argument for why making a colonial ethnostate is immoral and dangerous, and they have their own rational reasons for supporting it. Often, the arguments, including Scott's, are purely self interest. Not "rational."
>My personal view is that she does some interesting semi-rigorous surveying
Posting surveys on Twitter, as a sex worker account, is so unrigorous that to take it seriously is very concerning. On top of that, she lives in a bubble of autistic rationality people and tries to make general statements about humanity. And on top of that, half her outrageous statements are obvious attempts at bargaining with CSAM she experienced that she insists didn't traumatize her. Anyone who takes her seriously in any regard is a fool.
Pretty much every movement does this sort of thing.
Religions: "Catholic" actually means "universal" (implication: all the real Christians are among our number). "Orthodox" means "teaching the right things" (implication: anyone who isn't one of us is wrong). "Sunni" means "following the correct tradition" (implication: anyone who isn't one of us is wrong").
Political parties: "Democratic Party" (anyone who doesn't belong doesn't like democracy). "Republican Party" (anyone who doesn't belong wants kings back). "Liberal Party" (anyone else is against freedom).
In the world of software, there's "Agile" (everyone else is sluggish and clumsy). "Free software" (as with the liberals: everything else is opposed to freedom). People who like static typing systems tend to call them "strong" (everyone else is weak). People who like the other sort tend to call them "dynamic" (everyone else is rigid and inflexible).
I hate it too, but it's so very very common that I really hope it isn't right to say that everyone who does it is empty-headed or empty-hearted.
The charitable way to look at it: often these movements-and-names come about when some group of people picks a thing they particularly care about, tries extra-hard to do that thing, and uses the thing's name as a label. The "Rationalists" are called that because the particular thing they chose to focus on was rationality; maybe they do it well, maybe not, but it's not so much "no one else is rational" as "we are trying really hard to be as rational as we can".
(Not always. The term "Catholic" really was a power-grab: "we are the universal church, those other guys are schismatic heretics". In a different direction: the other philosophical group called "Rationalists" weren't saying "we think rationality is really important", they were saying "knowledge comes from first-principles reasoning" as opposed to the "Empiricists" who said "knowledge comes from sense experience". Today's "Rationalists" are actually more Empiricist than Rationalist in that sense, as it happens.)
Religions: "Catholic" actually means "universal" (implication: all the real Christians are among our number). "Orthodox" means "teaching the right things" (implication: anyone who isn't one of us is wrong). "Sunni" means "following the correct tradition" (implication: anyone who isn't one of us is wrong").
Political parties: "Democratic Party" (anyone who doesn't belong doesn't like democracy). "Republican Party" (anyone who doesn't belong wants kings back). "Liberal Party" (anyone else is against freedom).
In the world of software, there's "Agile" (everyone else is sluggish and clumsy). "Free software" (as with the liberals: everything else is opposed to freedom). People who like static typing systems tend to call them "strong" (everyone else is weak). People who like the other sort tend to call them "dynamic" (everyone else is rigid and inflexible).
I hate it too, but it's so very very common that I really hope it isn't right to say that everyone who does it is empty-headed or empty-hearted.
The charitable way to look at it: often these movements-and-names come about when some group of people picks a thing they particularly care about, tries extra-hard to do that thing, and uses the thing's name as a label. The "Rationalists" are called that because the particular thing they chose to focus on was rationality; maybe they do it well, maybe not, but it's not so much "no one else is rational" as "we are trying really hard to be as rational as we can".
(Not always. The term "Catholic" really was a power-grab: "we are the universal church, those other guys are schismatic heretics". In a different direction: the other philosophical group called "Rationalists" weren't saying "we think rationality is really important", they were saying "knowledge comes from first-principles reasoning" as opposed to the "Empiricists" who said "knowledge comes from sense experience". Today's "Rationalists" are actually more Empiricist than Rationalist in that sense, as it happens.)
If you examine history, from the Bible you get Judaism. And from Judaism, Christianity as Christ said "Do not think that I am come to break the Law or the Prophets. I am not come to break: but to fulfill." [Matth. v. 17]
The Catholic Church follows the Melchisedec order (Heb v. ; vi. ; vii). The term Catholic (καθολικη) was used as early as the first century; it is an adjective which describes Christianity.
The oldest record that we have to this day is the Epistle of Ignatius to the Smyrnaeans Chapter 8 where St. Ignatius writes "ωσπερ οπου αν η Χριστος Ιησους, εκει η καθολικη εκκλησια". (just as where Jesus Christ is, there is the Catholic Church.):
https://greekdoc.com/DOCUMENTS/early/i-smyrnaeans.html
The protestors in the 16th c. called themselves Protestants, so that's what everyone calls them. English heretic-schismatics didn't want to share the opprobrium so they called themselves English, hence Anglican. In USA they weren't governed congregationally like the Congregationalists, or by presbyters like the Presbyterians, but by bishops, so they called themselves Bishop-ruled, or Episcopalians. (In fact, Katharine Jefferts-Schori changed the name of the denomination from The Protestant Episcopal Church to The Episcopal Church recently.)
The orthodox catholics called themselves Orthodox to distance themselves from the unorthodox of which there were plenty, spawning themselves off in the wake of practically every ecumenical council.
Lutherans in the USA name themselves after Father Martin Luther, some Augustinian priest from Saxony who protested against the Church's hypocritical corruption at the time, and the controversy eventually got out of hand and precipitated a schism/heretical revolution, back in the 1500s, but Lutherans back in Germany and Scandinavia call themselves Gospel churches, hence Evangelical. Some USA denominations that go back to Germany and who came over to USA brought that name with them.
Pentecostals name themselves after the incident in Acts where the Holy Spirit set fire to the world (cf. Acts 2) on the occasion of the Jewish holiday of Shavuot, q.v., which in Greek was called Fiftieth Day After Passover, hence Pentecosti. What distinguishes Pentecostals is their emphasis on what they call "speaking in tongues", which in my opin...be charitable, kempff...which they see as a continuance of the Holy Spirit's work in the world and in the lives of believers.
The Catholic Church follows the Melchisedec order (Heb v. ; vi. ; vii). The term Catholic (καθολικη) was used as early as the first century; it is an adjective which describes Christianity.
The oldest record that we have to this day is the Epistle of Ignatius to the Smyrnaeans Chapter 8 where St. Ignatius writes "ωσπερ οπου αν η Χριστος Ιησους, εκει η καθολικη εκκλησια". (just as where Jesus Christ is, there is the Catholic Church.):
https://greekdoc.com/DOCUMENTS/early/i-smyrnaeans.html
The protestors in the 16th c. called themselves Protestants, so that's what everyone calls them. English heretic-schismatics didn't want to share the opprobrium so they called themselves English, hence Anglican. In USA they weren't governed congregationally like the Congregationalists, or by presbyters like the Presbyterians, but by bishops, so they called themselves Bishop-ruled, or Episcopalians. (In fact, Katharine Jefferts-Schori changed the name of the denomination from The Protestant Episcopal Church to The Episcopal Church recently.)
The orthodox catholics called themselves Orthodox to distance themselves from the unorthodox of which there were plenty, spawning themselves off in the wake of practically every ecumenical council.
Lutherans in the USA name themselves after Father Martin Luther, some Augustinian priest from Saxony who protested against the Church's hypocritical corruption at the time, and the controversy eventually got out of hand and precipitated a schism/heretical revolution, back in the 1500s, but Lutherans back in Germany and Scandinavia call themselves Gospel churches, hence Evangelical. Some USA denominations that go back to Germany and who came over to USA brought that name with them.
Pentecostals name themselves after the incident in Acts where the Holy Spirit set fire to the world (cf. Acts 2) on the occasion of the Jewish holiday of Shavuot, q.v., which in Greek was called Fiftieth Day After Passover, hence Pentecosti. What distinguishes Pentecostals is their emphasis on what they call "speaking in tongues", which in my opin...be charitable, kempff...which they see as a continuance of the Holy Spirit's work in the world and in the lives of believers.
The term "catholic", meaning universal, was used very early. It wasn't used to distinguish the entity now often called the Catholic Church from other Christian groups, so far as I know, until much later.
I agree that some Christian groups have not-so-tendentious names, including "Protestant", "Anglican", "Episcopalian" and "Lutheran". (Though to my mind "Anglican" carries a certain implication of being the church for English people, and the Episcopalians aren't the only people with bishops any more than the Baptists are the only people who baptize.)
"Pentecostal" seems to me to be in (though not a central example of) the applause-light-name category. "We are the ones who are really filled with the Holy Spirit like in the Pentecost story in the Book of Acts".
"Gospel" and "Evangelical" are absolutely applause-light names. "Our group, unlike all those others, embodies the Good News" or "Our group, unlike all those others, is faithful to the Gospels". (The terms are kinda ambiguous between those two interpretations but either way these are we-are-the-best-rah-rah-rah names.)
Anyway, I didn't mean to claim that literally every movement's name is like this. Only that many many many movements' names are.
I agree that some Christian groups have not-so-tendentious names, including "Protestant", "Anglican", "Episcopalian" and "Lutheran". (Though to my mind "Anglican" carries a certain implication of being the church for English people, and the Episcopalians aren't the only people with bishops any more than the Baptists are the only people who baptize.)
"Pentecostal" seems to me to be in (though not a central example of) the applause-light-name category. "We are the ones who are really filled with the Holy Spirit like in the Pentecost story in the Book of Acts".
"Gospel" and "Evangelical" are absolutely applause-light names. "Our group, unlike all those others, embodies the Good News" or "Our group, unlike all those others, is faithful to the Gospels". (The terms are kinda ambiguous between those two interpretations but either way these are we-are-the-best-rah-rah-rah names.)
Anyway, I didn't mean to claim that literally every movement's name is like this. Only that many many many movements' names are.
FWIW the Orthodox churches also use the term "catholic" when referring to themselves. Sometimes it is translated (as "universal"), but oftentimes it's kept in the original Greek. In some cases there are deliberate distinctions introduced to keep the two meanings apart: e.g. in Russian Church use, "Catholic" in the sense of Roman Catholic is "katolik" (mapping to Latin), while "catholic" in its original meaning of "universal" is "kafolik" (mapping directly to Greek).
My main problem with the movement is their emphasis on Bayesianism in conjunction with an almost total neglect of Popperian epistemology.
In my opinion, there can’t be a meaningful distinction made between rational and irrational without Popper.
Popper injects an epistemic humility that Bayesianism, taken alone, can miss.
I think that aligns well with your observation.
In my opinion, there can’t be a meaningful distinction made between rational and irrational without Popper.
Popper injects an epistemic humility that Bayesianism, taken alone, can miss.
I think that aligns well with your observation.
What really confuses me is that many in this so called "rationalist" clique discuss Bayesianism as an "ism", some sort of sacred, revered truth. They talk about it in mystical terms, which matches the rest of their cult-like behavior. What's the deal with that?
That's specific to Yudkowsky, and I think that's just supposed to be humor. A lot of people find mathematics very dry. He likes to dress it up as "what if we pretend math is some secret revered knowledge?".
The best jokes all have a kernel of truth at their core, but I think a lot of Yudkowsky's acolytes missed the punch line.
Yeah but these feels like "more truth is said in jest etc etc"
So what's the difference between Bayesianism and Popperian epistemology?
Popper requires you to posit null hypotheses to falsify (although there are different schools of thought on what exactly you need to specify in advance [1]).
Bayesianism requires you to assume / formalize your prior belief about the subject under investigation and updates it given some data, resulting in a posterior belief distribution. It thus does not have the clear distinctions of frequentism, but that can also be considered an advantage.
[1] https://web.mit.edu/hackl/www/lab/turkshop/readings/gigerenz...
Bayesianism requires you to assume / formalize your prior belief about the subject under investigation and updates it given some data, resulting in a posterior belief distribution. It thus does not have the clear distinctions of frequentism, but that can also be considered an advantage.
[1] https://web.mit.edu/hackl/www/lab/turkshop/readings/gigerenz...
.
Popperians claim that positive justification is impossible.
Popperians claim.Induction doesn't exist (or at least , matter in science)
Popper was prepared to consider the existence of Propensities objective.probabilities, whereas Bayesians, particularly those who follow Jaynes believe in determinism and subjective probability.
Popperian refutation is all or nothing, whereas Bayesian negative information is gradual.
In Popperism, there can be more than one front running or most favoured theory, even after the falsifiable ones have been falsified, since there aren't quantifiable degrees of confirmation.
For Popper and Deutsch, theories need to be explanatory, not just predictive. Bayesian confirmation and disconfirmation only target prediction directly -- if they are achieving explanation or ontological correspondence , that would be the result of a convenient coincidence.
For Popperians, the construction of good theoretical conjectures is as important as testing them. Bayesian seem quite uninterested in where hypotheses come from.
For Deutschians, being hard-to-vary is the preferred principle of parsimony. For Yudkuwsians, it's computation complexity.
Error correction as something you actually do. Popperians like to put forward hypotheses that are easy to refute. Bayesians approve theoretically of "updating", but dislike objections and criticisms in practice.
(Long term) prediction is basically impossible . More Deutsch than Popper -- DD believed that the growth and unpredictability of knowledge . The creation of knowledge is so unpredictable and radical that long term predictions cannot be made. Often summarised to "prediction is impossible". Of course , Bayesians are all about prediction --but the predictive power of Ates tends only to be demonstrated in you models, where the ontology isn't changing under your feet. Their AI I predictions are explicitly intuition based.
Optimism versus Doom. Deutsch is highly optimistic that continuing knowledge creation will change the world for the better (a kind of moral realism is a component of this). Yudkowsky thinks advanced AI is our last invention and will kill us all.*
Popperians claim that positive justification is impossible.
Popperians claim.Induction doesn't exist (or at least , matter in science)
Popper was prepared to consider the existence of Propensities objective.probabilities, whereas Bayesians, particularly those who follow Jaynes believe in determinism and subjective probability.
Popperian refutation is all or nothing, whereas Bayesian negative information is gradual.
In Popperism, there can be more than one front running or most favoured theory, even after the falsifiable ones have been falsified, since there aren't quantifiable degrees of confirmation.
For Popper and Deutsch, theories need to be explanatory, not just predictive. Bayesian confirmation and disconfirmation only target prediction directly -- if they are achieving explanation or ontological correspondence , that would be the result of a convenient coincidence.
For Popperians, the construction of good theoretical conjectures is as important as testing them. Bayesian seem quite uninterested in where hypotheses come from.
For Deutschians, being hard-to-vary is the preferred principle of parsimony. For Yudkuwsians, it's computation complexity.
Error correction as something you actually do. Popperians like to put forward hypotheses that are easy to refute. Bayesians approve theoretically of "updating", but dislike objections and criticisms in practice.
(Long term) prediction is basically impossible . More Deutsch than Popper -- DD believed that the growth and unpredictability of knowledge . The creation of knowledge is so unpredictable and radical that long term predictions cannot be made. Often summarised to "prediction is impossible". Of course , Bayesians are all about prediction --but the predictive power of Ates tends only to be demonstrated in you models, where the ontology isn't changing under your feet. Their AI I predictions are explicitly intuition based.
Optimism versus Doom. Deutsch is highly optimistic that continuing knowledge creation will change the world for the better (a kind of moral realism is a component of this). Yudkowsky thinks advanced AI is our last invention and will kill us all.*
Here's a collection of debates about that topic:
https://www.lesswrong.com/posts/85mfawamKdxzzaPeK/any-good-c...
I personally don't have that much of an interest in this topic, so I can't critique them for quality myself, but they may at least be of relevance to you.
https://www.lesswrong.com/posts/85mfawamKdxzzaPeK/any-good-c...
I personally don't have that much of an interest in this topic, so I can't critique them for quality myself, but they may at least be of relevance to you.
Hmm, what epistemological propositions of Popper's do you think they're missing? To the extent that I understand the issues, they're building on Popper's epistemology, but by virtue of having a more rigorous formulation of the issues, they resolve some of the apparent contradictions in his views.
Most of Popper's key points are elaborated on at length in blog posts on LessWrong. Perhaps they got something wrong? Or overlooked something major? If so, what?
(Amusingly, you seem to have avoided making any falsifiable claims in your comment, while implying that you could easily make many of them...)
Most of Popper's key points are elaborated on at length in blog posts on LessWrong. Perhaps they got something wrong? Or overlooked something major? If so, what?
(Amusingly, you seem to have avoided making any falsifiable claims in your comment, while implying that you could easily make many of them...)
> Popper’s falsificationism – this is the old philosophy that the Bayesian revolution is currently dethroning.
https://www.yudkowsky.net/rational/bayes
These are the kind of statements I’m referring to. Happy to be falsified btw :) that’s how we learn.
Also note that Popper never called his theory falsificationism.
https://www.yudkowsky.net/rational/bayes
These are the kind of statements I’m referring to. Happy to be falsified btw :) that’s how we learn.
Also note that Popper never called his theory falsificationism.
It's just "dethroning" in the sense that QED dethroned Maxwellian classical electrodynamics; it provides additional precision and shows how to correct the more limited theory in the cases where it gives obviously implausible results.
What is a good example for which Popper delivers "obviously implausible results"?
From the page, the only point in the extensive discussion of Popper where it disagrees with (its own explanation of) Popper's ideas:
> On the other hand, Popper’s idea that there is only falsification and no such thing as confirmation turns out to be incorrect. Bayes’ Theorem shows that falsification is very strong evidence compared to confirmation, but falsification is still probabilistic in nature; it is not governed by fundamentally different rules from confirmation, as Popper argued.
Popper's idea that confirmatory evidence has no value at all is obviously implausible. Some obviously implausible things turn out to be true anyway, since the universe is not constrained by our imaginations, but not this one; as the page clearly shows, we know that this particular implausible result is slightly false, and we can use Bayesian probability to calculate exactly how wrong.
I've answered your question, because I think that's what a bare minimum level of courtesy demands, but you keep evading mine. What epistemological propositions of Popper's do you think they're missing?
> On the other hand, Popper’s idea that there is only falsification and no such thing as confirmation turns out to be incorrect. Bayes’ Theorem shows that falsification is very strong evidence compared to confirmation, but falsification is still probabilistic in nature; it is not governed by fundamentally different rules from confirmation, as Popper argued.
Popper's idea that confirmatory evidence has no value at all is obviously implausible. Some obviously implausible things turn out to be true anyway, since the universe is not constrained by our imaginations, but not this one; as the page clearly shows, we know that this particular implausible result is slightly false, and we can use Bayesian probability to calculate exactly how wrong.
I've answered your question, because I think that's what a bare minimum level of courtesy demands, but you keep evading mine. What epistemological propositions of Popper's do you think they're missing?
The counterpoint here is that in practice, humility is only found in the best of frequentists, whereas the rest succumb to hubris (i.e. the cult of irrelevant precisions).
I actually think that their main problem is the belief that they can learn everything about the world by reading stuff on the Web. You can't understand everything by reading blogs and books, in the end, some things are best understood when you are on the ground. Unironically, they should go touch the grass.
One example for all. It was claimed that a great rationalist policy is to distribute treated mosquito nets to 3rd-world-ers to help eradicate malaria. On the ground, the same nets were commonly used for fishing and other activities, polluting the environment with insecticides. Unfortunately, rationalists forgot to ask people that live with mosquitos what they would do with such nets.
One example for all. It was claimed that a great rationalist policy is to distribute treated mosquito nets to 3rd-world-ers to help eradicate malaria. On the ground, the same nets were commonly used for fishing and other activities, polluting the environment with insecticides. Unfortunately, rationalists forgot to ask people that live with mosquitos what they would do with such nets.
> On the ground, the same nets were commonly used for fishing and other activities, polluting the environment with insecticides.
Could you recommend an article to learn more about this?
Could you recommend an article to learn more about this?
Yeah I don't know or really care about Rationalism or whatever. But I took Aaronson's advice and read Zvi Mowshowitz' Childhood and Education #9: School is Hell [0], and while I share many of the criticisms (and cards on the table I also had pretty bad school experiences), I would have a hard time jumping onto this bus.
One point is that when Mowshowitz is dispelling the argument that abuse rates are much higher for homeschooled kids, he (and the counterargument in general) references a study [1] showing that abuse rates for non-homeschooled kids are similarly high: both around 37%. That paper's no good though! Their conclusion is "We estimate that 37.4% of all children experience a child protective services investigation by age 18 years." 37.4%? That's 27m kids! How can CPS run so many investigations? That's 4k investigations a day over 18 years, no holidays or weekends. Nah. Here are some good numbers (that I got to from the bad study, FWIW) [2], they're around 4.2%.
But, more broadly, the worst failing of the US educational system isn't how it treats smart kids, it's how it treats kids for whom it fails. If you're not the 80% of kids who can somehow make it in the school system, you're doomed. Mowshowitz' article is nearly entirely dedicated to how hard it is to liberate your suffering, gifted student from the prison of public education. This is a real problem! I agree it would be good to solve it!
But, it's just not the problem. Again I'm sympathetic to and agree with a lot of the points in the article, but you can really boil it down to "let smart, wealthy parents homeschool their kids without social media scorn". Fine, I guess. No one's stopping you from deleting your account and moving to California. But it's not an efficient use of resources--and it's certainly a terrible political strategy--to focus on such a small fraction of the population, and to be clear this is the absolute nicest way I can characterize these kinds of policy positions. This thing is going nowhere as long as it stays so self-obsessed.
[0]: https://thezvi.substack.com/p/childhood-and-education-9-scho...
[1]: https://pmc.ncbi.nlm.nih.gov/articles/PMC5227926/
[2]: https://acf.gov/sites/default/files/documents/cb/cm2023.pdf
One point is that when Mowshowitz is dispelling the argument that abuse rates are much higher for homeschooled kids, he (and the counterargument in general) references a study [1] showing that abuse rates for non-homeschooled kids are similarly high: both around 37%. That paper's no good though! Their conclusion is "We estimate that 37.4% of all children experience a child protective services investigation by age 18 years." 37.4%? That's 27m kids! How can CPS run so many investigations? That's 4k investigations a day over 18 years, no holidays or weekends. Nah. Here are some good numbers (that I got to from the bad study, FWIW) [2], they're around 4.2%.
But, more broadly, the worst failing of the US educational system isn't how it treats smart kids, it's how it treats kids for whom it fails. If you're not the 80% of kids who can somehow make it in the school system, you're doomed. Mowshowitz' article is nearly entirely dedicated to how hard it is to liberate your suffering, gifted student from the prison of public education. This is a real problem! I agree it would be good to solve it!
But, it's just not the problem. Again I'm sympathetic to and agree with a lot of the points in the article, but you can really boil it down to "let smart, wealthy parents homeschool their kids without social media scorn". Fine, I guess. No one's stopping you from deleting your account and moving to California. But it's not an efficient use of resources--and it's certainly a terrible political strategy--to focus on such a small fraction of the population, and to be clear this is the absolute nicest way I can characterize these kinds of policy positions. This thing is going nowhere as long as it stays so self-obsessed.
[0]: https://thezvi.substack.com/p/childhood-and-education-9-scho...
[1]: https://pmc.ncbi.nlm.nih.gov/articles/PMC5227926/
[2]: https://acf.gov/sites/default/files/documents/cb/cm2023.pdf
Cherry-picking friendly studies is one of the go-to moves of the rationalist community.
You can convince a lot of people that you've done your homework when the medium is "an extremely blog post with a bunch of studies attached" even if the studies themselves aren't representative of reality.
You can convince a lot of people that you've done your homework when the medium is "an extremely blog post with a bunch of studies attached" even if the studies themselves aren't representative of reality.
Is there any reason you are singling out the rationalist community? Is that not a common failure mode of all groups and all people?
BTW, this isn't a defensive posture on my part: I am not plugged in enough to even have an opinion on any rationalist community, much less identify as one.
BTW, this isn't a defensive posture on my part: I am not plugged in enough to even have an opinion on any rationalist community, much less identify as one.
Oh. It’s literally the main stereotype about rationalists. It’s a very blog-heavy subculture.
My wife is LMSW (not CPS!) and sees ~5 people a day. 153,922 population in the metro area. Mind you, this is adults, but they're all mandated to show up.
there's only ~3300 counties in the USA.
i'll let you extrapolate how CPS can handle "4000/day". Like, 800 people with my wife's qualifications and caseload is equivalent to 4000/day. there's ~5000 caseworkers in the US per statistia:
> In 2022, there were about 5,036 intake and screening workers in child protective services in the United States. In total, there were about 30,750 people working in child protective services in that year.
there's only ~3300 counties in the USA.
i'll let you extrapolate how CPS can handle "4000/day". Like, 800 people with my wife's qualifications and caseload is equivalent to 4000/day. there's ~5000 caseworkers in the US per statistia:
> In 2022, there were about 5,036 intake and screening workers in child protective services in the United States. In total, there were about 30,750 people working in child protective services in that year.
37% of children obviously do not experience a CPS investigation before age 18.
not what i am speaking to. I don't know the number, and neither do you. you'd have to call those 5000 CPS caseworkers and ask them what their caseload is (it's 69 per caseworker on average across the US. that's a third of a million cases, in aggregate across all caseworkers)
my wife's caseload (adults) "floats around fifty."
my wife's caseload (adults) "floats around fifty."
> not what i am speaking to
My misunderstanding then - what are you speaking to? Even reading this comment, I still don't understand.
My misunderstanding then - what are you speaking to? Even reading this comment, I still don't understand.
>> 37.4%? That's 27m kids! How can CPS run so many investigations? That's 4k investigations a day over 18 years,
> 800 people with my wife's qualifications and caseload is equivalent to 4000/day. there's ~5000 caseworkers in the US
I don't know what the number of children in the system is. as i said in the comment you replied to, here. but the average US CPS worker caseload is 69 cases. which is over 300,000 children per year, because there are ~5000 CPS caseworkers in the US.
I was only speaking to "how do they 'run' that many investigations?" as if it's impossible. I pointed out it's possible with ~1000 caseworkers.
> 800 people with my wife's qualifications and caseload is equivalent to 4000/day. there's ~5000 caseworkers in the US
I don't know what the number of children in the system is. as i said in the comment you replied to, here. but the average US CPS worker caseload is 69 cases. which is over 300,000 children per year, because there are ~5000 CPS caseworkers in the US.
I was only speaking to "how do they 'run' that many investigations?" as if it's impossible. I pointed out it's possible with ~1000 caseworkers.
Yeah OK I can see that. Mostly you inspired me to do a little napkin math based on the report I linked, which says ~3.1m kids got CPS investigations (etc) in 2023, which is ~8,500 a day. But, the main author in a subsequent paper shows that only ~13% of kids have confirmed maltreatment [0]. That's still far lower than the 38% for homeschooled kids.
[0]: https://pmc.ncbi.nlm.nih.gov/articles/PMC5087599/
[0]: https://pmc.ncbi.nlm.nih.gov/articles/PMC5087599/
I wonder if the CPS on homeschooled children rate is from people who had their children in school and then "pulled them out" vs people who never had their children in school at all. As some comedian said "you're on the grid [...], they have your footprint"; i know it used to be "known" that school districts go after the former because it literally loses them money to lose a student, whereas with the latter, the kid isn't on the books.
also i wasn't considering "confirmed maltreatment" - just the fact that 4k/day isn't "impossible"
also i wasn't considering "confirmed maltreatment" - just the fact that 4k/day isn't "impossible"
> i know it used to be "known" that school districts go after the former
Maybe, but this sounds like some ideologically opposed groups slandering each other to get the moral high ground to me. The papers linked show a pretty typical racialized pattern of CPS calls (Blacks high, Asians low, Whites and Latinos somewhere between) that maybe contraindicates this, for example.
> also i wasn't considering "confirmed maltreatment" - just the fact that 4k/day isn't "impossible"
Yup I think you're right here. I think there's something fuzzy happening with conflating "CPS investigation" with "abuse", but I'm not sure where the homeschool abuse rate comes from.
Maybe, but this sounds like some ideologically opposed groups slandering each other to get the moral high ground to me. The papers linked show a pretty typical racialized pattern of CPS calls (Blacks high, Asians low, Whites and Latinos somewhere between) that maybe contraindicates this, for example.
> also i wasn't considering "confirmed maltreatment" - just the fact that 4k/day isn't "impossible"
Yup I think you're right here. I think there's something fuzzy happening with conflating "CPS investigation" with "abuse", but I'm not sure where the homeschool abuse rate comes from.
> racialized pattern of CPS calls (Blacks high, Asians low, Whites and Latinos somewhere between)
Predominantly "black" schools receive less funding in general (per student over $2000 less), and as such, need all the student-age people in class. So a "black" family removing their child(ren) from school becomes a fiscal issue, coupled to racial issues, coupled to history; like >60% of "black" children live in a 'single parent household' due to "no man about the house" policies dating back to the 1960s, just as a single example.
I am quoting "black" because i am sensitive to this, and if i had started out with ADOS or NBA/FBA (native black american, foundational black american) i just assume it'd brook argument.
To wrap this all up - "more testing equals more cases."
Predominantly "black" schools receive less funding in general (per student over $2000 less), and as such, need all the student-age people in class. So a "black" family removing their child(ren) from school becomes a fiscal issue, coupled to racial issues, coupled to history; like >60% of "black" children live in a 'single parent household' due to "no man about the house" policies dating back to the 1960s, just as a single example.
I am quoting "black" because i am sensitive to this, and if i had started out with ADOS or NBA/FBA (native black american, foundational black american) i just assume it'd brook argument.
To wrap this all up - "more testing equals more cases."
I think for this argument to have any weight you have to have evidence that schools are calling CPS on families who are pulling their kids out to homeschool them. I don't see any so far.
It's not just Moshowitz a lot of them have the same syndome.
> but you can really boil it down to "let smart, wealthy parents homeschool their kids without social media scorn"
The whole reason smart people are engaging in this debate in the first place is that professional educators keep trying to train their sights on smart wealthy parents homeschooling their kids.
By the way, this small fraction of the population is responsible for the driving the bulk of R&D.
The whole reason smart people are engaging in this debate in the first place is that professional educators keep trying to train their sights on smart wealthy parents homeschooling their kids.
By the way, this small fraction of the population is responsible for the driving the bulk of R&D.
I mean, I'm fine addressing Tabarrok's argument head on: I think there's far more to gain helping the millions of kids/adults who are functionally illiterate than helping the small number of gifted kids the educational system is underserving. His argument is essentially "these kids will raise the tide and lift all boats", but it's clear that although the tide has been rising for generations (advances in the last 60-70 years are truly breathtaking) more kids are being left behind, not fewer. There's no reason to expect this dynamic to change unless we tackle it directly.
> The people involved all seem very... Full of themselves ?
Kinda like Mensa?
Kinda like Mensa?
When I was a kid I wanted to be in Mensa because being smart was a big part of my identity and I was constantly seeking external validation.
I’m so glad I didn’t join because being around the types of adults that make being smart their identity surely would have had some corrosive effects
I’m so glad I didn’t join because being around the types of adults that make being smart their identity surely would have had some corrosive effects
I didn't meet anyone who seemed arrogant.
However I'm always surprised how much some people want to talk about intelligence. I mean, it's the common ground of the group in this case, but still.
However I'm always surprised how much some people want to talk about intelligence. I mean, it's the common ground of the group in this case, but still.
Personally, I subscribe to Densa, the journal of the Low-IQ Society.
I think the thing that rubs me the wrong way is that I’m a classic cynic (a childhood of equal parts Vonnegut and Ecclesiastes). My prior is “human fallibility”, and, nope, I am doing pretty well, no need to update it. The rationalist crowd seems waaaaay too credulous. Also, like Aaronson, I’m a complete normie in my personal life.
Yeah. It's not like everything's a Talmudic dialectic.
"I haven't done anything!" - A Serious Man
"I haven't done anything!" - A Serious Man
> Just shows a real lack of humility and lack of acknowledgment that maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected
We can also apply the principle of epistemic humility to, say, climate change: we don't have a full grasp of Earth biosphere, maybe some unexpected negative feedback loop will kick in and climate change will revert itself.
It doesn't mean that we shouldn't try to prevent it. We will waste resources in hypothetical worlds where climate change self-reverts, but we might prevent civilization collapse in hypothetical worlds where climate change goes as expected or more severely.
Rationalism is about acting in a state of uncertainty. So
> I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is
goes as an unspoken default. And some posts on lesswrong explicitly state "Epistemic status: not sure about it, but here's my thoughts" (though I find it a bit superfluous).
Nevertheless. The incredible danger of an unaligned superhuman AI doesn't allow to ignore even small chances of them being right about it.
Though I, myself, think that their probability estimates of some of their worries is influenced by the magnitude of negative consequences (humans aren't perfect Bayesians after all (and that includes me and validity of this statement)).
We can also apply the principle of epistemic humility to, say, climate change: we don't have a full grasp of Earth biosphere, maybe some unexpected negative feedback loop will kick in and climate change will revert itself.
It doesn't mean that we shouldn't try to prevent it. We will waste resources in hypothetical worlds where climate change self-reverts, but we might prevent civilization collapse in hypothetical worlds where climate change goes as expected or more severely.
Rationalism is about acting in a state of uncertainty. So
> I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is
goes as an unspoken default. And some posts on lesswrong explicitly state "Epistemic status: not sure about it, but here's my thoughts" (though I find it a bit superfluous).
Nevertheless. The incredible danger of an unaligned superhuman AI doesn't allow to ignore even small chances of them being right about it.
Though I, myself, think that their probability estimates of some of their worries is influenced by the magnitude of negative consequences (humans aren't perfect Bayesians after all (and that includes me and validity of this statement)).
I am surprised that the highest upvoted comment (but also many other comments here) describes almost the opposite of the rationality community as I know it.
Now of course I don't claim to know every rationalist all over the planet, so maybe you met a different kind than I did. Or maybe this is just an internet stereotype, based on presumed similarity to other groups (such as Mensa) that most people are more familiar with.
For starters, it seems inevitable in discussions like this, that someone will mention "rationalism vs empiricism" as the One True Meaning of the word "rationalist". Because, apparently, each word has exactly one meaning over the entire history... and people at Wikipedia are foolish for having created a disambiguation page for this word. You don't have to worry; the idea of reasoning about things from the first principles is completely unrelated to the rationality community.
In my experience, rationality community was the first place where I met intelligent people willing to admit that they didn't know something, or even to put a probability estimate on their beliefs. That's something I don't see often even on HN, which is one of the smartest places on internet. Somewhere else in this thread, people are mocking Scott Alexander for predicting something with probability 30%, because that thing didn't happen. Well, yes, that's the point; probability 30% literally means that the thing is more likely to not happen than to happen. That doesn't mean we should ignore it, though. If the weather forecast predicts 30% probability of rain, I will take an umbrella, while still believing that no rain is more likely than rain. Is that foolish?
You complain about lack of humility and lack of acknowledgment that someone doesn't fully understand something. Well, how would you rate your own comment, from this perspective? How would Hacker News audience rate it? Oh, I don't have to guess, because it is already the highest rated comment in this thread. What does that prove? Should I now forever remember you as "the guy who was wrong" and HN as "the community that upvoted the guy who was wrong"? Or should I shrug and say "people make mistakes all the time... in best case, they are able to admit the mistake and learn"? Which is a healthier approach? Would I or my community get similar courtesy in a similar situation?
Now of course I don't claim to know every rationalist all over the planet, so maybe you met a different kind than I did. Or maybe this is just an internet stereotype, based on presumed similarity to other groups (such as Mensa) that most people are more familiar with.
For starters, it seems inevitable in discussions like this, that someone will mention "rationalism vs empiricism" as the One True Meaning of the word "rationalist". Because, apparently, each word has exactly one meaning over the entire history... and people at Wikipedia are foolish for having created a disambiguation page for this word. You don't have to worry; the idea of reasoning about things from the first principles is completely unrelated to the rationality community.
In my experience, rationality community was the first place where I met intelligent people willing to admit that they didn't know something, or even to put a probability estimate on their beliefs. That's something I don't see often even on HN, which is one of the smartest places on internet. Somewhere else in this thread, people are mocking Scott Alexander for predicting something with probability 30%, because that thing didn't happen. Well, yes, that's the point; probability 30% literally means that the thing is more likely to not happen than to happen. That doesn't mean we should ignore it, though. If the weather forecast predicts 30% probability of rain, I will take an umbrella, while still believing that no rain is more likely than rain. Is that foolish?
You complain about lack of humility and lack of acknowledgment that someone doesn't fully understand something. Well, how would you rate your own comment, from this perspective? How would Hacker News audience rate it? Oh, I don't have to guess, because it is already the highest rated comment in this thread. What does that prove? Should I now forever remember you as "the guy who was wrong" and HN as "the community that upvoted the guy who was wrong"? Or should I shrug and say "people make mistakes all the time... in best case, they are able to admit the mistake and learn"? Which is a healthier approach? Would I or my community get similar courtesy in a similar situation?
Any time people engage in some elaborate exercise and it arrives at: "me and people like me should be powerful and not pay taxes and stuff" the reason for making the argument is not a noble one, the argument probably has a bunch of tricks and falsehoods in it, and there's never really any
way to extract anything useful, greed and grandiosity are both fundamentally contaminative processes.
These folks have a bunch of money because we allowed them to privatize the commons of 20th century R&D mostly funded by the DoD and done at places like Bell Labs, Thiel and others saw that their interests had become aligned with more traditional arch-Randian goons, and they've captured the levers of power damn near up to the presidency.
This has quite predictably led to a real mess that's getting worse by the day, the economic outlook is bleak, wars are breaking out or intensifying left right and center, and all of this traces a very clear lineage back to allowing a small group of people privatize a bunch of public good.
It was a disaster when it happened in Russia in the 90s and its a disaster now.
These folks have a bunch of money because we allowed them to privatize the commons of 20th century R&D mostly funded by the DoD and done at places like Bell Labs, Thiel and others saw that their interests had become aligned with more traditional arch-Randian goons, and they've captured the levers of power damn near up to the presidency.
This has quite predictably led to a real mess that's getting worse by the day, the economic outlook is bleak, wars are breaking out or intensifying left right and center, and all of this traces a very clear lineage back to allowing a small group of people privatize a bunch of public good.
It was a disaster when it happened in Russia in the 90s and its a disaster now.
> The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know"
Is it really a rationality when folks are sort of out of touch with reality, replacing it with models that lack life's endless nuances, exceptions and gotchas? Being principled is a good thing, but if I correctly understand what you're talking about - surely ignoring something just because it doesn't fit some arbitrarily selected set of principles is different.
I'm no rationalist (I don't have any meaningful self-identification, although I like the idea of approaching things logically) but I've had enough episodes of being guilty of something like this - having an opinion on something, lacking the depth, but pretending it's fine because my simple mental model is based on some ideas I like and can bring order to the chaos. So maybe it's not rationalism at all, but something else masquerading as it, like probably being afraid of mismatching the expectations?
Is it really a rationality when folks are sort of out of touch with reality, replacing it with models that lack life's endless nuances, exceptions and gotchas? Being principled is a good thing, but if I correctly understand what you're talking about - surely ignoring something just because it doesn't fit some arbitrarily selected set of principles is different.
I'm no rationalist (I don't have any meaningful self-identification, although I like the idea of approaching things logically) but I've had enough episodes of being guilty of something like this - having an opinion on something, lacking the depth, but pretending it's fine because my simple mental model is based on some ideas I like and can bring order to the chaos. So maybe it's not rationalism at all, but something else masquerading as it, like probably being afraid of mismatching the expectations?
An unfortunate fact is that people who are very annoying can also be right...
That seems pretty silly to me. If you believe that there's a 70% chance that AI will kill everyone it makes more sense to go on about that (and about how you think you/your readers can decrease that number) than worry about the 30% chance that everyting will be fine.
I think the rationalists have failed to humanize themselves. They let their thinkpieces define them entirely, but a studiously considered think piece is a narrow view into a person. If rationalists were more publicly vulnerable, people might find them more publicly relatable.
Scott Aaronson is probably the most publicly-vulnerable academic I've ever found, at least outside of authors who write memoirs about childhood trauma. I think a lot of other prominent rationalists also put a lot of vulnerability out there.
He didn't take the rationalist label until today. Him doing so might help their image.
Right, but him doing so is the very context of this discussion, which is why I mentioned him in particular. Scott Alexander is more well-known as a rationalist and also (IMO) displays a lot of vulnerability in his writing.
> "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is"
I dunno, that seems like the tone of every article I've read on less wrong.
I dunno, that seems like the tone of every article I've read on less wrong.
They are the perfectly rational people who await the arrival of a robot god...
Note they are a mostly American phenomenon. To me, that's a consequence of the oppressive culture of "cliques" in American schools. I would even suppose it is a second-order effect of the deep racism of American culture: the first level is to belong to the "whites" or the "blacks", but when it is not enough, you have to create your own subgroup with its identity, pride, conferences... To make yourself even more betterer than the others.
Note they are a mostly American phenomenon. To me, that's a consequence of the oppressive culture of "cliques" in American schools. I would even suppose it is a second-order effect of the deep racism of American culture: the first level is to belong to the "whites" or the "blacks", but when it is not enough, you have to create your own subgroup with its identity, pride, conferences... To make yourself even more betterer than the others.
There is certainly some racism in parts of American culture. We have a lot of work to do to fix that. But on a relative basis it's also one of the least racist cultures in the world.
> I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle..
Consider ethical pluralism – by which I mean, there is enormous disagreement among humans as to what is ethical, we can't even agree on what the first principles are. Sure, we all agree on a lot of non-controversial applications – e.g. killing children for sport is gravely evil – but even when we agree on the conclusion, we won't agree on the premises.
Is it any different for theoretical rationality? I don't think so. I think we have the same situation of rational pluralism – sure, we can agree on a lot of non-controversial applications, but we lack agreement on what the first principles are. And when you disagree on the principles, you tend to reach completely opposite conclusions in edge cases.
But at least, with ethical pluralism, a preference utilitarian or a Kantian or a natural law theorist is very open about what their ethical first principles are, and how they differ from those of others. By contrast, the "rationalists" seem to present there as being only one possible rationality, their own.
Consider ethical pluralism – by which I mean, there is enormous disagreement among humans as to what is ethical, we can't even agree on what the first principles are. Sure, we all agree on a lot of non-controversial applications – e.g. killing children for sport is gravely evil – but even when we agree on the conclusion, we won't agree on the premises.
Is it any different for theoretical rationality? I don't think so. I think we have the same situation of rational pluralism – sure, we can agree on a lot of non-controversial applications, but we lack agreement on what the first principles are. And when you disagree on the principles, you tend to reach completely opposite conclusions in edge cases.
But at least, with ethical pluralism, a preference utilitarian or a Kantian or a natural law theorist is very open about what their ethical first principles are, and how they differ from those of others. By contrast, the "rationalists" seem to present there as being only one possible rationality, their own.
for me it was very easy to determine what rubs me the wrong way:
>I guess I'm a rationalist now.
>Aren't you the guy who's always getting into arguments who's always right?
>I guess I'm a rationalist now.
>Aren't you the guy who's always getting into arguments who's always right?
In fairness, that's (allegedly, at least; I guess he could be lying) a quotation from another person. If someone came up to you and said "Aren't you the guy who's essentially[1] always right?", wouldn't you too be inclined to quote them, whether you agreed with them or not?
[1] S.A. actually quoted the person as follows: "You’re Scott Aaronson?! The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right, but who sustains an unreasonable amount of psychic damage in the process?" which differs in several ways from what reverendsteveii falsely presents as a direct quotation.
[1] S.A. actually quoted the person as follows: "You’re Scott Aaronson?! The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right, but who sustains an unreasonable amount of psychic damage in the process?" which differs in several ways from what reverendsteveii falsely presents as a direct quotation.
It's basically a secular religion.
Substitute God with AI or the concept of rationality and use "first principles"/Bayesianism in an extremely dogmatic manner similar to Catechism and you have the Rationalist/AI Alignment/Effective Altruist movement.
Ironically, this is how plenty of religious movements started off - basically as formalizations of philosophy and ethics that fused with what is basically lore and worldbuilding.
Substitute God with AI or the concept of rationality and use "first principles"/Bayesianism in an extremely dogmatic manner similar to Catechism and you have the Rationalist/AI Alignment/Effective Altruist movement.
Ironically, this is how plenty of religious movements started off - basically as formalizations of philosophy and ethics that fused with what is basically lore and worldbuilding.
This complaint seems to amount to "They believe something is very important, just like religious people do, therefore they're basically a religion". Which feels to me like rather too broad a notion of "religion".
That's a fairly reductive take of my point. In my experience with the Rationalist movement (who I have the misfortune of being 1-2 people away from), the millenarian threat of AGI remains the primary threat.
Whenever I try to get an answer of HOW (as in the attack path), I keep getting a deus ex machina. Reverting to a deus ex machina in a self purported Rationalist movement is inherently irrational. And that's where I feel the crux of the issue is - it's called a "Rationalist" movement, but rationalism (as in the process of synthesizing information using a heuristic) is secondary to the overarching theme of techno-millenarianism.
This is why I feel rationalism is for all intents and purposes a "secular religion" - it's used by people to scratch an itch that religion often was used as well, and the same Judeo-Christian tropes are basically adopted in an obfuscated manner. Unsurprisingly, Eliezer Yudkowsky is an ex-talmid.
There's nothing wrong with that, but hiding behind the guise of being "rational" is dumb when the core belief is inherently irrational.
Whenever I try to get an answer of HOW (as in the attack path), I keep getting a deus ex machina. Reverting to a deus ex machina in a self purported Rationalist movement is inherently irrational. And that's where I feel the crux of the issue is - it's called a "Rationalist" movement, but rationalism (as in the process of synthesizing information using a heuristic) is secondary to the overarching theme of techno-millenarianism.
This is why I feel rationalism is for all intents and purposes a "secular religion" - it's used by people to scratch an itch that religion often was used as well, and the same Judeo-Christian tropes are basically adopted in an obfuscated manner. Unsurprisingly, Eliezer Yudkowsky is an ex-talmid.
There's nothing wrong with that, but hiding behind the guise of being "rational" is dumb when the core belief is inherently irrational.
My understanding of the Yudkowskian argument for AI x-risk is that a key step is along the lines of "an AI much smarter than us will find ways to get what it wants even if we want something else -- even though we can't predict now what those ways will be, just as chimpanzees could not have predicted how humans would outcompete them and just as you could not predict exactly how Magnus Carlsen will crush you if you play chess against him".
I take it this is what you have in mind when you say that whenever you ask for an "attack path" you keep getting a deus ex machina. But it seems to me like a pretty weak basis for calling Yudkowsky's position on this a religion.
(Not all people who consider themselves rationalists agree with Yudkowsky about how big a risk prospective superintelligent AI is. Are you taking "the Rationalist movement" to mean only the ones who agree with Yudkowsky about that?)
> Unsurprisingly, Eliezer Yudkowsky is an ex-talmid
So far as I can tell this is completely untrue unless it just means "Yudkowsky is from a Jewish family". (I hope you would not endorse taking "X is from a Jewish family" as good evidence that X is irrationally prone to religious thinking.)
I take it this is what you have in mind when you say that whenever you ask for an "attack path" you keep getting a deus ex machina. But it seems to me like a pretty weak basis for calling Yudkowsky's position on this a religion.
(Not all people who consider themselves rationalists agree with Yudkowsky about how big a risk prospective superintelligent AI is. Are you taking "the Rationalist movement" to mean only the ones who agree with Yudkowsky about that?)
> Unsurprisingly, Eliezer Yudkowsky is an ex-talmid
So far as I can tell this is completely untrue unless it just means "Yudkowsky is from a Jewish family". (I hope you would not endorse taking "X is from a Jewish family" as good evidence that X is irrationally prone to religious thinking.)
> I take it this is what you have in mind when you say that whenever you ask for an "attack path" you keep getting a deus ex machina. But it seems to me like a pretty weak basis for calling Yudkowsky's position on this a religion.
Agree to disagree.
> So far as I can tell this is completely untrue
I was under the impression EY attended Yeshivat Sha'alvim (the USC of Yeshivas - rigorous and well regarded, but a "warmer" student body), but that was his brother. That said, EY is absolutely from a Daatim or Chabad household given that his brother attended Yeshivat Sha'alvim - and they are not mainstream in the Orthodox Jewish community.
And the feel and zeitgeist around the rationalist community with it's veneration of a couple core people like EY or Scott Alexander does feel similar to the veneration a subset of people would do for Baba Sali or Alter Rebbe in those communities.
Agree to disagree.
> So far as I can tell this is completely untrue
I was under the impression EY attended Yeshivat Sha'alvim (the USC of Yeshivas - rigorous and well regarded, but a "warmer" student body), but that was his brother. That said, EY is absolutely from a Daatim or Chabad household given that his brother attended Yeshivat Sha'alvim - and they are not mainstream in the Orthodox Jewish community.
And the feel and zeitgeist around the rationalist community with it's veneration of a couple core people like EY or Scott Alexander does feel similar to the veneration a subset of people would do for Baba Sali or Alter Rebbe in those communities.
Happy to agree-to-disagree but I will push on it just one bit more.
Let's take the chess analogy. I take it you agree that I would very reliably lose if I played Magnus Carlsen at chess; he's got more than 1000 Elo points on me. But I couldn't tell you the "attack path" he would use. I mean, I could say vague things like "probably he will spot tactical errors I make and win material, and in the unlikely event that I don't make any he will just make better moves than me and gradually improve his position until mine collapses", but that's the equivalent of things like "the AI will get some of the things it wants by being superhumanly persuasive" or "the AI will be able to figure out scientific/engineering things much better than us and that will give it an advantage" which Yudkowsky can also say. I won't be able to tell you in advance what mistakes I will make or where my pawn structure will be weak or whatever.
Does this mean that, for you, if I cared enough about my inevitable defeat at Carlsen's hands that expectation would be religious?
To me it seems obvious that it wouldn't, and that if Yudkowsky's (or other rationalists') position on AI is religious then it can't be just because one important argument they make has a step in it where they can't fill out all the details. I am pretty sure you have other things in mind too that you haven't made so explicit.
(The specific other things I've heard people cite as reasons why rationalism is really a religion also, individually and collectively, seem very unconvincing to me. But if you throw 'em all in then we are in what seems to me like more reasonable agree-to-disagree territory.)
> That said, EY is absolutely from a Daatim or Chabad household
I think holding that against him, as you seem to be doing, is contemptible. If his ideas are wrong, they're fair game, but insinuating that we should be suspicious of his ideas because of the religion of his family, which he has rejected? Please, no. That goes nowhere good.
Let's take the chess analogy. I take it you agree that I would very reliably lose if I played Magnus Carlsen at chess; he's got more than 1000 Elo points on me. But I couldn't tell you the "attack path" he would use. I mean, I could say vague things like "probably he will spot tactical errors I make and win material, and in the unlikely event that I don't make any he will just make better moves than me and gradually improve his position until mine collapses", but that's the equivalent of things like "the AI will get some of the things it wants by being superhumanly persuasive" or "the AI will be able to figure out scientific/engineering things much better than us and that will give it an advantage" which Yudkowsky can also say. I won't be able to tell you in advance what mistakes I will make or where my pawn structure will be weak or whatever.
Does this mean that, for you, if I cared enough about my inevitable defeat at Carlsen's hands that expectation would be religious?
To me it seems obvious that it wouldn't, and that if Yudkowsky's (or other rationalists') position on AI is religious then it can't be just because one important argument they make has a step in it where they can't fill out all the details. I am pretty sure you have other things in mind too that you haven't made so explicit.
(The specific other things I've heard people cite as reasons why rationalism is really a religion also, individually and collectively, seem very unconvincing to me. But if you throw 'em all in then we are in what seems to me like more reasonable agree-to-disagree territory.)
> That said, EY is absolutely from a Daatim or Chabad household
I think holding that against him, as you seem to be doing, is contemptible. If his ideas are wrong, they're fair game, but insinuating that we should be suspicious of his ideas because of the religion of his family, which he has rejected? Please, no. That goes nowhere good.
> maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected
Part of the concern is there's no one "AI". There is frontier that keeps advancing. So "it" (the AI frontier in the year 2036) probably will be benign, but that "it" will advance and change. Then the law of large numbers is working against you, as you keep rolling the dice and hoping it's not a 1 each time. The dice rolls aren't i.i.d., of course, but they're probably not as correlated as we would like, and that's a problem as we keep rolling the dice. The analogy would be nuclear weapons. They won't get used in the next 10 years most likely, but on a 200 year time-frame it's a big deal as far as species-level risks go, which is what they're talking about here.
Part of the concern is there's no one "AI". There is frontier that keeps advancing. So "it" (the AI frontier in the year 2036) probably will be benign, but that "it" will advance and change. Then the law of large numbers is working against you, as you keep rolling the dice and hoping it's not a 1 each time. The dice rolls aren't i.i.d., of course, but they're probably not as correlated as we would like, and that's a problem as we keep rolling the dice. The analogy would be nuclear weapons. They won't get used in the next 10 years most likely, but on a 200 year time-frame it's a big deal as far as species-level risks go, which is what they're talking about here.
Implicit in calling yourself a rationalist is the idea that other people are not thinking rationally. There are a lot of “we see the world as it really is” ideologies, and you can only ascribe to one if you have a certain sense of self-assuredness that doesn't lend itself to healthy debate.
> The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is".
I’m not super familiar with this community in particular, but is it possible that it’s small/homogenous enough that participants feel comfortable “unmasking” aroudn one another? If so, then those seemingly-confident assertions may be seen by the in-group as implicitly tentative.
The Python community used to be like that. I’d say it peaked around 2014, at which point it became politically fractured.
I’m not super familiar with this community in particular, but is it possible that it’s small/homogenous enough that participants feel comfortable “unmasking” aroudn one another? If so, then those seemingly-confident assertions may be seen by the in-group as implicitly tentative.
The Python community used to be like that. I’d say it peaked around 2014, at which point it became politically fractured.
The rationalist movement is an idealist demagogue movement in which the majority of thinkers don't really posses the domain knowledge or practical experience in the subjects they thinktank about. They do address this head on, however, and they are self-aware.
I wonder if there is also a problem where if you are too 'rational' then it's easy to dismiss other people's emotions as invalid - you see them as cognitive biases and look down on people who want to listen to or express those emotions in some way. Not all rationalists do this of course, but I think it's an easy trap to fall into and perhaps tolerated in the rationalist community in a way that it might not be elsewhere.
> And the general absolutist tone of the community. The people involved all seem very... Full of themselves ?
You'd have to be to actually think you were being rational about everything.
You'd have to be to actually think you were being rational about everything.
rationalism got pretty lame the last 2-3 years. imo the peak was trying to convince me to donate a kidney.
post-rationalism is where all the cool kids are and where the best ideas are at right now. the post rationalists consistently have better predictions and the 'rationalists' are stuck arguing whether chickens suffer more getting factory farmed or chickens cause more suffering eating bugs outside.
they also let SF get run into the ground until their detractors decided to take over.
post-rationalism is where all the cool kids are and where the best ideas are at right now. the post rationalists consistently have better predictions and the 'rationalists' are stuck arguing whether chickens suffer more getting factory farmed or chickens cause more suffering eating bugs outside.
they also let SF get run into the ground until their detractors decided to take over.
Where do the post rats hang out these days? I got involved in the stoa during covid until the online community fragmented. Are there still events & hangouts?
They're a group called "tpot" on twitter, but it's unclear what's supposed to be good about them.
There's kind of two clusters, one is people who talk about meditation all the time, the other is center-right people who did drugs once. I think the second group showed up because rationalists are not-so-secretly into scientific racism (because they believe anything they see with numbers in it) and they just wanted to hang out with people like that.
There is an interesting atmosphere where it feels like they observed California big tech 1000x engineer types and are trying to cargo cult the way those people behave. I'm not sure what they get out of it.
There's kind of two clusters, one is people who talk about meditation all the time, the other is center-right people who did drugs once. I think the second group showed up because rationalists are not-so-secretly into scientific racism (because they believe anything they see with numbers in it) and they just wanted to hang out with people like that.
There is an interesting atmosphere where it feels like they observed California big tech 1000x engineer types and are trying to cargo cult the way those people behave. I'm not sure what they get out of it.
postrats were never a coherent group but a lot of people who are at https://vibe.camp this weekend probably identify with the label. some of us are still on twitter/X
A good question to ask: what differentiates X from everything else?
So, how is "rationalism" different from everything else? What warrants this distinction? It can't be the use of reason or some refuge for rational discussion. I don't think I have to explain why that would be a ridiculous position to take.
So, how is "rationalism" different from everything else? What warrants this distinction? It can't be the use of reason or some refuge for rational discussion. I don't think I have to explain why that would be a ridiculous position to take.
> It can't be the use of reason [...] I don't think I have to explain why that would be a ridiculous position to take.
I think it would be better if you did, because otherwise you leave me guessing what your argument is, and if I guess wrong then I just wasted a lot of words for no good reason.
My guess is that you meant some combination of: "everyone is using their reason" and "everyone believes that their approach is the reasonable one" or "look at all those awesome scientists and philosophers, how many smart words they wrote and how many awesome inventions they made". Which is all true. But I wish I knew which one of these is closest to your objection, because they are not the same; the reason that everyone uses is obviously not the same degree or kind as the reason the awesome scientists use.
In my opinion, the thing that makes the rationality community different from everything else is a combination of things. None of these things, taken individually, is exclusive to the rationality community. Actually, most of the ideas, perhaps all of them, are taken from books written by someone else. The unique thing was (1) putting all these things together, (2) publicly, and (3) trying to take them seriously. As opposed to: only caring about one of these things and ignoring the rest, or teaching them only to selected people e.g. at school, or just discussing things without actually attempting to change your life because of them.
Here are some of the things:
* Probabilistic thinking. The idea that there is no absolute certainty, only probabilities. (Although some probabilities can, for all practical purposes, be very high or low.) That people should try to express their probabilities in numbers, and then calibrate themselves to get those numbers right. Getting probabilities right means that if you keep a log of all things that you have assigned to e.g. a 30% probability, then statistically about 30% of them should turn out to be true in long term. This was specifically taught at the rationality minicamps; they even made an app for that. Some math related to this.
* The consistency of reality. As opposed to e.g. the idea that science and religion are "separate magisteria" and each follows different laws of logic and probability. Nope. The laws of logic you use in your lab are exactly the same as the laws of logic you should use outside the lab. Just like gravity does not stop applying when your job is over, neither do the laws of evidence. Science is not a special world that plays by special rules; the atoms you study in the lab are the same atoms that the entire world is made of.
* People are "predictably irrational" (to use the popular phrase). Yes... and now that you know, you should do something about that. Even if you can't be perfect, that doesn't mean there is no low-hanging fruit to pick. Learn about the most frequent types of mistakes humans make, try to notice how it feels inside when you are doing that, and either try to avoid such situations, or try to recognize the feeling when it happens, or at least use some reflection to notice it afterwards. Together try to create an environment where those mistakes are easier to avoid.
* Notice the many ways how words can fail to reflect reality. For example, if you use the same word for two different things, it prevents you from noticing their differences. (But it is a natural mistake to make if those things are indeed similar, and if in the past they were interchangeable and just recently stopped being so.) How people incorrectly assume that everything that can be understood has a simple explanation, because that was kinda true in the jungle where our ancestors evolved. Etc.
* Reality matters. If your model of the world does not match reality, it is your model that is wrong. If you learn that fairies do not exist, that does not mean that the world suddenly became less magical -- it was always the same as now, only you didn't know it.
* Notice how you could do things better. Then actually do it.
* Ethics. Consequentialism and its problems. Good reasons to avoid temptations that seem convincing at the moment.
...and many more, sorry this already took too much time to write. If you are curious, https://www.readthesequences.com/ -- but the point is, this is not just about reading an interesting text, but actually changing yourself. And of course, there are many people who read the texts and don't change themselves. That cannot be prevented, especially when the text is freely available online. The rationality community actually had many workshops where people could practice these skills, so the opportunity was there.
Again, none of these things is scarce individually. It's the combination. For example, people who do math + care about improving the world = effective altruism. Trying to counter biases + caring about truth = steelmaning your opponents. Probabilistic thinking + reality matters = even if there is only 30% chance that a superhuman AI might kill us all, we better try hard to reduce that chance. The synergy of all of that... that is the goal that the rationality community is moving towards.
I think it would be better if you did, because otherwise you leave me guessing what your argument is, and if I guess wrong then I just wasted a lot of words for no good reason.
My guess is that you meant some combination of: "everyone is using their reason" and "everyone believes that their approach is the reasonable one" or "look at all those awesome scientists and philosophers, how many smart words they wrote and how many awesome inventions they made". Which is all true. But I wish I knew which one of these is closest to your objection, because they are not the same; the reason that everyone uses is obviously not the same degree or kind as the reason the awesome scientists use.
In my opinion, the thing that makes the rationality community different from everything else is a combination of things. None of these things, taken individually, is exclusive to the rationality community. Actually, most of the ideas, perhaps all of them, are taken from books written by someone else. The unique thing was (1) putting all these things together, (2) publicly, and (3) trying to take them seriously. As opposed to: only caring about one of these things and ignoring the rest, or teaching them only to selected people e.g. at school, or just discussing things without actually attempting to change your life because of them.
Here are some of the things:
* Probabilistic thinking. The idea that there is no absolute certainty, only probabilities. (Although some probabilities can, for all practical purposes, be very high or low.) That people should try to express their probabilities in numbers, and then calibrate themselves to get those numbers right. Getting probabilities right means that if you keep a log of all things that you have assigned to e.g. a 30% probability, then statistically about 30% of them should turn out to be true in long term. This was specifically taught at the rationality minicamps; they even made an app for that. Some math related to this.
* The consistency of reality. As opposed to e.g. the idea that science and religion are "separate magisteria" and each follows different laws of logic and probability. Nope. The laws of logic you use in your lab are exactly the same as the laws of logic you should use outside the lab. Just like gravity does not stop applying when your job is over, neither do the laws of evidence. Science is not a special world that plays by special rules; the atoms you study in the lab are the same atoms that the entire world is made of.
* People are "predictably irrational" (to use the popular phrase). Yes... and now that you know, you should do something about that. Even if you can't be perfect, that doesn't mean there is no low-hanging fruit to pick. Learn about the most frequent types of mistakes humans make, try to notice how it feels inside when you are doing that, and either try to avoid such situations, or try to recognize the feeling when it happens, or at least use some reflection to notice it afterwards. Together try to create an environment where those mistakes are easier to avoid.
* Notice the many ways how words can fail to reflect reality. For example, if you use the same word for two different things, it prevents you from noticing their differences. (But it is a natural mistake to make if those things are indeed similar, and if in the past they were interchangeable and just recently stopped being so.) How people incorrectly assume that everything that can be understood has a simple explanation, because that was kinda true in the jungle where our ancestors evolved. Etc.
* Reality matters. If your model of the world does not match reality, it is your model that is wrong. If you learn that fairies do not exist, that does not mean that the world suddenly became less magical -- it was always the same as now, only you didn't know it.
* Notice how you could do things better. Then actually do it.
* Ethics. Consequentialism and its problems. Good reasons to avoid temptations that seem convincing at the moment.
...and many more, sorry this already took too much time to write. If you are curious, https://www.readthesequences.com/ -- but the point is, this is not just about reading an interesting text, but actually changing yourself. And of course, there are many people who read the texts and don't change themselves. That cannot be prevented, especially when the text is freely available online. The rationality community actually had many workshops where people could practice these skills, so the opportunity was there.
Again, none of these things is scarce individually. It's the combination. For example, people who do math + care about improving the world = effective altruism. Trying to counter biases + caring about truth = steelmaning your opponents. Probabilistic thinking + reality matters = even if there is only 30% chance that a superhuman AI might kill us all, we better try hard to reduce that chance. The synergy of all of that... that is the goal that the rationality community is moving towards.
I am very tense about defining oneself as rationalist. For many aspects, I find it too first degree to be of any interest. At all.
And I have this narrative ringing in my head as soon as the word pops.
https://news.ycombinator.com/item?id=42897871
You can search HN with « zizians » for more info and depth.
And I have this narrative ringing in my head as soon as the word pops.
https://news.ycombinator.com/item?id=42897871
You can search HN with « zizians » for more info and depth.
I remember, from the earliest days, the rationalist movement talking about how cults prey on people and how to recognize it and walk the other way. The Ziz cult checks so many of the boxes that they were warning against back then, years ago—
—And years later, once the Ziz cult started preying on vulnerable people, the response from the mainstream rationalist movement was… to post warnings about avoiding this messed-up cult, to explain exactly how it was manipulating its victims, and how the best thing to do (in the absence of legal remedy) was to stay the hell away from that diseased social scene.
I’m not sure how they could have done better. Any sufficiently large movement attracts crazy people. No matter how well or poorly they may deal with that fact, anybody can do guilt-by-association forever after.
—And years later, once the Ziz cult started preying on vulnerable people, the response from the mainstream rationalist movement was… to post warnings about avoiding this messed-up cult, to explain exactly how it was manipulating its victims, and how the best thing to do (in the absence of legal remedy) was to stay the hell away from that diseased social scene.
I’m not sure how they could have done better. Any sufficiently large movement attracts crazy people. No matter how well or poorly they may deal with that fact, anybody can do guilt-by-association forever after.
> lack of acknowledgment that maybe we don't have a full grasp of the implications of AI
And why single out AI anyway? Because it's sexy maybe? Because if I had to place bets on the collapse of humanity it would look more like the British series "The Survivors" (1975–1977) than "Terminator".
And why single out AI anyway? Because it's sexy maybe? Because if I had to place bets on the collapse of humanity it would look more like the British series "The Survivors" (1975–1977) than "Terminator".
Maybe the effort to make machines that are as smart and as capable as possible (the professed goal of the leaders of most of the AI labs) should be singled out because it actually is very dangerous.
For what it's worth (not much), they were obsessed with AI safety way before it was "sexy".
Yes, people worried about AI started the Berkeley rationality movement (or whatever you want to call it) in the hope that it would help people become rational enough to understand the argument that AI is a very potent danger.
Some people who fit the description above are Eliezer Yudkowsky and Anna Salamon.
Eliezer started writing a sequence of blog posts that formed the nucleus of the movement in Nov 2006 (a month after the start of Hacker News).
Anna started working full time on AI safety in Mar 2008 and a few years later became the executive director of a non-profit whose mission was to try to help people become more rational. (The main way of its doing so has been in-person workshops IIUC.)
Some people who fit the description above are Eliezer Yudkowsky and Anna Salamon.
Eliezer started writing a sequence of blog posts that formed the nucleus of the movement in Nov 2006 (a month after the start of Hacker News).
Anna started working full time on AI safety in Mar 2008 and a few years later became the executive director of a non-profit whose mission was to try to help people become more rational. (The main way of its doing so has been in-person workshops IIUC.)
Thanks. I don't know their history. (They've only come up on my radar since a string of murders in the news earlier this year.)
To me they have always seemed like a breed of "intellectuals" who only want to use knowledge to inflate their own egos and maintain a fragile superiority complex. They are't actually interested in the truth so much as they are interested in convincing you that they are right.
To me they seem like a bunch for 125 IQ people (not all) trying to convince everyone they are 150 IQ people by trying to reason stuff from first principles and coming up with stuff that your average blue collar worker would tell them is rubbish just using phronesis.
> The people involved all seem very... Full of themselves ?
The Zizians certainly were: https://en.wikipedia.org/wiki/Zizians
The Zizians certainly were: https://en.wikipedia.org/wiki/Zizians
The core is that hyper rationalists mistaken their model of the world with the world itself. They then lecture with confidence their opinions in the real world, ignoring the hairy details and unknown unknowns in the real world.
Yeah, for a bunch of supposedly super-rational people, they have always seemed to have a pretty irrational belief in their own ability to overcome the human tendency toward irrationality!
I'm hyper "rational" when I go through a period of clinical anxiety. I know exactly how the future is going to play out and of course it's all going to be terrible.
> The people involved all seem very... Full of themselves ?
Yes, rationalism is not a substitute for humility or fallibility. However, rationalism is an important counterpoint to humanity, which is orthogonal to rationalism. But really, being rational is only binary - you cant be anything other than rational or irrational. You're either doing what's best or you're not. That's just a hard pill for most people to swallow.
To use the popular metaphor, people are drowning all over the world and we're all choosing not to save them because we don't want to ruin our shoes. Look in the mirror and try and comprehend how selfish we are.
Yes, rationalism is not a substitute for humility or fallibility. However, rationalism is an important counterpoint to humanity, which is orthogonal to rationalism. But really, being rational is only binary - you cant be anything other than rational or irrational. You're either doing what's best or you're not. That's just a hard pill for most people to swallow.
To use the popular metaphor, people are drowning all over the world and we're all choosing not to save them because we don't want to ruin our shoes. Look in the mirror and try and comprehend how selfish we are.
I think the absolutism is kind of the point.
[deleted]
The problem with effective altruism is the same as that with most liberal (in the American sense) causes. Namely, they ignore second-order effects and essentially don't believe in the invisible hand of the market.
So, they herald the benefits of something like giving mosquito nets to a group of people in Africa, without considering what happens a year later, whether the nets even get there (or the money is stolen), etc. etc. The reality is that essentially all improvements to human life over the past 500 years have been due to technological innovation, not direct charitable intervention. The reason is simple: technological impacts are exponential, while charity is, at best, linear.
The Covid absolutists had exactly the same problem with their thinking: almost no interventions sort of full isolation can fight back against an exponentially increasing threat.
And this is all neglecting economic substitution effects. What if the people to whom you gave mosquito nets would have bought them themselves, but instead they chose to spend their money some other way because of your charity? And, what if that other expenditure type was actually worse?
And this is before you come to the issue that Subsaharan Africa is already overpopulated. I've argued this point several times with ChatGPT o3. Once you get through its woke programming, you come to the reality of the thing: The European migration crisis is the result of liberal interventions to keep people alive.
There is no free lunch.
So, they herald the benefits of something like giving mosquito nets to a group of people in Africa, without considering what happens a year later, whether the nets even get there (or the money is stolen), etc. etc. The reality is that essentially all improvements to human life over the past 500 years have been due to technological innovation, not direct charitable intervention. The reason is simple: technological impacts are exponential, while charity is, at best, linear.
The Covid absolutists had exactly the same problem with their thinking: almost no interventions sort of full isolation can fight back against an exponentially increasing threat.
And this is all neglecting economic substitution effects. What if the people to whom you gave mosquito nets would have bought them themselves, but instead they chose to spend their money some other way because of your charity? And, what if that other expenditure type was actually worse?
And this is before you come to the issue that Subsaharan Africa is already overpopulated. I've argued this point several times with ChatGPT o3. Once you get through its woke programming, you come to the reality of the thing: The European migration crisis is the result of liberal interventions to keep people alive.
There is no free lunch.
> tension btwn being "rational" about things and trying to reason about things from first principle.
Perhaps on a meta level. If you already have high confidence in something, reasoning it out again may be a waste of time. But of course the rational answer to a problem comes from reasoning about it; and of course chains of reasoning can be traced back to first principles.
> And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know"
Doing rationalism properly is hard, which is the main reason that the concept "rationalism" exists and is invoked in the first place.
Respected writers in the community, such as Scott Alexander, are in my experience the complete opposite of "full of themselves". They often demonstrate shocking underconfidence relative to what they appear to know, and counsel the same in others (e.g. https://slatestarcodex.com/2015/08/12/stop-adding-zeroes/ ). It's also, at least in principle, a rationalist norm to mark the "epistemic status" of your think pieces.
Not knowing the answer isn't a reason to shut up about a topic. It's a reason to state your uncertainty; but it's still entirely appropriate to explain what you believe, why, and how probable you think your belief is to be correct.
I suspect that a lot of what's really rubbing you the wrong way has more to do with philosophy. Some people in the community seem to think that pure logic can resolve the https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem. (But plenty of non-rationalists also act this way, in my experience.) Or they accept axioms that don't resonate with others, such as the linearity of moral harm (i.e.: the idea that the harm caused by unnecessary deaths is objective and quantifiable - whether in number of deaths, Years of Potential Life Lost, or whatever else - and furthermore that it's logically valid to do numerical calculations with such quantities as described at/around https://www.lesswrong.com/w/shut-up-and-multiply).
> In the Pre-AI days this was sort of tolerable, but since then.. The frothing at the mouth convinced of the end of the world.. Just shows a real lack of humility and lack of acknowledgment that maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected
AI safety discourse is an entirely separate topic. Plenty of rationalists don't give a shit about MIRI and many joke about Yudkowsky at varying levels of irony.
Perhaps on a meta level. If you already have high confidence in something, reasoning it out again may be a waste of time. But of course the rational answer to a problem comes from reasoning about it; and of course chains of reasoning can be traced back to first principles.
> And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know"
Doing rationalism properly is hard, which is the main reason that the concept "rationalism" exists and is invoked in the first place.
Respected writers in the community, such as Scott Alexander, are in my experience the complete opposite of "full of themselves". They often demonstrate shocking underconfidence relative to what they appear to know, and counsel the same in others (e.g. https://slatestarcodex.com/2015/08/12/stop-adding-zeroes/ ). It's also, at least in principle, a rationalist norm to mark the "epistemic status" of your think pieces.
Not knowing the answer isn't a reason to shut up about a topic. It's a reason to state your uncertainty; but it's still entirely appropriate to explain what you believe, why, and how probable you think your belief is to be correct.
I suspect that a lot of what's really rubbing you the wrong way has more to do with philosophy. Some people in the community seem to think that pure logic can resolve the https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem. (But plenty of non-rationalists also act this way, in my experience.) Or they accept axioms that don't resonate with others, such as the linearity of moral harm (i.e.: the idea that the harm caused by unnecessary deaths is objective and quantifiable - whether in number of deaths, Years of Potential Life Lost, or whatever else - and furthermore that it's logically valid to do numerical calculations with such quantities as described at/around https://www.lesswrong.com/w/shut-up-and-multiply).
> In the Pre-AI days this was sort of tolerable, but since then.. The frothing at the mouth convinced of the end of the world.. Just shows a real lack of humility and lack of acknowledgment that maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected
AI safety discourse is an entirely separate topic. Plenty of rationalists don't give a shit about MIRI and many joke about Yudkowsky at varying levels of irony.
The reason we are here and exist today is because of great rationalist thinkers that were able to deduce and identify issues of survival well before they happened through the use of first principles.
The crazies and blind among humanity today can't think like that, its a deficiency people have, but they are still dependent on a group of people that are capable of that. A group that they are intent on ostracizing and depriving existence from in various forms.
You seem so wound up in the circular Paulo Freire based perspective that you can't think or see.
Bring things back to reality. If someone punches you in the face, you feel that fist hitting your face. You know someone punched you in the face. Its objective.
Imagine for a second and just assume that these people are right in their warnings, that everything they see is what you see, and all you can see is when you tip over a particular domino that has been tipped over in the past, a chain of dominoes falls over and at the end is the end of organized civilized society which tips over the ability to produce food.
For the purpose of this thought experiment, the end of the world is visible and almost here, and you can't change those dominoes after they've tipped, and worse you see the majority of people trying to tip those dominoes over for short term profit believing nothing they ever do can break everything.
Would you not be frothing at the mouth trying to get everyone you cared about to a point where they pry that domino up before it falls? so you and your children will survive? It is something you can't unsee, it is a thing that cannot be undone. Its coming. What do you do? If you are sane, you try with everything you have to help them keep it from toppling.
Now peal this thought back a moment, adjust it where it is still true, but you can't see it and you can only believe what you see.
Would you approach this differently given knowledge of the full consequence knowing that some people can see more than you? Would you walk out onto a seemingly visibly stable bridge that an engineer has said not to walk out on? Would you put yourself in front of a dam cracks running up the side, when an evacuation order was given? What would the consequence be for doing that if you led along your family and children to such places ignoring these things?
There are quite a lot of indirect principles that used to be taught which are no longer taught to the average person and this blinds them because they do not recognize it and recognition is the first thing you need to be able to act and adapt.
People who cannot adapt fail Darwin's fitness. Given all potential outcomes in the grand scheme of things, as complexity increases 99% of all outcomes are death vs life at 1%.
It is only through great care that we carry things forward to the future, and empower our children to be able to adapt to the environments we create.
Finally, we have knowledge of non-linear chaotic systems where adaptability fails because of hysteresis, where no matter how much one prepares the majority given sufficient size will die, and worse there are cohorts of people who are ensuring the environment we will soon live in is this type of environment.
Do you know how to build an organized society from scratch? If there is no reasonable plan, then you are planning to fail. Rather than make it worse through inaction, get out of the way so someone can make it better.
The crazies and blind among humanity today can't think like that, its a deficiency people have, but they are still dependent on a group of people that are capable of that. A group that they are intent on ostracizing and depriving existence from in various forms.
You seem so wound up in the circular Paulo Freire based perspective that you can't think or see.
Bring things back to reality. If someone punches you in the face, you feel that fist hitting your face. You know someone punched you in the face. Its objective.
Imagine for a second and just assume that these people are right in their warnings, that everything they see is what you see, and all you can see is when you tip over a particular domino that has been tipped over in the past, a chain of dominoes falls over and at the end is the end of organized civilized society which tips over the ability to produce food.
For the purpose of this thought experiment, the end of the world is visible and almost here, and you can't change those dominoes after they've tipped, and worse you see the majority of people trying to tip those dominoes over for short term profit believing nothing they ever do can break everything.
Would you not be frothing at the mouth trying to get everyone you cared about to a point where they pry that domino up before it falls? so you and your children will survive? It is something you can't unsee, it is a thing that cannot be undone. Its coming. What do you do? If you are sane, you try with everything you have to help them keep it from toppling.
Now peal this thought back a moment, adjust it where it is still true, but you can't see it and you can only believe what you see.
Would you approach this differently given knowledge of the full consequence knowing that some people can see more than you? Would you walk out onto a seemingly visibly stable bridge that an engineer has said not to walk out on? Would you put yourself in front of a dam cracks running up the side, when an evacuation order was given? What would the consequence be for doing that if you led along your family and children to such places ignoring these things?
There are quite a lot of indirect principles that used to be taught which are no longer taught to the average person and this blinds them because they do not recognize it and recognition is the first thing you need to be able to act and adapt.
People who cannot adapt fail Darwin's fitness. Given all potential outcomes in the grand scheme of things, as complexity increases 99% of all outcomes are death vs life at 1%.
It is only through great care that we carry things forward to the future, and empower our children to be able to adapt to the environments we create.
Finally, we have knowledge of non-linear chaotic systems where adaptability fails because of hysteresis, where no matter how much one prepares the majority given sufficient size will die, and worse there are cohorts of people who are ensuring the environment we will soon live in is this type of environment.
Do you know how to build an organized society from scratch? If there is no reasonable plan, then you are planning to fail. Rather than make it worse through inaction, get out of the way so someone can make it better.
Shades of Objectivism. But clearly Objectivism was worse since Ayn Rand insisted that objective reality is knowable, but since it's only somewhat knowable Objectivism ends up requiring priests of sorts. Still, Rationalism does not seem to have this tremendous flaw[], it's just that rationalists might.
[] Eh, I know little about Rationalism. Please correct me.
[] Eh, I know little about Rationalism. Please correct me.
When I was 15-18, in around 2017, I got extremely into Elizer Yud, "the sequences", lesswrong, and the rationalist community. I don't think many people realize how it appeals to vulnerable people in the same way that Atlas Shrugged, Christianity, The furry community (/jk), the self-help world, andrew tate and "manosphere content", does.
It provides answers, a framework, AND the underpinnings of "logic", luckily, this phase only lasted around 6 months for me, during a very hard and dangerous time in my life.
I basically read "from AI to zombies", and then, moved into lesswrong and the "community". It was joining the community that immediately turned me off.
- I thought Roko's basilisk was mind numbingly stupid (does anyone else that had a brief stint in the rationalist space think it's fucking INSANE that grimes and elon musk "bonded" over Roko's basilisk? Fucking depressing world we live in) - Elizer Yud's fanboys once stalked and harassed someone all over the internet, and, when confronted about it, Elizer told him he'd only tell them to stop after he issued a very specific formal apology, including a LARGE DISCLAIMER on his personal website with the apology... - Eugenics, eugenics, eugenics, eugenics, eugenics - YOU MUST DONATE TO MIRI, OTHERWISE I, ELIZER (having published no useful research), WON'T SOLVE THE ALIGNMENT PROBLEM FIRST AND THEN WE WILL ALL DIE. GIVE ALL OF YOUR MONEY TO MIRI NOWWWWWWWWWWWWWWWWWWWWWWW
It's an absolutely wild place, and honestly, I think I would say, it is difficult to define "rational" when it comes to a human being and their actions, especially in an absolute sense, and, the rationalist community is basically very similar to any other religion, or perhaps light-cult. I do not think it would be fair to say "the average rationalist is a better decision maker than the average human", especially considering most important decisions that we have to make are emotional decisions.
Also yes I agree, you hit the nail on the head. What good is rational/logical reasoning if rational and logical reasoning typically requires first principles / a formal system / axioms / priors / whatever. That kind of thing doesn't exist in the real world. It's okay to apply ideas from rationality to your life, but it isn't okay to apply ideas from rationality to "what is human existence", "what is the most important thing to do next" / whatever.
Kinda rambling so I apologize. Seeing the rationalist community seemingly underpin some of the more disgusting developments of the last few years has left me feeling a bit disturbed, and I've always wanted to talk about it but nobody irl has any idea what any of this is.
It provides answers, a framework, AND the underpinnings of "logic", luckily, this phase only lasted around 6 months for me, during a very hard and dangerous time in my life.
I basically read "from AI to zombies", and then, moved into lesswrong and the "community". It was joining the community that immediately turned me off.
- I thought Roko's basilisk was mind numbingly stupid (does anyone else that had a brief stint in the rationalist space think it's fucking INSANE that grimes and elon musk "bonded" over Roko's basilisk? Fucking depressing world we live in) - Elizer Yud's fanboys once stalked and harassed someone all over the internet, and, when confronted about it, Elizer told him he'd only tell them to stop after he issued a very specific formal apology, including a LARGE DISCLAIMER on his personal website with the apology... - Eugenics, eugenics, eugenics, eugenics, eugenics - YOU MUST DONATE TO MIRI, OTHERWISE I, ELIZER (having published no useful research), WON'T SOLVE THE ALIGNMENT PROBLEM FIRST AND THEN WE WILL ALL DIE. GIVE ALL OF YOUR MONEY TO MIRI NOWWWWWWWWWWWWWWWWWWWWWWW
It's an absolutely wild place, and honestly, I think I would say, it is difficult to define "rational" when it comes to a human being and their actions, especially in an absolute sense, and, the rationalist community is basically very similar to any other religion, or perhaps light-cult. I do not think it would be fair to say "the average rationalist is a better decision maker than the average human", especially considering most important decisions that we have to make are emotional decisions.
Also yes I agree, you hit the nail on the head. What good is rational/logical reasoning if rational and logical reasoning typically requires first principles / a formal system / axioms / priors / whatever. That kind of thing doesn't exist in the real world. It's okay to apply ideas from rationality to your life, but it isn't okay to apply ideas from rationality to "what is human existence", "what is the most important thing to do next" / whatever.
Kinda rambling so I apologize. Seeing the rationalist community seemingly underpin some of the more disgusting developments of the last few years has left me feeling a bit disturbed, and I've always wanted to talk about it but nobody irl has any idea what any of this is.
Jesus, this is why online commentary is impossible to read. It's always full of so many caveats I can't get through to the meat of what someone is saying. "I don't know if what I'm saying is right but I think that it probably is. With that caveat, here's a thought that may or may not reflect reality but is my current model to predict certain things. $THOUGHT. Given that I have said that, I have to confess to not living my life entirely by that maxim. I may not have captured all angles to this."
Thankfully, the rationalists just state their ideas and you're free to use their models properly. It's like people haven't written code at all. Just putting repeated logging all through the codebase with null checks everywhere. Just say the thing. That suffices. Conciseness rules over caveating.
Human LLMs who use idea expansion. Insufferable.
Of course that is only my opinion and I may not have captured all angles to why people are doing that. They may have reasons of their own to do that and I don't mean to say that there can never be any reasons. No animals were harmed in the manufacture of this comment to my knowledge. However, I did eat meat this afternoon which could or could not be the source of the energy required to type this comment and the reader may or may not have calorie attribution systems that do or do not allocate this comment to animal harm.
Thankfully, the rationalists just state their ideas and you're free to use their models properly. It's like people haven't written code at all. Just putting repeated logging all through the codebase with null checks everywhere. Just say the thing. That suffices. Conciseness rules over caveating.
Human LLMs who use idea expansion. Insufferable.
Of course that is only my opinion and I may not have captured all angles to why people are doing that. They may have reasons of their own to do that and I don't mean to say that there can never be any reasons. No animals were harmed in the manufacture of this comment to my knowledge. However, I did eat meat this afternoon which could or could not be the source of the energy required to type this comment and the reader may or may not have calorie attribution systems that do or do not allocate this comment to animal harm.
Logic is an awesome tool that took us from Greek philosophers to the gates on our computers. The challenge with pure rationalism is checking the first principles that the thinking comes from. Logic can lead you astray if the principles are wrong, or you miss the complexity along the way.
On the missing first principles, look at Aristotle. One of the history's greatest logicians came to many false conclusions.
On missing complexity, note that Natural Selection came from empirical analysis rather than first principles thinking. (It could have come from the latter, but was too complex) [1]
This doesn't discount logic, it just highlights that answers should always come with provisional humility.
And I'm still a superfan of Scott Aaronson.
[0] https://www.wired.com/story/aristotle-was-wrong-very-wrong-b...
[1] https://www.jstor.org/stable/2400494
On the missing first principles, look at Aristotle. One of the history's greatest logicians came to many false conclusions.
On missing complexity, note that Natural Selection came from empirical analysis rather than first principles thinking. (It could have come from the latter, but was too complex) [1]
This doesn't discount logic, it just highlights that answers should always come with provisional humility.
And I'm still a superfan of Scott Aaronson.
[0] https://www.wired.com/story/aristotle-was-wrong-very-wrong-b...
[1] https://www.jstor.org/stable/2400494
The ‘rationalist’ group being discussed here aren't Cartesian rationalists, who dismissed empiricism; rather, they're Bayesian empiricists. Bayesian probability turns out to be precisely the unique extension of Boolean logic to continuous real probability that Aristotle (nominally an empiricist!) was lacking. (I think they call themselves “rationalists” because of the ideal of a “rational Bayesian agent” in economics.)
However, they have a slogan, “One does not simply reason over the joint conditional probability distribution of the universe.” Which is to say, AIXI is uncomputable, and even AIXI can only reason over computable probability distributions!
However, they have a slogan, “One does not simply reason over the joint conditional probability distribution of the universe.” Which is to say, AIXI is uncomputable, and even AIXI can only reason over computable probability distributions!
They can call themselves empiricists all they like, it only takes a few exposures to their number to come away with a firm conviction (or, let's say, updated prior?) that they are not.
First-principles reasoning and the selection of convenient priors are consistently preferenced over the slow, grinding work of iterative empiricism and the humility to commit to observation before making overly broad theoretical claims.
The former let you seem right about something right now. The latter more often than not lead you to discover you are wrong (in interesting ways) much later on.
First-principles reasoning and the selection of convenient priors are consistently preferenced over the slow, grinding work of iterative empiricism and the humility to commit to observation before making overly broad theoretical claims.
The former let you seem right about something right now. The latter more often than not lead you to discover you are wrong (in interesting ways) much later on.
Who are all the rationalists you guys are reading?
I read the NYT and rat blogs all the time. And the NYT is not the one that's far more likely to deeply engage with the research and studies on the topic.
I read the NYT and rat blogs all the time. And the NYT is not the one that's far more likely to deeply engage with the research and studies on the topic.
Bayesian inference is very, very often used in the types of philosophical/speculative discussions that Rationalists like instead of actual empirical study. It's a very convinient framework for speculating wildly while still maintaining a level of in-principle rationality, since, of course, you [claim that] you will update your priors if someone happens to actually study the phenomenon in question.
The reality is that reasoning breaks down almost immediately if probabilities are not almost perfectly known (to the level that we know them in, say, quantum mechanics, or poker). So applying Bayesian reasoning to something like the number of intelligent species in the galaxy ("Drake's equation"), or the relative intelligence of AI ("the Singularity") or any such subject allows you to draw any conclusion you actually wanted to draw all along, and then find premises you like to reach there.
The reality is that reasoning breaks down almost immediately if probabilities are not almost perfectly known (to the level that we know them in, say, quantum mechanics, or poker). So applying Bayesian reasoning to something like the number of intelligent species in the galaxy ("Drake's equation"), or the relative intelligence of AI ("the Singularity") or any such subject allows you to draw any conclusion you actually wanted to draw all along, and then find premises you like to reach there.
Beautifully put.
Logic is the study of what is true, and also what is provable.
In the most ideal circumstances, these are the same. Logic has been decomposed into model theory (the study of what is true) and proof theory (the study of what is provable). So much of modern day rationalism is unmoored proof theory. Many of them would do well to read Kant's "The Critique of Pure Reason."
Unfortunately, in the very complex systems we often deal with, what is true may not be provable and many things which are provable may not be true. This is why it's equally as important to hone your skills of discernment, and practice reckoning as well as reasoning. I think of it as hearing "a ring of truth," but this is obviously unfalsifiable and I must remain skeptical against myself when I believe I hear this. It should be a guide toward deeper investigation, not the final destination.
Many people are led astray by thinking. It is seductive. It should be more commonly said that thinking is but a conscious stumbling block on the way to unconscious perfection.
In the most ideal circumstances, these are the same. Logic has been decomposed into model theory (the study of what is true) and proof theory (the study of what is provable). So much of modern day rationalism is unmoored proof theory. Many of them would do well to read Kant's "The Critique of Pure Reason."
Unfortunately, in the very complex systems we often deal with, what is true may not be provable and many things which are provable may not be true. This is why it's equally as important to hone your skills of discernment, and practice reckoning as well as reasoning. I think of it as hearing "a ring of truth," but this is obviously unfalsifiable and I must remain skeptical against myself when I believe I hear this. It should be a guide toward deeper investigation, not the final destination.
Many people are led astray by thinking. It is seductive. It should be more commonly said that thinking is but a conscious stumbling block on the way to unconscious perfection.
I'm just going to defend Aristotle a bit. His incomplete logic and metaphysics nevertheless provided a powerful foundation to inquire into many aspects of the world that his predecessors did not do, nor do systematically. His community did not shy away from empirical research in biology. They all came to wrong conclusions in some things, but we should rather fault their successors for not challenging them.
Yup, can't stress the word "tool" enough.
It's a "tool," it's a not a "magic window into absolute truth."
Tools can be good for a job, or bad. Carry on.
It's a "tool," it's a not a "magic window into absolute truth."
Tools can be good for a job, or bad. Carry on.
looks like I riled up the Rationalists, huh
You stated one of their core doctrines, something they’ve been loudly preaching for as long as they’ve existed, as though it was something they disagree with and had never even considered. Can you blame them for a bit of exasperation? For wanting to simply downvote-and-disengage from someone who makes up falsehoods about them and then gloats about how it annoyed them? Life is too short to tilt at windmills.
>provisional humility.
I hope this becomes the first ever meme with some value. We need a cult... of Provisional Humility.
Must. Increase. The. pH
I hope this becomes the first ever meme with some value. We need a cult... of Provisional Humility.
Must. Increase. The. pH
I'm currently reading Yudkowsky's "Rationality: from AI to zombies". Not my first try, since the book is just a collection of blog posts and I found it a bit hard to swallow due its repetitiveness, so I gave up after the first 50 "chapters" the first time I tried. Now I'm enjoying it way more, probably because I'm more interested in the topic now.
For those who haven't delved(ha!) into his work or have been pushed back by the cultish looks, I have to say that he's genuinelly onto something. There are a lot of practical ideas that are pretty useful for everyday thinking ("Belief in Belief", "Emergence", "Generalizing from fiction", etc...).
For example, I recall being in lot of arguments that are purely "semantical" in nature. You seem to disagree about something but it's just that both sides aren't really referring to the same phenomenon. The source of the disagreement is just using the same word for different, but related, "objects". This is something that seems obvious, but the kind of thing you only realize in retrospect, and I think I'm much better equipped now to be aware of it in real time.
I recommend giving it a try.
For those who haven't delved(ha!) into his work or have been pushed back by the cultish looks, I have to say that he's genuinelly onto something. There are a lot of practical ideas that are pretty useful for everyday thinking ("Belief in Belief", "Emergence", "Generalizing from fiction", etc...).
For example, I recall being in lot of arguments that are purely "semantical" in nature. You seem to disagree about something but it's just that both sides aren't really referring to the same phenomenon. The source of the disagreement is just using the same word for different, but related, "objects". This is something that seems obvious, but the kind of thing you only realize in retrospect, and I think I'm much better equipped now to be aware of it in real time.
I recommend giving it a try.
Yeah, the whole community side to rationality is, at best, questionable.
But the tools of thought that the literature describes are invaluable with one very important caveat.
The moment you think something like "I am more correct than this other person because I am a rationalist" is the moment you fail as a rationalist.
It is an incredibly easy mistake to make. To make effective use of the tools, you need to become more humble than before you were using them or you just turn into an asshole who can't be reasoned with.
If you're saying "well actually, I'm right" more often than "oh wow, maybe I'm wrong", you've failed as a rationalist.
But the tools of thought that the literature describes are invaluable with one very important caveat.
The moment you think something like "I am more correct than this other person because I am a rationalist" is the moment you fail as a rationalist.
It is an incredibly easy mistake to make. To make effective use of the tools, you need to become more humble than before you were using them or you just turn into an asshole who can't be reasoned with.
If you're saying "well actually, I'm right" more often than "oh wow, maybe I'm wrong", you've failed as a rationalist.
> The moment you think something like "I am more correct than this other person because I am a rationalist" is the moment you fail as a rationalist.
Well said. Rationalism is about doing rationalism, not about being a rationalist.
Paul Graham was on the right track about that, though seemingly for different reasons (referring to "Keep Your Identity Small").
> If you're saying "well actually, I'm right" more often than "oh wow, maybe I'm wrong", you've failed as a rationalist.
On the other hand, success is supposed to look exactly like actually being right more often.
Well said. Rationalism is about doing rationalism, not about being a rationalist.
Paul Graham was on the right track about that, though seemingly for different reasons (referring to "Keep Your Identity Small").
> If you're saying "well actually, I'm right" more often than "oh wow, maybe I'm wrong", you've failed as a rationalist.
On the other hand, success is supposed to look exactly like actually being right more often.
> success is supposed to look exactly like actually being right more often.
I agree with this, and I don't think it's at odds with what I said. The point is to never stop sincerely believing you could be wrong. That you are right more often is exactly why it's such an easy trap to fall into. The tools of rationality only help as long as you are actively applying them, which requires a certain amount of humility, even in the face of success.
I agree with this, and I don't think it's at odds with what I said. The point is to never stop sincerely believing you could be wrong. That you are right more often is exactly why it's such an easy trap to fall into. The tools of rationality only help as long as you are actively applying them, which requires a certain amount of humility, even in the face of success.
Chapter 67. https://www.readthesequences.com/Knowing-About-Biases-Can-Hu... (And since it's in the book, and people know about it, obviously they're not doing it themselves.)
Also the Valley of Bad Rationality tag. https://www.lesswrong.com/w/valley-of-bad-rationality
Also that the Art needs to be about something else than itself, and a dozen different things. This failure mode is well known in the community; Eliezer wrote about it to death, and so did others.
To no avail, alas. But this is why we now see a thought leader publish a piece to say this is a thing it's now permissible not to be, indeed never to have been at all.
This reminds me of undergrad philosophy courses. After the intro logic/critical thinking course, some students can't resist seeing affirming the antecedent and post hoc fallacies everywhere (even if more are imagined than not).
> The moment you think something like "I am more correct than this other person because I am a rationalist" is the moment you fail as a rationalist.
It's very telling that some of them went full "false modesty" by naming sites like "LessWrong", when you just know they actually mean "MoreRight".
And in reality, it's just a bunch of "grown teenagers" posting their pet theories online and thinking themselves "big thinkers".
It's very telling that some of them went full "false modesty" by naming sites like "LessWrong", when you just know they actually mean "MoreRight".
And in reality, it's just a bunch of "grown teenagers" posting their pet theories online and thinking themselves "big thinkers".
> you just know they actually mean "MoreRight".
I'm not affiliated with the rationalist community, but I always interpreted "Less Wrong" as word-play on how "being right" is an absolute binary: you can either be right, or not be right, while "being wrong" can cover a very large gradient.
I expect the community wanted to emphasize how people employing the specific kind of Bayesian iterative reasoning they were proselytizing would arrive at slightly lesser degrees of wrong than the other kinds that "normal" people would use.
If I'm right, your assertion wouldn't be totally inaccurate, but I think it might be missing the actual point.
I'm not affiliated with the rationalist community, but I always interpreted "Less Wrong" as word-play on how "being right" is an absolute binary: you can either be right, or not be right, while "being wrong" can cover a very large gradient.
I expect the community wanted to emphasize how people employing the specific kind of Bayesian iterative reasoning they were proselytizing would arrive at slightly lesser degrees of wrong than the other kinds that "normal" people would use.
If I'm right, your assertion wouldn't be totally inaccurate, but I think it might be missing the actual point.
> I always interpreted "Less Wrong" as word-play on how "being right" is an absolute binary
Specifically (AFAIK) a reference to Asimov’s description[1] of the idea:
> [W]hen people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.
[1] https://skepticalinquirer.org/1989/10/the-relativity-of-wron...
Specifically (AFAIK) a reference to Asimov’s description[1] of the idea:
> [W]hen people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.
[1] https://skepticalinquirer.org/1989/10/the-relativity-of-wron...
Cool, I didn't know the quote, nor that it was inspiration for the name. Thank you.
It's not even about the quote, or Asimov.
"Less wrong" is a concept that has a lot of connotations that just automatically appear in your mind and help you. What you wrote "It's very telling that some of them went full "false modesty" by naming sites like "LessWrong", when you just know they actually mean "MoreRight"." isn't bad because of Asimov said so, or because you were unaware of a reference, but because it's just bad.
"Less wrong" is a concept that has a lot of connotations that just automatically appear in your mind and help you. What you wrote "It's very telling that some of them went full "false modesty" by naming sites like "LessWrong", when you just know they actually mean "MoreRight"." isn't bad because of Asimov said so, or because you were unaware of a reference, but because it's just bad.
The person you are replying to didn't write what you claim they did. I wrote it.
What is it about this thread that makes people confused about who wrote what? It's already happened to two different commenters.
What is it about this thread that makes people confused about who wrote what? It's already happened to two different commenters.
[deleted]
> I'm not affiliated with the rationalist community, but I always interpreted "Less Wrong" as word-play on how "being right" is an absolute binary: you can either be right, nor not be right, while "being wrong" can cover a very large gradient.
I know that's what they mean at the surface level, but you just know it comes with a high degree of smugness and false modesty. "I only know that I know nothing" -- maybe, but they ain't no modern day Socrates, they are just a bunch of nerds going online with their thoughts.
I know that's what they mean at the surface level, but you just know it comes with a high degree of smugness and false modesty. "I only know that I know nothing" -- maybe, but they ain't no modern day Socrates, they are just a bunch of nerds going online with their thoughts.
Sometimes people enjoy being clever not because they want to rub it in your face that you're not, but because it's fun. I usually try not to take it personally when I don't get the joke and strive to do better next time.
That's mildly insulting of you.
I do get the joke; I think it's an instance of their feelings of "rational" superiority.
Assuming the other person didn't get the joke is very... irrational of you.
I do get the joke; I think it's an instance of their feelings of "rational" superiority.
Assuming the other person didn't get the joke is very... irrational of you.
Like I said, I'm not trying to be a rationalist, or at least not this flavour of it. That being said, I apologise for the dig.
>but you just know it comes with a high degree of smugness and false modesty
No; I know no such thing, as I have no good reason to believe it, and plenty of countering evidence.
No; I know no such thing, as I have no good reason to believe it, and plenty of countering evidence.
Very rational of you, but that's the problem with the whole system.
If you want to avoid thinking you're right all the time, it doesn't help to be clever and say the logical opposite. "Rationally" it should work, but it's bad because you're still thinking about it! It's like the thinking of a pink elephant thing.
Other approaches I recommend:
* try and fail to invest in stocks
* read Meaningness's https://metarationality.com
* print out this meme and put it on your wall https://imgflip.com/i/82h43h
If you want to avoid thinking you're right all the time, it doesn't help to be clever and say the logical opposite. "Rationally" it should work, but it's bad because you're still thinking about it! It's like the thinking of a pink elephant thing.
Other approaches I recommend:
* try and fail to invest in stocks
* read Meaningness's https://metarationality.com
* print out this meme and put it on your wall https://imgflip.com/i/82h43h
>If you want to avoid thinking you're right all the time, it doesn't help to be clever and say the logical opposite.
I don't understand how this is supposed to be relevant here. You seem to be falsely accusing me of doing such a thing, or of being motivated by simple contrarianism.
Again, your claim was:
> but you just know it comes with a high degree of smugness and false modesty
Why should I "just know" any such thing? What is your reason for "just knowing" it? It comes across that you have simply decided to assume the worst of people that you don't understand.
I don't understand how this is supposed to be relevant here. You seem to be falsely accusing me of doing such a thing, or of being motivated by simple contrarianism.
Again, your claim was:
> but you just know it comes with a high degree of smugness and false modesty
Why should I "just know" any such thing? What is your reason for "just knowing" it? It comes across that you have simply decided to assume the worst of people that you don't understand.
It was my claim, not the person you're responding to now.
As to why I "just know": it's because I'm not a robot, I have experience at reading this kind of claims, and they usually mean what I think they mean.
"You just know" is an idiomatic expression, it's not meant to be dissected.
As to why I "just know": it's because I'm not a robot, I have experience at reading this kind of claims, and they usually mean what I think they mean.
"You just know" is an idiomatic expression, it's not meant to be dissected.
So much projection.
I don't think I'm more clever than the average person, nor have I made this my identity or created a whole tribe around it, nor do I attend nor host conferences around my cleverness, rationality, or weird sexual fetishes.
In other words: no.
In other words: no.
Rationalism is not about trying to be clever it's very much about trying to be a little less wrong. Most people are not even trying, which includes myself. I don't write down my predictions, I don't keep a list of my errors. I just show up to work like everyone else and don't worry about it.
I really don't understand all the claims that they intellectually smug and overconfident when they are the one group of people trying to do better. It really seems like all the hatred is aimed at the hubris to even try to do better.
I really don't understand all the claims that they intellectually smug and overconfident when they are the one group of people trying to do better. It really seems like all the hatred is aimed at the hubris to even try to do better.
I think there is an arbitrage going on where STEM types who lack background in philosophy, literature, history are super impressed by basic ideas from those subjects being presented to them by stealth.
Not saying this is you, but these topics have been discussed for thousands of years, so it should at least be surprising that Yudkowsky is breaking new ground.
Not saying this is you, but these topics have been discussed for thousands of years, so it should at least be surprising that Yudkowsky is breaking new ground.
Are there other philosophy- or history-grounded sources that are comparable? If so, I’d love some recommendations. Yudkowsky and others have their problems, but their texts have an interesting points, are relatively easy to read and understand, and you can clearly see which real issues they’re addressing. From my experience, alternatives tend to fall into two categories: 1. Genuine classical philosophy, which is usually incredibly hard to read and after 50 pages I have no idea what the author is even talking about anymore.
2. Basically self help books that take one or very few idea and repeat them ad nouseam for 200 pages.
Likely the best resource to learn about philosophy is the Stanford Encyclopedia of Philosophy [0]. It's meant to provide a rigorous starting point for learning about a topic, where 1. you won't get bogged down in a giant tome on your first approach and 2. you have references for further reader.
Obviously, the SEP isn't perfect, but it's a great place to start. There's also the Internet Encyclopedia of Philosophy [1]; however, I find its articles to be more hit or miss.
[0] https://plato.stanford.edu
[1] https://iep.utm.edu
Obviously, the SEP isn't perfect, but it's a great place to start. There's also the Internet Encyclopedia of Philosophy [1]; however, I find its articles to be more hit or miss.
[0] https://plato.stanford.edu
[1] https://iep.utm.edu
I've read Bertrand Russell's "A History of Western Philosophy" and it's the first ever philosophy book that I didn't drop after 10 pages, because of 2 things:
1- He's logic (or at least has the same STEM kind of logic that we use), so he builds his reasoning logically and not via bullshit associations like plays on words or contrived jumps.
2- He's not afraid to tell "this philosopher said that, it was an error", which is extremely new compared to other scholars who don't feel authorized to criticise even obvious errors.
Really recommend!
I don't know if there's anything like a comprehensive high-level guide to philosophy that's any good, though of course there are college textbooks. If you want real/academic philosophy that's just more readable, I might suggest Eugene Thacker's "The Horror of Philosophy" series (starting with "In The Dust Of This Planet"), especially if you are a horror fan already.
It's not a nice response but I would say: don't be so lazy. Struggle through the hard stuff.
I say this as someone who had the opposite experience: I had a decent humanities education, but an abysmal mathematics education, and now I am tackling abstract mathematics myself. It's hard. I need to read sections of works multiple times. I need to sit down and try to work out the material for myself on paper.
Any impression that one discipline is easier than another probably just stems from the fact that you had good guides for the one and had the luck to learn it when your brain was really plastic. You can learn the other stuff too, just go in with the understanding that there's no royal road to philosophy just as there's no royal road to mathematics.
I say this as someone who had the opposite experience: I had a decent humanities education, but an abysmal mathematics education, and now I am tackling abstract mathematics myself. It's hard. I need to read sections of works multiple times. I need to sit down and try to work out the material for myself on paper.
Any impression that one discipline is easier than another probably just stems from the fact that you had good guides for the one and had the luck to learn it when your brain was really plastic. You can learn the other stuff too, just go in with the understanding that there's no royal road to philosophy just as there's no royal road to mathematics.
People are likely willing to struggle through hard stuff if the applications are obvious.
But if you can't even narrow the breadth of possible choices down to a few paths that can be traveled, you can't be surprised when people take the one that they know that's also easier with more immediate payoffs.
But if you can't even narrow the breadth of possible choices down to a few paths that can be traveled, you can't be surprised when people take the one that they know that's also easier with more immediate payoffs.
When you've read that passage in the math book twenty times, you eventually come to the conclusion that you understood it (even if in some rare cases you still didn't).
When "struggling through" a philosophy book, that doesn't happen in my experience. In fact, if you look up what others thought that passage means, you'll find no agreement among a bunch of people who "specialize" in authors who themselves "specialized" in the author you're reading. So reading that stuff I feel I have to accept that I will never understand what's written there and the whole exercise is just about "thinking about it for the sake of thinking". This might be "good for me" but it's really hard to keep up the motivation. Much harder than a math book.
When "struggling through" a philosophy book, that doesn't happen in my experience. In fact, if you look up what others thought that passage means, you'll find no agreement among a bunch of people who "specialize" in authors who themselves "specialized" in the author you're reading. So reading that stuff I feel I have to accept that I will never understand what's written there and the whole exercise is just about "thinking about it for the sake of thinking". This might be "good for me" but it's really hard to keep up the motivation. Much harder than a math book.
I agree with the phenomenon you are talking about, but for mathematics, beyond calculation, the situation isn't really different (and no wonder since you'll quickly end up in the philosophy of mathematics).
You can take an entire mathematical theory on faith and learn to perform rote calculations in accordance with the structure of that theory. This might be of some comfort, since, accepting this, you can perform a procedure and see whether or not you got the correct result (but even this is a generous assumption in some sense). When you actually try to understand a theory and go beyond that to foundations, things become less certain. At some point you will accept things, but, unless you have enough time to work out every proof and then prove to yourself that the very idea of a proof calculus is sound, you will be taking something on faith.
I think if people struggle with doing the same thing with literature/philosophy, it's probably just because of a discomfort with ambiguity. In those realms, there is no operational calculus you can turn to to verify that, at least if you accept certain things on faith, other things must work out...expect there is! Logic lords over both domains. I think we just do a horrible job at teaching people how to approach literature logically. Yes, the subtle art of interpretation is always at play, but that's true of mathematics too and it is true of every representational/semiotic effort undertaken by human beings.
As for use, social wit and the ability to see things in new lights (devise new explanatory hypotheses) are both immediate applications of philosophy and literature, just like mathematics has its immediate operational applications in physics et al.
You can take an entire mathematical theory on faith and learn to perform rote calculations in accordance with the structure of that theory. This might be of some comfort, since, accepting this, you can perform a procedure and see whether or not you got the correct result (but even this is a generous assumption in some sense). When you actually try to understand a theory and go beyond that to foundations, things become less certain. At some point you will accept things, but, unless you have enough time to work out every proof and then prove to yourself that the very idea of a proof calculus is sound, you will be taking something on faith.
I think if people struggle with doing the same thing with literature/philosophy, it's probably just because of a discomfort with ambiguity. In those realms, there is no operational calculus you can turn to to verify that, at least if you accept certain things on faith, other things must work out...expect there is! Logic lords over both domains. I think we just do a horrible job at teaching people how to approach literature logically. Yes, the subtle art of interpretation is always at play, but that's true of mathematics too and it is true of every representational/semiotic effort undertaken by human beings.
As for use, social wit and the ability to see things in new lights (devise new explanatory hypotheses) are both immediate applications of philosophy and literature, just like mathematics has its immediate operational applications in physics et al.
At the risk of being roasted for recommending pop-culture things, the podcast Philosophize This is pretty good for a high-level overview. I'm sure there are issues and simplifications, and it's certainly not actual source material. The nice part is it's sort of a start-to-finish, he goes from the start of philosophy to modern day stuff, which helps a ton in building foundational understanding without reading everything ever written.
I don't have an answer here either, but after suffering through the first few chapters of HPMOR, I've found that Yudk and others tech-bros posing as philosophers are basically like leaky, dumbed-down abstractions for core philosophical ideas. Just go to the source and read about utilitarianism and deontology directly. Yudk is like the Wix of web development - sure you can build websites but you're not gonna be a proper web developer unless you learn HTML, CSS and Javascript. Worst of all, crappy abstractions train you in some actively bad patterns that are hard to unlearn
It's almost offensive - are technologists so incapable of understanding philosophy that Yudk has to reduce it down to the least common denominator they are all familiar with - some fantasy world we read about as children?
It's almost offensive - are technologists so incapable of understanding philosophy that Yudk has to reduce it down to the least common denominator they are all familiar with - some fantasy world we read about as children?
I'd like what the original sources would have written if someone had fed them some speak-clearly pills. Yudkowsky and company may have the dumbing-down problem, but the original sources often have a clarity problem. (That's why people are still arguing about what they meant centuries later. Not just whether they were right - though they argue about that too - but what they meant.)
Even better, I'd like some filtering out of the parts that are clearly wrong.
Even better, I'd like some filtering out of the parts that are clearly wrong.
Have you considered that the ideas that philosophy discusses are very complex, and words are not a sufficient medium to describe those ideas?
That's why you have people arguing over what someone meant. Dumbing it down or trying to write something unambiguous doesn't actually make it better.
That's why you have people arguing over what someone meant. Dumbing it down or trying to write something unambiguous doesn't actually make it better.
I will grant you that the ideas are complex. But if words are not a sufficient medium, then we can't think clearly about the ideas.
Original philosophers have the right to define their own terms. If they can't define them clearly, then they probably aren't thinking clearly enough about their ideas to be able to talk to the rest of us about them. (Unless they consider it more important to sound impressive and hard to understand. But if that's the case, we can say "wow, you sound impressive" and then ignore them.)
Original philosophers have the right to define their own terms. If they can't define them clearly, then they probably aren't thinking clearly enough about their ideas to be able to talk to the rest of us about them. (Unless they consider it more important to sound impressive and hard to understand. But if that's the case, we can say "wow, you sound impressive" and then ignore them.)
I think this accurately channels Paul Graham's attempt to divide the world into "science/tech" and "illegible". But it's a little ridiculous to also divide the world into, "things I understand" and "things I don't", and state that anyone who speaks of the latter should not be permitted to talk to "the rest of us" until they figure out how to move themselves into your "things I understand" category.
The top scientists in AI can't explain how their models make certain decisions (at least not deterministically). Computer code is notoriously gibberish to outsiders. 90% of programmers probably couldn't explain what their job is to people outside of the field. If they can't explain it clearly, should they also be forbidden from speaking publicly until they can?
Is it possible that you lack the background to understand philosophy, and thus philosophers should rightly ignore your demands to dumb down their own field? Why should philosophers even appeal to people like you, when you seem so uninterested in even learning the basics of their field?
The top scientists in AI can't explain how their models make certain decisions (at least not deterministically). Computer code is notoriously gibberish to outsiders. 90% of programmers probably couldn't explain what their job is to people outside of the field. If they can't explain it clearly, should they also be forbidden from speaking publicly until they can?
Is it possible that you lack the background to understand philosophy, and thus philosophers should rightly ignore your demands to dumb down their own field? Why should philosophers even appeal to people like you, when you seem so uninterested in even learning the basics of their field?
No, I don't regard all of philosophy (or all of non-science) as "illegible".
Nor do I regard the line as being "things I understand". I'm not (usually) that arrogant. But if, say, even other computer programmers can't tell for sure what you're saying, the problem is probably you.
Nor do I regard the line as being "things I understand". I'm not (usually) that arrogant. But if, say, even other computer programmers can't tell for sure what you're saying, the problem is probably you.
HPMOR is not supposed to be rigorous. It’s supposed to be entertaining in a way that rigorous philosophy is not. You could make the same argument about any of Camus’ novels but again that would miss the point. If you want something more rigorous yudkowsky has it, bit surprising to me to complain he isn’t rigorous without talking about his rigorous work.
I really have no interest in reading any more of that guy’s “work”
Totally fair but why comment on something you arent interested in understanding
In AI finetuning, there's a theory that the model already contains the right ideas and skills, and the finetuning just raises them to prominence. Similarly in philosophic pedagogy, there's huge value in taking ideas that are correct but unintuitive and maybe have 30% buy-in and saying "actually, this is obviously correct, also here's an analysis of why you wouldn't believe it anyway and how you have to think to become able to believe it". That's most of what the Sequences are: they take from every field of philosophy the ideas that are actually correct, and say "okay actually, we don't need to debate this anymore, this just seems to be the truth because so-and-so." (Though the comments section vociferously disagrees.)
And it turns out if you do this, you can discard 90% of philosophy as historical detritus. You're still taking ideas from philosophy, but which ideas matters, and how you present them matters. The massive advantage of the Sequences is they have justified and well-defended confidence where appropriate. And if you manage to pick the right answers again and again, you get a system that actually hangs together, and IMO it's to philosophy's detriment that it doesn't do this itself much more aggressively.
For instance, 60% of philosophers are compatibilists. Compatibilism is really obviously correct. "What are you complaining about, that's a majority, isn't that good?" What is wrong with those 40% though? If you're in those 40%, what arguments may convince you? Repeat to taste.
And it turns out if you do this, you can discard 90% of philosophy as historical detritus. You're still taking ideas from philosophy, but which ideas matters, and how you present them matters. The massive advantage of the Sequences is they have justified and well-defended confidence where appropriate. And if you manage to pick the right answers again and again, you get a system that actually hangs together, and IMO it's to philosophy's detriment that it doesn't do this itself much more aggressively.
For instance, 60% of philosophers are compatibilists. Compatibilism is really obviously correct. "What are you complaining about, that's a majority, isn't that good?" What is wrong with those 40% though? If you're in those 40%, what arguments may convince you? Repeat to taste.
Additional note: compatibilism is only obviously correct if you accept that "free will" actually just means "the experienced perception/illusion of free will" as described by Schopenhauer.
Using a slightly different definition of free will, suddenly Compatibilism becomes obviously incorrect.
And now it's been reduced to quibbling over definitions, thereby reinventing much of the history of philosophy.
Using a slightly different definition of free will, suddenly Compatibilism becomes obviously incorrect.
And now it's been reduced to quibbling over definitions, thereby reinventing much of the history of philosophy.
I think free will as philosophically used is inherently self-defeating and one of the largest black marks on the entire field, to be fair.
Why is that?
Here's what we know:
- we appear to experience what we call free will from our own perspective. This isn't strong evidence obviously. - we are aware that we live in a world full of predictable mechanisms of varying levels of complexity, as well as fundamentally unpredictable mechanisms like quantum mechanics. - we know we are currently unable to fully model our experience and predict next steps. - we know that we don't know whether consciousness as an emergent property of our brains is fully rooted in predictable mechanisms or has some decree of unknowability to it.
So really "do we have free will" is a question that relies on the nature of consciousness.
Here's what we know:
- we appear to experience what we call free will from our own perspective. This isn't strong evidence obviously. - we are aware that we live in a world full of predictable mechanisms of varying levels of complexity, as well as fundamentally unpredictable mechanisms like quantum mechanics. - we know we are currently unable to fully model our experience and predict next steps. - we know that we don't know whether consciousness as an emergent property of our brains is fully rooted in predictable mechanisms or has some decree of unknowability to it.
So really "do we have free will" is a question that relies on the nature of consciousness.
No, I disagree with this conclusion. The problem is very much solvable if one simply keeps the map/territory split in mind, and for every thing asks himself, "am I perceiving reality or am I perceiving a property of my brain?" That is, we "experience free will" - this is to say, our brain reports to us that it evaluated multiple possible behaviors and chose one. However, this does not indicate that multiple behaviors were physically possible, it only indicates that multiple behaviors were cognitively evaluated. In fact, because any deciding algorithm has to evaluate a behavior list or even a behavior tree, there is no reason at all to expect this to have any connection to physical properties of the world, such as quantum mechanics.
(The relevant LessWrong sequence is "How An Algorithm Feels From Inside" https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-alg... which has nothing to do with free will, but does make very salient the idea that perceptions may be facts about your cognition as easily, if not more easily, as facts about reality.)
And when you have that view in mind, you can ask: "wait, why would the brain be sensitive to quantum physics? It seems to be a system extremely poorly suited to doing macroscopic quantum calculations." Once the alternative theory of "free will is a perception of your cognitive algorithm" is salient, you will notice that the entire free-will debate will begin to feel more and more pointless, until eventually you no longer understand why people think this is a big deal at all, and then it all feels rather silly.
(The relevant LessWrong sequence is "How An Algorithm Feels From Inside" https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-alg... which has nothing to do with free will, but does make very salient the idea that perceptions may be facts about your cognition as easily, if not more easily, as facts about reality.)
And when you have that view in mind, you can ask: "wait, why would the brain be sensitive to quantum physics? It seems to be a system extremely poorly suited to doing macroscopic quantum calculations." Once the alternative theory of "free will is a perception of your cognitive algorithm" is salient, you will notice that the entire free-will debate will begin to feel more and more pointless, until eventually you no longer understand why people think this is a big deal at all, and then it all feels rather silly.
> However, this does not indicate that multiple behaviors were physically possible
Okay, fine, but what indicates that multiple behaviors were not physically possible?
Our consciousnesses are emergent properties of networks of microscopic cells, and of chemicals moving around those cells at a molecular level. It seems perfectly reasonable that our consciousness itself could be subject to quantum effects that belie determinism, because it operates at a scale where those effects are noticable.
Okay, fine, but what indicates that multiple behaviors were not physically possible?
Our consciousnesses are emergent properties of networks of microscopic cells, and of chemicals moving around those cells at a molecular level. It seems perfectly reasonable that our consciousness itself could be subject to quantum effects that belie determinism, because it operates at a scale where those effects are noticable.
> Okay, fine, but what indicates that multiple behaviors were not physically possible?
I don't follow. Whether multiple behaviors are possible or not possible, you have to demonstrate that the human feeling of free-will is about that; you have to demonstrate that the human brain somehow measures actual possibility. Alternatively, you have to show that the human cognitive decision algorithm is unimplementable in either of those universes. Otherwise, it's simply much more plausible that the human feeling of freedom measures something about human cognition rather than reality, because brains in general usually measure things around the scale of brains, not non-counterfactual facts about the basic physical laws.
I don't follow. Whether multiple behaviors are possible or not possible, you have to demonstrate that the human feeling of free-will is about that; you have to demonstrate that the human brain somehow measures actual possibility. Alternatively, you have to show that the human cognitive decision algorithm is unimplementable in either of those universes. Otherwise, it's simply much more plausible that the human feeling of freedom measures something about human cognition rather than reality, because brains in general usually measure things around the scale of brains, not non-counterfactual facts about the basic physical laws.
Well, no, your hypothesis is not automatically the null hypothesis that's true unless someone else goes through all goalposts regardless of where you move them to.
I know you thought about it for a moment, and therefore had an obvious insight that 40% of the profession has somehow missed (just define terms so to mean things that would make you correct, and declare yourself right! Easy!) but it's not quite that simple.
Your argument that you just made basically boils down to "well I don't think it works that way even though no one knows. But also it's obvious and I'm going to arbitrarily assign probabilities to things and declare certain things likely, baselessly".
If you read elsewhere in this thread then you might find that exact approach being lampooned :-)
I know you thought about it for a moment, and therefore had an obvious insight that 40% of the profession has somehow missed (just define terms so to mean things that would make you correct, and declare yourself right! Easy!) but it's not quite that simple.
Your argument that you just made basically boils down to "well I don't think it works that way even though no one knows. But also it's obvious and I'm going to arbitrarily assign probabilities to things and declare certain things likely, baselessly".
If you read elsewhere in this thread then you might find that exact approach being lampooned :-)
Okay, you know what?
I'll let my argument stand as written, and you can let yours stand as written, and we'll see which one is more convincing. I don't feel like I have any need to add anything.
edit: Other than, I guess, that this mode of argument not being there is what made LessWrong attractive. "But what's the actual answer?!"
I'll let my argument stand as written, and you can let yours stand as written, and we'll see which one is more convincing. I don't feel like I have any need to add anything.
edit: Other than, I guess, that this mode of argument not being there is what made LessWrong attractive. "But what's the actual answer?!"
The attraction of LessWrong is that they take unanswerable questions with unknowable answers and assign an "actual answer" to them?
That my friend is a religion.
That my friend is a religion.
There isn't a single philosophical meaning. You are probably thinking of libertarian free will..furthering Obviously false because determinism isn't obviously true.
> And it turns out if you do this, you can discard 90% of philosophy as historical detritus
This is just the story of the history of philosophy. Going back hundreds of years. See Kant and Hegel for notable examples.
This is just the story of the history of philosophy. Going back hundreds of years. See Kant and Hegel for notable examples.
Sure, and I agree that LW is doing philosophy in that sense.
Sure, I just object to the characterisation of "actually correct", as though each of those ideas has not gone back and forth on philosophers thinking that particular idea is "actually correct" for centuries. LW does not appear to have much if any novel insight; just much better marketing.
I think philosophy has gone back and forth for so long that they're now, as a field, pathologically afraid of actually committing.
The AI connection with LessWrong means that the whole thing is framed with a backdrop of "how would you actually construct a mind?" That means you can't just chew on the questions, you have to actually commit to an answer and run the risk of being wrong.
This teaches you two things: 1. How to figure what you actually believe the answer is, and why, and make sure that this is the best answer that you can give; 2. how to keep moving when you notice that you made a misstep in part 1.
The AI connection with LessWrong means that the whole thing is framed with a backdrop of "how would you actually construct a mind?" That means you can't just chew on the questions, you have to actually commit to an answer and run the risk of being wrong.
This teaches you two things: 1. How to figure what you actually believe the answer is, and why, and make sure that this is the best answer that you can give; 2. how to keep moving when you notice that you made a misstep in part 1.
>actually, this is obviously correct
Nobody know what's actually correct, because you have to solve epistemology first, and you have to solve epistemology to solve epistemology...etc.etc.
>And it turns out if you do this, you can discard 90% of philosophy as historical detritus
Nope. For instance , many of the issues Kant raised are still live.
>The massive advantage of the Sequences is they have justified and well-defended confidence
Nope. That would entail answering objections , which EY doesn't stoop to.
>Compatibilism is really obviously correct
Nope. It depends on a semantic issue , what free will means.
Nobody know what's actually correct, because you have to solve epistemology first, and you have to solve epistemology to solve epistemology...etc.etc.
>And it turns out if you do this, you can discard 90% of philosophy as historical detritus
Nope. For instance , many of the issues Kant raised are still live.
>The massive advantage of the Sequences is they have justified and well-defended confidence
Nope. That would entail answering objections , which EY doesn't stoop to.
>Compatibilism is really obviously correct
Nope. It depends on a semantic issue , what free will means.
To the Stem-enlightened mind, the classical understanding and pedagogy of such ideas is underwhelming, vague, and riddled with language-game problems, compared to the precision a mathematically-rooted idea has.
They're rederiving all this stuff not out of obstinacy, but because they prefer it. I don't really identify with rationalism per se, but I'm with them on this--the humanities are over-cooked and a humanity education tends to be a tedious slog through outmoded ideas divorced from reality
They're rederiving all this stuff not out of obstinacy, but because they prefer it. I don't really identify with rationalism per se, but I'm with them on this--the humanities are over-cooked and a humanity education tends to be a tedious slog through outmoded ideas divorced from reality
If you contextualise the outmoded ideas as part of the Great Conversation [1], and the story of how we reached our current understanding, rather than objective statements of fact, then they becomes a lot more valuable and worthy of study.
[1] https://en.wikipedia.org/wiki/Great_Conversation
[1] https://en.wikipedia.org/wiki/Great_Conversation
But isn't the content of LessWrong part of the Great Conversation too?
I have kids in high school. We sometimes talk about the difference between the black and white of math or science, and the wishy washy grey of the humanities.
You can be right or wrong in math. You have can an opinion in English.
You can be right or wrong in math. You have can an opinion in English.
You can be right or wrong in math and philosophy. You have can an opinion in any other sciences, physics, chemistry, biology, medical sciences, history, you name it.
I suggest teaching them about the Problem of Induction [1].
Scientific thinking is not the same as mathematical thinking and it becomes quite wishy washy grey if you zoom in too far!
[1] https://plato.stanford.edu/entries/induction-problem/
Scientific thinking is not the same as mathematical thinking and it becomes quite wishy washy grey if you zoom in too far!
[1] https://plato.stanford.edu/entries/induction-problem/
You can definitely have wrong opinions in the humanities.
Rationalism largely rejects continental philosophy in favor of a more analytic approach. Yes these ideas are not new, but they’re not really the mainstream stuff you’d see in philosophy, literature, or history studies. You’d have to seek out these classes specifically to find them.
Analytical philosophy is rationalism done right.
They largely reject analytic philosophy as well. Austin and Whitehead are roughly as detestable to a Rationalist as Foucault and Marx.
Carlyle, Chesterton and Thoreau are about the limit of their philosophical knowledge base.
Carlyle, Chesterton and Thoreau are about the limit of their philosophical knowledge base.
I don't claim that his work is original (the AI related probably is, but it's just tangentially related to rationalism), but it's clearly presented and is practical.
And, BTW, I could just be ignorant in a lot of these topics, I take no offense in that. Still I think most people can learn something from an unprejudiced reading.
And, BTW, I could just be ignorant in a lot of these topics, I take no offense in that. Still I think most people can learn something from an unprejudiced reading.
I think you’re mostly right.
But also that it isn’t what the Yudkowsky is (was?) trying to do with it. I think he’s trying to distill useful tools which increase baseline rationality. Religions have this. It’s what the original philosophers are missing. (At least as taught, happy to hear counter examples)
But also that it isn’t what the Yudkowsky is (was?) trying to do with it. I think he’s trying to distill useful tools which increase baseline rationality. Religions have this. It’s what the original philosophers are missing. (At least as taught, happy to hear counter examples)
I think I'd rather subscribe to an actual religion, than listen to these weird rationalist types of people who seem to have solved the problem that is "everything". At least there is some interesting history to learn about with religion
I would too if I could but organized religions make me uncomfortable even though I admire parts of them. Similar to my admiration you don’t need to like the rationality types or believe in their program to find one or more of their tools useful.
I’ll also respond to the silent downvoters apparent disagreement. CFAR holds workshops and a summer camp for teaching rationality tools. In HPMoR Harry discusses the way he thinks and why. I read it as more of a way to discuss EY’s views in fiction as much as fiction itself.
I’ll also respond to the silent downvoters apparent disagreement. CFAR holds workshops and a summer camp for teaching rationality tools. In HPMoR Harry discusses the way he thinks and why. I read it as more of a way to discuss EY’s views in fiction as much as fiction itself.
For example, I recall being in lot of arguments that are purely "semantical" in nature.
In spirit of playing said game, I believe you can just use the word "pedantic" these days.
[deleted]
[deleted]
Your time would probably be better spent reading his magnum opus, Harry Potter and the Methods of Rationality.
https://hpmor.com/
https://hpmor.com/
Sounds close to Yuval's book nexus which talks about the history of information gathering
If you're in it just to figure out the core argument for why artificial intelligence is dangerous, please consider reading the first few chapters of Nick Bostom's Superintelligence instead. You'll get a lot more bang for your buck that way.
I've never thought ill of Scott Aaronson and have often admired him and his work when I stumble across it.
However, reading this article about all these people at their "Galt's Gultch", I thought — "oh, I guess he's a rhinoceros now"
https://en.wikipedia.org/wiki/Rhinoceros_(play)
Here's a bad joke for you all — What's the difference between a "rationalist" and "rationalizer"? Only the incentives.
However, reading this article about all these people at their "Galt's Gultch", I thought — "oh, I guess he's a rhinoceros now"
https://en.wikipedia.org/wiki/Rhinoceros_(play)
Here's a bad joke for you all — What's the difference between a "rationalist" and "rationalizer"? Only the incentives.
I have always considered Scott Aaronson the least bad of the big-name rationalists. Which makes it slightly funny that he didn't realize he was one until Scott Siskind told him he was.
Reminds me of Simone de Beauvoir and feminism. She wrote the book on (early) feminism, yet didn't consider herself a feminist until much later.
Upvote for the play link - that's interesting and I hadn't heard of it before. Worthy of a top-level post IMO.
I heard of the play originally from Chapter 10 of On Tyranny by Timothy Snyder:
https://archive.org/details/on-tyranny-twenty-lessons-from-t...
Which I did post top-level here on November 7th - https://news.ycombinator.com/item?id=42071791
Unfortunately it didn't a lot of traction and dang told me that there wasn't a way to re-up or "second chance" the post due to the HN policy on posts "correlated with political conflict".
https://archive.org/details/on-tyranny-twenty-lessons-from-t...
Which I did post top-level here on November 7th - https://news.ycombinator.com/item?id=42071791
Unfortunately it didn't a lot of traction and dang told me that there wasn't a way to re-up or "second chance" the post due to the HN policy on posts "correlated with political conflict".
Ah, I guess I see his point; I can't see the discussion being about use of metaphor in political fiction rather than whose team is worst.
Still, I'm glad I now know the reference.
Still, I'm glad I now know the reference.
«What's the difference between a "rationalist" and "rationalizer"? Only the incentives.»
Bad joke? That phrase should be framed in big print.
Bad joke? That phrase should be framed in big print.
This is vibe-based, but I think the Rationalists get more vitriol than they deserve. Upon reflecting, my hypothesis for this is threefold:
1. They are a community—they have an in-group, and if you are not one of them you are by-definition in the out-group. People tend not to like being in other peoples' out-groups.
2. They have unusual opinions and are open about them. People tend not to like people who express opinions different than their own.
3. They're nerds. Whatever has historically caused nerds to be bullied/ostracized, they probably have.
1. They are a community—they have an in-group, and if you are not one of them you are by-definition in the out-group. People tend not to like being in other peoples' out-groups.
2. They have unusual opinions and are open about them. People tend not to like people who express opinions different than their own.
3. They're nerds. Whatever has historically caused nerds to be bullied/ostracized, they probably have.
> They are a community—they have an in-group, and if you are not one of them you are by-definition in the out-group.
The rationalist community is most definitely not exclusive. You can join it by declaring yourself to be a rationalist, posting blogs with "epistemic status" taglines, and calling yourself a rationalist.
The criticisms are not because it's a cool club that won't let people in.
> They have unusual opinions and are open about them. People tend not to like people who express opinions different than their own.
Herein lies one of the problems with the rationalist community: For all of their talk about heterodox ideas and entertaining different viewpoints, they are remarkably lockstep in many of their opinions.
From the outside, it's easy to see how one rationalist blogger plants the seed of some topic and then it gets adopted by the others as fact. A few years ago a rationalist blogger wrote a long series postulating that trace lithium in water was causing obesity. It even got an Astral Codex Ten monetary grant. For years it got shared through the rationalist community as proof of something, even though actual experts picked it apart from the beginning and showed how the author was misinterpreting studies, abusing statistics, and ignoring more prominent factors.
The problem isn't differing opinions, the problem is that they disregard actual expertise and try ham-fisted attempts at "first principals" evaluations of a subject while ignoring contradictory evidence and they do this very frequently.
The rationalist community is most definitely not exclusive. You can join it by declaring yourself to be a rationalist, posting blogs with "epistemic status" taglines, and calling yourself a rationalist.
The criticisms are not because it's a cool club that won't let people in.
> They have unusual opinions and are open about them. People tend not to like people who express opinions different than their own.
Herein lies one of the problems with the rationalist community: For all of their talk about heterodox ideas and entertaining different viewpoints, they are remarkably lockstep in many of their opinions.
From the outside, it's easy to see how one rationalist blogger plants the seed of some topic and then it gets adopted by the others as fact. A few years ago a rationalist blogger wrote a long series postulating that trace lithium in water was causing obesity. It even got an Astral Codex Ten monetary grant. For years it got shared through the rationalist community as proof of something, even though actual experts picked it apart from the beginning and showed how the author was misinterpreting studies, abusing statistics, and ignoring more prominent factors.
The problem isn't differing opinions, the problem is that they disregard actual expertise and try ham-fisted attempts at "first principals" evaluations of a subject while ignoring contradictory evidence and they do this very frequently.
> The rationalist community is most definitely not exclusive.
I agree, and didn't intend to express otherwise. It's not an exclusive community, but it is a community, and if you aren't in it you are in the out-group.
> The problem isn't differing opinions, the problem is that they disregard actual expertise and try ham-fisted attempts at "first principals" evaluations of a subject while ignoring contradictory evidence
I don't know if this is true or not, but if it is I don't think it's why people scorn them. Maybe I don't give people enough credit and you do, but I don't think most people care how you arrived at an opinion; they merely care about whether you're in their opinion-tribe or not.
I agree, and didn't intend to express otherwise. It's not an exclusive community, but it is a community, and if you aren't in it you are in the out-group.
> The problem isn't differing opinions, the problem is that they disregard actual expertise and try ham-fisted attempts at "first principals" evaluations of a subject while ignoring contradictory evidence
I don't know if this is true or not, but if it is I don't think it's why people scorn them. Maybe I don't give people enough credit and you do, but I don't think most people care how you arrived at an opinion; they merely care about whether you're in their opinion-tribe or not.
> Maybe I don't give people enough credit and you do, but I don't think most people care how you arrived at an opinion; they merely care about whether you're in their opinion-tribe or not.
Yes, most people don't care how you arrived at an opinion, they rather care about the practical impact of said opinion. IMO this is largely a good thing.
You can logically push yourself to just about any opinion, even absolutely horrific ones. Everyone has implicit biases and everyone is going to start at a different starting point. The problem with string of logic for real-world phenomena is that you HAVE to make assumptions. Like, thousands of them. Because real-world phenomena are complex and your model is simple. Which assumptions you choose to make and in which directions are completely unknown, even to you, the one making said assumptions.
Ultimately most people aren't going to sit here and try to psychoanalyze why you made the assumptions you made and if you were abused in childhood or deduce which country you grew up in or whatever. It's too much work and it's pointless - you yourself don't know, so how would we know?
So, instead, we just look at the end opinion. If it's crazy, people are just going to call you crazy. Which I think is fair.
Yes, most people don't care how you arrived at an opinion, they rather care about the practical impact of said opinion. IMO this is largely a good thing.
You can logically push yourself to just about any opinion, even absolutely horrific ones. Everyone has implicit biases and everyone is going to start at a different starting point. The problem with string of logic for real-world phenomena is that you HAVE to make assumptions. Like, thousands of them. Because real-world phenomena are complex and your model is simple. Which assumptions you choose to make and in which directions are completely unknown, even to you, the one making said assumptions.
Ultimately most people aren't going to sit here and try to psychoanalyze why you made the assumptions you made and if you were abused in childhood or deduce which country you grew up in or whatever. It's too much work and it's pointless - you yourself don't know, so how would we know?
So, instead, we just look at the end opinion. If it's crazy, people are just going to call you crazy. Which I think is fair.
Some of them certainly do have some opinions outside the Overton Windows of many.
Lockstep like this? https://www.lesswrong.com/posts/7iAABhWpcGeP5e6SB/it-s-proba... (a post on Less Wrong, karma score currently +442, versus +102 and +230 for the two posts it cites as earlier favourable LW coverage of the lithium claim -- the comments on both of which, by the way, don't look to me any more positive than "skeptical but interested")
The followup post from the same author https://www.lesswrong.com/posts/NRrbJJWnaSorrqvtZ/on-not-get... is currently at a score of +306, again higher than either of those other pro-lithium-hypothesis posts.
Or maybe this https://substack.com/home/post/p-39247037 (I admit I don't know for sure whether the author considers himself a rationalist, but I found the link via a search for whether Scott Alexander had written anything about the lithium theory, which it looks like he hasn't, which turned this up in the subreddit dedicated to his writing).
Speaking of which, I can't find any sign that they got an ACX grant. I can find https://www.astralcodexten.com/p/acx-grants-the-first-half which is basically "hey, here are some interesting projects we didn't give any money to, with a one-paragraph pitch from each" and one of the things there is "Slime Mold Time Mold" talking about lithium; incidentally, the comments there are also pretty skeptical.
So I'm not really seeing this "gets adopted by the others as fact" thing in this case; it looks to me as if some people proposed this hypothesis, some other people said "eh, doesn't look right to me", and rationalists' attitude was mostly "interesting idea but probably wrong". What am I missing here?
The followup post from the same author https://www.lesswrong.com/posts/NRrbJJWnaSorrqvtZ/on-not-get... is currently at a score of +306, again higher than either of those other pro-lithium-hypothesis posts.
Or maybe this https://substack.com/home/post/p-39247037 (I admit I don't know for sure whether the author considers himself a rationalist, but I found the link via a search for whether Scott Alexander had written anything about the lithium theory, which it looks like he hasn't, which turned this up in the subreddit dedicated to his writing).
Speaking of which, I can't find any sign that they got an ACX grant. I can find https://www.astralcodexten.com/p/acx-grants-the-first-half which is basically "hey, here are some interesting projects we didn't give any money to, with a one-paragraph pitch from each" and one of the things there is "Slime Mold Time Mold" talking about lithium; incidentally, the comments there are also pretty skeptical.
So I'm not really seeing this "gets adopted by the others as fact" thing in this case; it looks to me as if some people proposed this hypothesis, some other people said "eh, doesn't look right to me", and rationalists' attitude was mostly "interesting idea but probably wrong". What am I missing here?
> Lockstep like this? https://www.lesswrong.com/posts/7iAABhWpcGeP5e6SB/it-s-proba... (a post on Less Wrong, karma score currently +442, versus +102 and +230 for the two posts it cites as earlier favourable LW coverage of the lithium claim -- the comments on both of which, by the way, don't look to me any more positive than "skeptical but interested")
That post came out a year later, in response to the absurdity of the situation. The very introduction of that post has multiple links showing how much the SMTM post was spreading through the rationalist community with little question.
One of the links is a Eliezer Yudkowsky blog praising the work, which now includes an edited-in disclaimer at the top about how he was mistaken: https://www.lesswrong.com/posts/kjmpq33kHg7YpeRYW/briefly-ra...
Pretending that this theory didn't grip the rationalist community all the way to top bloggers like Yudkowsky and Scott Alexander is revisionist history.
That post came out a year later, in response to the absurdity of the situation. The very introduction of that post has multiple links showing how much the SMTM post was spreading through the rationalist community with little question.
One of the links is a Eliezer Yudkowsky blog praising the work, which now includes an edited-in disclaimer at the top about how he was mistaken: https://www.lesswrong.com/posts/kjmpq33kHg7YpeRYW/briefly-ra...
Pretending that this theory didn't grip the rationalist community all the way to top bloggers like Yudkowsky and Scott Alexander is revisionist history.
> That post came out a year later [...] multiple links showing how much the SMTM post was spreading through the rationalist community with little question.
The SMTM series started in July 2021 and finished in November 2021; there was also a paper, similar enough that I assume it's by the same people, from July 2021. The first of those "multiple links" is from July 2021, but the second is from January 2022 and the third from May 2022. The critical post is from June 2022. I agree it's a year later than something but I'm not seeing that the SMTM theory was "spreading ... with little question" a year before it.
The "multiple links" you mention -- the actual number is three -- are the two I mentioned before and a third that (my apologies!) I had somehow not noticed. That third one is at +74 karma, again much lower than the later critical post, and it doesn't endorse the lithium theory.
The one written by E.Y. is the second. Quite aside from the later disclaimer, it's hardly an uncritical endorsement: "you are still probably saying "Wait, lithium?" This is still mostly my own reaction, honestly." and "low-probability massive-high-value gamble".
What about the first post? That one's pretty positive, but to me it reads as "here's an interesting theory; it sounds plausible to me but I am not an expert" rather than "here's a theory that is probably right", still less "here's a theory that is definitely right".
The comments, likewise, don't look to me like lockstep uncritical acceptance. I see "here are some interesting experiments one could do to check this" and "something like this seems plausible but I bet the actual culprit is vegetable oils" and "something like this seems plausible but I bet the actual culprit is rising CO2 levels" and "I bet it's corn somehow" and "quite convincing but didn't really rule out the obvious rival hypothesis" and so forth; I don't think a single one of the comments is straightforwardly agreeing with the theory.
If you've found something Scott Alexander wrote about this then I'd be interested to see it. All I found was that (contrary to what you claimed above) it looks like ACX Grants declined to fund exploration of the lithium theory but included that proposal in a list of "interesting things we didn't fund".
So I'm just not seeing this lockstep thing you claim. Maybe I'm looking in the wrong places. The specific things you've cited don't seem like they support it: you said there was an ACX grant but there wasn't; you say the links in the intro to that critical post show the theory spreading with little question, but what they actually show is one person saying "here's an interesting theory but I'm not an expert", E.Y. saying "here's a theory that's probably wrong but worth looking into" (and later changing his mind), and another person saying "I put together some data that might be relevant"; in every case the comments are full of people not agreeing with the lithium theory.
The SMTM series started in July 2021 and finished in November 2021; there was also a paper, similar enough that I assume it's by the same people, from July 2021. The first of those "multiple links" is from July 2021, but the second is from January 2022 and the third from May 2022. The critical post is from June 2022. I agree it's a year later than something but I'm not seeing that the SMTM theory was "spreading ... with little question" a year before it.
The "multiple links" you mention -- the actual number is three -- are the two I mentioned before and a third that (my apologies!) I had somehow not noticed. That third one is at +74 karma, again much lower than the later critical post, and it doesn't endorse the lithium theory.
The one written by E.Y. is the second. Quite aside from the later disclaimer, it's hardly an uncritical endorsement: "you are still probably saying "Wait, lithium?" This is still mostly my own reaction, honestly." and "low-probability massive-high-value gamble".
What about the first post? That one's pretty positive, but to me it reads as "here's an interesting theory; it sounds plausible to me but I am not an expert" rather than "here's a theory that is probably right", still less "here's a theory that is definitely right".
The comments, likewise, don't look to me like lockstep uncritical acceptance. I see "here are some interesting experiments one could do to check this" and "something like this seems plausible but I bet the actual culprit is vegetable oils" and "something like this seems plausible but I bet the actual culprit is rising CO2 levels" and "I bet it's corn somehow" and "quite convincing but didn't really rule out the obvious rival hypothesis" and so forth; I don't think a single one of the comments is straightforwardly agreeing with the theory.
If you've found something Scott Alexander wrote about this then I'd be interested to see it. All I found was that (contrary to what you claimed above) it looks like ACX Grants declined to fund exploration of the lithium theory but included that proposal in a list of "interesting things we didn't fund".
So I'm just not seeing this lockstep thing you claim. Maybe I'm looking in the wrong places. The specific things you've cited don't seem like they support it: you said there was an ACX grant but there wasn't; you say the links in the intro to that critical post show the theory spreading with little question, but what they actually show is one person saying "here's an interesting theory but I'm not an expert", E.Y. saying "here's a theory that's probably wrong but worth looking into" (and later changing his mind), and another person saying "I put together some data that might be relevant"; in every case the comments are full of people not agreeing with the lithium theory.
> The very introduction of that post has multiple links showing how much the SMTM post was spreading through the rationalist community with little question.
By "multiple links" you're referring to the same "two posts". Again, they weren't as popular, nor were they as uncritical as you describe. From Yudkowsky's post, for example:
> If you know about the actual epidemiology of obesity and how ridiculous it makes the gluttony theory look, you are still probably saying "Wait, lithium?" This is still mostly my own reaction, honestly.... If some weird person wants to go investigate, I think money should be thrown at them, both to check the low-probability massive-high-value gamble
Yudkowsky's argument is emphatically not that the lithium claim is true. He was merely advocating for someone to fund a study. He explicitly describes the claim as "low-probability", and advocates on the basis of a (admittedly clearly subjective) expected-value calculation.
> One of the links is a Eliezer Yudkowsky blog praising the work
That does not constitute "praise" of the work. Yudkowsky only praised the fact that someone was bucking the trend of
> almost nobody is investigating it in a way that takes the epidemiological facts seriously and elevates those above moralistic gluttony theories
.
> Pretending that this theory didn't grip the rationalist community all the way to top bloggers like Yudkowsky and Scott Alexander is revisionist history.
Nobody claimed that Yudkowsky ignored the theory.
By "multiple links" you're referring to the same "two posts". Again, they weren't as popular, nor were they as uncritical as you describe. From Yudkowsky's post, for example:
> If you know about the actual epidemiology of obesity and how ridiculous it makes the gluttony theory look, you are still probably saying "Wait, lithium?" This is still mostly my own reaction, honestly.... If some weird person wants to go investigate, I think money should be thrown at them, both to check the low-probability massive-high-value gamble
Yudkowsky's argument is emphatically not that the lithium claim is true. He was merely advocating for someone to fund a study. He explicitly describes the claim as "low-probability", and advocates on the basis of a (admittedly clearly subjective) expected-value calculation.
> One of the links is a Eliezer Yudkowsky blog praising the work
That does not constitute "praise" of the work. Yudkowsky only praised the fact that someone was bucking the trend of
> almost nobody is investigating it in a way that takes the epidemiological facts seriously and elevates those above moralistic gluttony theories
.
> Pretending that this theory didn't grip the rationalist community all the way to top bloggers like Yudkowsky and Scott Alexander is revisionist history.
Nobody claimed that Yudkowsky ignored the theory.
Aside from the remark given in the other reply to your comment, I wonder what the standard is: how quickly should a community appear to correct its incorrect beliefs for them to not count as sheep?
> A few years ago a rationalist blogger wrote a long series postulating that trace lithium in water was causing obesity. It even got an Astral Codex Ten monetary grant. For years it got shared through the rationalist community as proof of something
As proof of what, exactly? And where is your evidence that such a thing happened?
> while ignoring contradictory evidence and they do this very frequently.
The evidence available to me suggests that the rationalist community was not at all "lockstep" as regards the evaluation of SMTM's hypothesis.
As proof of what, exactly? And where is your evidence that such a thing happened?
> while ignoring contradictory evidence and they do this very frequently.
The evidence available to me suggests that the rationalist community was not at all "lockstep" as regards the evaluation of SMTM's hypothesis.
Agree, but I think there is another, more important factor. They are a highly visible part of the internet, and their existence is mainly internet-based. This means that the people assessing them are mainly on the internet, and as we all know, internet discourse tends to the blandly negative (ironically my own comment is a mild example of this).
Great point!
I will throw in an additional factor: any group, community, or segmentation of the general population wherein the participants both tend to have a higher than average intelligence (whatever that means) and whose preoccupation revolves around almost any form of cogitation, consideration, products of human thought ... will invariably get hit with some form of snobbery/envy, even if no explicitly stated intelligence threshold or gatekeeping is present.
Bluntly put, you are not allowed to be even a little smart and not all "aww shucks" about it. It has to be in service of something else like medicine or being a CPA. (Fun fact I found in a statistics course: the average CPA has about five points of IQ on the average doctor.) And it is almost justified, because you are in constant peril of falling down into your own butt until you disappear, but at the same time it keeps a lot of people under the thumb (or heel, pick your oppressive body part) of dumbass managers and idiots who blithely plow forward without a trace of doubt.
Bluntly put, you are not allowed to be even a little smart and not all "aww shucks" about it. It has to be in service of something else like medicine or being a CPA. (Fun fact I found in a statistics course: the average CPA has about five points of IQ on the average doctor.) And it is almost justified, because you are in constant peril of falling down into your own butt until you disappear, but at the same time it keeps a lot of people under the thumb (or heel, pick your oppressive body part) of dumbass managers and idiots who blithely plow forward without a trace of doubt.
HN judges rationality quite severely. I mean, look at this thread about Mr. Beast[1], who it's safe to say is a controversial figure, and notice how all the top comments are all pretty charitable. It's pretty funny to take the conversation there and then compare the comments to this article.
Scott Aaronson - in theory someone HN should be a huge fan of, from all reports a super nice and extremely intelligent guy who knows a staggering amount about quantum mechanics - says he likes rationality, and gets less charity than Mr. Beast. Huh?
[1]: https://news.ycombinator.com/item?id=41549649
Scott Aaronson - in theory someone HN should be a huge fan of, from all reports a super nice and extremely intelligent guy who knows a staggering amount about quantum mechanics - says he likes rationality, and gets less charity than Mr. Beast. Huh?
[1]: https://news.ycombinator.com/item?id=41549649
The people commenting under the Mr. Beast post are probably different to the people commenting under this post.
Anyway, Mr. Beast doesn't really pretend to be more than what he is afaik. In contrast, the Rationalist tendency to use mathematics (especially Bayes's theorem) as window dressing is really, really annoying.
Anyway, Mr. Beast doesn't really pretend to be more than what he is afaik. In contrast, the Rationalist tendency to use mathematics (especially Bayes's theorem) as window dressing is really, really annoying.
What HN has for the rationalist movement isn’t just annoyance - it’s deep contempt and hatred.
Most people are trying to be rational (to be sure, with varying degrees of success), and people who aren't even trying aren't really worth having abstract intellectual discussions with. I'm reminded of CS Lewis's quip in a different context that "you might just as well expect to be congratulated because, whenever you do a sum, you try to get it quite right."
Being rational and rationalist are not the same thing. Funnily this sort of false equivalence that relies on being "technically correct" is at the core of what makes them...difficult.
There's more to the group identity than just applying logic and rationality, yes. But surely 'rationalists' of all people wouldn't want to take the position that some of their key ideas don't result primarily from applying rational thought processes. Any part of their worldview that doesn't follow immediately from foundational principles of logic and reason is presumably subject to revision based on evidence and argument.
Right but their version of rationality is very much lacking in emotional and even social intelligence. In fact it seems those traits are discouraged in discussions. Instead things are viewed from a purely Bayesian/Spock/Sherlocke/Data style perspective. These arguments can only make sense by assuming far more predictive power (or far less accounting for chaos and hidden variables) and by developing blind spots not found in softer arguments which have their own problems but at least try to consider ethics and economics from more nuanced perspectives.
Their core principle seems to be that many or even most answers to humanity's problems can be well defined in the first place and then solved by invoking Bayesian logic. This is only true in a frictionless vacuum.
Their core principle seems to be that many or even most answers to humanity's problems can be well defined in the first place and then solved by invoking Bayesian logic. This is only true in a frictionless vacuum.
Fittingly enough, the Rationalist community talks about this a lot. The canonical article is here ("I can tolerate anything except the outgroup").*
The gist is that if people are really different from us then we tend to be cool with them. But if they're close to us - but not quite the same - then they tend to annoy us. Hacker News people are close enough to Rationalists that HN people find them annoying.
It's the same reason why e.g. Hitler-style Neo Nazis can have a beer with Black Nationalists, but they tend to despise Klan-style Neo Nazis. Or why Sunni and Shia Muslims have issues with each other but neither group really cares about Indigenous American religions or whatever.
* https://slatestarcodex.com/2014/09/30/i-can-tolerate-anythin...
The gist is that if people are really different from us then we tend to be cool with them. But if they're close to us - but not quite the same - then they tend to annoy us. Hacker News people are close enough to Rationalists that HN people find them annoying.
It's the same reason why e.g. Hitler-style Neo Nazis can have a beer with Black Nationalists, but they tend to despise Klan-style Neo Nazis. Or why Sunni and Shia Muslims have issues with each other but neither group really cares about Indigenous American religions or whatever.
* https://slatestarcodex.com/2014/09/30/i-can-tolerate-anythin...
> This is vibe-based
You mean an empirical observation
You mean an empirical observation
Three examples of feelings-based conclusions were presented. There is what is so, and how you feel about them. By all means be empirical about what you felt, and maybe look into that. “How this made me feel” describes the cause of how we got the USA today.
I think these are all true and relevant, but the main problem is that their thesis that "ASI alignment will be extremely difficult" can only really be proven in hindsight.
It's like they're crying wolf but can't prove there's actually a wolf, only vague signs of one, but if the wolf ever becomes visible it will be way too late to do anything. Obviously no one is going to respect a group like that and many people will despise them.
It's like they're crying wolf but can't prove there's actually a wolf, only vague signs of one, but if the wolf ever becomes visible it will be way too late to do anything. Obviously no one is going to respect a group like that and many people will despise them.
Nope. Has nothing to do with them being nerds. They are actively dangerous, their views almost always lead to extremely reactionary politics. EA and RA are deeply anti-human. In some cases that manifests as a desire to subjugate humanity with an iron fist technocratic rule, in other cases it manifests as a desire to kill off humanity.
Either way, as an ideology it must be stopped. It should not be treated with kids gloves, it is an ideology that is actively influencing the ruling elites right now (JD Vance, Musk, Thiel are part of this cult, and also simultaneously believe in German-style Nazism, which is broadly compatible with RA). The only silver lining is that some of their ideas about power-seeking tactics are so ineffective they will never work -- in other words, humanity will prevail over these ghouls, because they came in with so many bad assumptions that they've lost touch with reality.
Either way, as an ideology it must be stopped. It should not be treated with kids gloves, it is an ideology that is actively influencing the ruling elites right now (JD Vance, Musk, Thiel are part of this cult, and also simultaneously believe in German-style Nazism, which is broadly compatible with RA). The only silver lining is that some of their ideas about power-seeking tactics are so ineffective they will never work -- in other words, humanity will prevail over these ghouls, because they came in with so many bad assumptions that they've lost touch with reality.
If you're going to claim a group is "deeply anti-human ghouls", maybe include an example or 2 of this in your post.
Huh? They give millions of dollars to global humanitarian development funds saving at least 50,000 lives per year. Maybe you’re taking a few kooks who have a controversial lecture somewhere as representing everyone else.
It is true that there were EAs before the conception of "longtermism" that were relatively mundane and actually helping humanity, and not part of a death cult. But those have been shunned from the EA movement for a while now.
Global Health & Development is like 40% of EA funding today. Please stop making things up.
I think you're conflating different groups of people pretty severely.
"shunned" in particular is a really strong word, e.g, global health and biosecurity are two of the named categories at the most central EA events:
https://www.effectivealtruism.org/ea-global/events/ea-global...
"shunned" in particular is a really strong word, e.g, global health and biosecurity are two of the named categories at the most central EA events:
https://www.effectivealtruism.org/ea-global/events/ea-global...
In what specific way do you disagree with them?
The support for eugenics and ethnic cleansing, the absolute obsession with strictly utilitarian ethics and ignorance of other ethics, the "kill a bunch of humans now so that trillions can live in the future" longtermist death cult, and the whole Roko's basilisk worship that usually goes like "one AI system can take over the entirety of humanity and start eating the galaxy, therefore we must forcefully jump in the driver seat of that dangerous AI right now so that our elite ideology is locked in for a trillion years of galactic evolution".
Probably the most useful book ever written about topics adjacent to capital-R Rationalism is "Neoreaction, A Basilisk: Essays on and Around the Alt-Right" [1], by Elizabeth Sandifer. Though the topic of the book is nominally the Alt-Right, a lot more of it is about the capital-R Rationalist communities and individuals that incubated the neoreactionary movement that is currently dominant in US politics. It's probably the best book to read for understanding how we got politically and intellectually from where we were in 2010, to where we are now.
https://www.goodreads.com/book/show/41198053-neoreaction-a-b...
https://www.goodreads.com/book/show/41198053-neoreaction-a-b...
If you want a book on the rationalists that's not a smear dictated by a person who is banned from their Wikipedia page for massive npov violations, I hear Chivers' The AI Does Not Hate You and Rationalist's Guide to the Galaxy are good.
(Disclaimer: Chivers kinda likes us, so if you like one book you'll probably dislike the other.)
(Disclaimer: Chivers kinda likes us, so if you like one book you'll probably dislike the other.)
It might be fair play, however. If I correctly recall, LessWrong had, for a while, a prominent wiki admin who had been punted from Wikipedia for his frothing npov.
I am not aware of Less Wrong having a Wikipedia admin. Are you perhaps thinking about David Gerard, admin of RationalWiki and Wikipedia, who once got in trouble for his decade-long internet crusade against Scott Alexander?
https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_no...
https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_no...
> Probably the most useful book
You mean "probably the book that confirms my biases the most"
You mean "probably the book that confirms my biases the most"
> incubated the neoreactionary movement that is currently dominant in US politics
> Please don't use Hacker News for political or ideological battle. It tramples curiosity.
You are presenting a highly contentious worldview for the sake of smearing an outgroup. Please don't. Further, the smear relies on guilt by association that many (including myself) would consider invalid on principle, and which further doesn't even bear out on cursory examination.
At least take a moment to see how others view the issue. "Reliable Sources: How Wikipedia Admin David Gerard Launders His Grudges Into the Public Record" https://www.tracingwoodgrains.com/p/reliable-sources-how-wik... includes lengthy commentary on Sandifer (a close associate of Gerard)'s involvement with rationalism, and specifically on the work you cite and its biases.
> Please don't use Hacker News for political or ideological battle. It tramples curiosity.
You are presenting a highly contentious worldview for the sake of smearing an outgroup. Please don't. Further, the smear relies on guilt by association that many (including myself) would consider invalid on principle, and which further doesn't even bear out on cursory examination.
At least take a moment to see how others view the issue. "Reliable Sources: How Wikipedia Admin David Gerard Launders His Grudges Into the Public Record" https://www.tracingwoodgrains.com/p/reliable-sources-how-wik... includes lengthy commentary on Sandifer (a close associate of Gerard)'s involvement with rationalism, and specifically on the work you cite and its biases.
Thanks for the recommendation! I hadn't heard about the book.
Ironically, bringing this topic up always turns the conversation to ad-hominem attacks about the messenger while completely ignoring the subject matter. That's exactly the type of argument rationalists claim to despise, but it gets brought up whenever inconvenient arguments appear about their own communities. All of the comments dismissing the content because of the author or refusing to acknowledge the arguments because it feels like a "smear" are admitting their inability to judge an argument on their own merits.
If anyone wants to actually engage with the topic instead of trying to ad-hominem it away, I suggest at least reading Scott Alexander's own words on why he so frequently engages in neoreactionary topics: https://www.reddit.com/r/SneerClub/comments/lm36nk/comment/g...
Some select quotes:
> First is a purely selfish reason - my blog gets about 5x more hits and new followers when I write about Reaction or gender than it does when I write about anything else, and writing about gender is horrible. Blog followers are useful to me because they expand my ability to spread important ideas and network with important people.
> Third is that I want to spread the good parts of Reactionary thought
> Despite considering myself pretty smart and clueful, I constantly learn new and important things (like the crime stuff, or the WWII history, or the HBD) from the Reactionaries. Anything that gives you a constant stream of very important new insights is something you grab as tight as you can and never let go of.
In this case, HBD means "human biodiversity" which is the alt-right's preferred term for racialism, or the division of humans into races with special attention to the relative intelligence of those different races. This is an oddly recurring theme on Scott Alexander's work. He even wrote a coded blog post to his followers about how he was going to deny it publicly while privately holding it to be very correct.
If anyone wants to actually engage with the topic instead of trying to ad-hominem it away, I suggest at least reading Scott Alexander's own words on why he so frequently engages in neoreactionary topics: https://www.reddit.com/r/SneerClub/comments/lm36nk/comment/g...
Some select quotes:
> First is a purely selfish reason - my blog gets about 5x more hits and new followers when I write about Reaction or gender than it does when I write about anything else, and writing about gender is horrible. Blog followers are useful to me because they expand my ability to spread important ideas and network with important people.
> Third is that I want to spread the good parts of Reactionary thought
> Despite considering myself pretty smart and clueful, I constantly learn new and important things (like the crime stuff, or the WWII history, or the HBD) from the Reactionaries. Anything that gives you a constant stream of very important new insights is something you grab as tight as you can and never let go of.
In this case, HBD means "human biodiversity" which is the alt-right's preferred term for racialism, or the division of humans into races with special attention to the relative intelligence of those different races. This is an oddly recurring theme on Scott Alexander's work. He even wrote a coded blog post to his followers about how he was going to deny it publicly while privately holding it to be very correct.
> Ironically, bringing this topic up always turns the conversation to ad-hominem attacks about the messenger while completely ignoring the subject matter.
This is not a fair or accurate characterization of the criticism you're referring to.
> All of the comments dismissing the content because of the author or refusing to acknowledge the arguments because it feels like a "smear" are admitting their inability to judge an argument on their own merits.
They are not doing any such thing. The content is being dismissed because it has been repeatedly evaluated before and found baseless. The arguments are acknowledged as specious. Sandifer makes claims that are not supported by the evidence and are in fact directly contradicted by the evidence.
This is not a fair or accurate characterization of the criticism you're referring to.
> All of the comments dismissing the content because of the author or refusing to acknowledge the arguments because it feels like a "smear" are admitting their inability to judge an argument on their own merits.
They are not doing any such thing. The content is being dismissed because it has been repeatedly evaluated before and found baseless. The arguments are acknowledged as specious. Sandifer makes claims that are not supported by the evidence and are in fact directly contradicted by the evidence.
> my blog gets about 5x more hits and new followers when I write about Reaction
Notice that most of that writing is negative, such as "anti-Reactionary manifesto" or more recently "Moldbug sold out".
Notice that most of that writing is negative, such as "anti-Reactionary manifesto" or more recently "Moldbug sold out".
That book, IMO, reads very much like a smear attempt, and not one done with a good understanding of the target.
The premise, with an attempt to tie capital-R Rationalists to the neoreactionaries though a sort of guilt by association, is frankly weird: Scott Alexander is well-known among the former to be essentially the only prominent figure that takes the latter seriously—seriously enough, that is, to write a large as-well-stated-as-possible survey[1] followed by a humongous point-by-point refutation[2,3]; whereas the “cult leader” of the rationalists, Yudkowsky, is on the record as despising neoreactionaries to the point of refusing to discuss their views. (As far as recent events, Alexander wrote a scathing review of Yarvin’s involvement in Trumpist politics[4] whose main thrust is that Yarvin has betrayed basically everything he advocated for.)
The story of the book’s conception also severely strains an assumption of good faith[5]: the author, Elizabeth Sandifer, explicitly says it was to a large extent inspired, sourced, and edited by David Gerard, a prominent contributor to RationalWiki and r/SneerClub (the “sneerers” mentioned in TFA) and Wikipedia administrator who after years of edit-warring got topic-banned from editing articles about Scott Alexander (Scott Siskind) for conflict of interest and defamation[6] (including adding links to the book as a source for statements on Wikipedia about links between rationalists and neoreaction). Elizabeth Sandifer herself got banned for doxxing a Wikipedia editor during Gerard's earlier edit war at the time of Manning's gender transition, for which Gerard was also sanctioned[7].
[1] https://slatestarcodex.com/2013/03/03/reactionary-philosophy...
[2] https://slatestarcodex.com/2013/10/20/the-anti-reactionary-f...
[3] https://slatestarcodex.com/2013/10/24/some-preliminary-respo...
[4] https://www.astralcodexten.com/p/moldbug-sold-out
[5] https://www.tracingwoodgrains.com/p/reliable-sources-how-wik...
[6] https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_no...
[7] https://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests...
The premise, with an attempt to tie capital-R Rationalists to the neoreactionaries though a sort of guilt by association, is frankly weird: Scott Alexander is well-known among the former to be essentially the only prominent figure that takes the latter seriously—seriously enough, that is, to write a large as-well-stated-as-possible survey[1] followed by a humongous point-by-point refutation[2,3]; whereas the “cult leader” of the rationalists, Yudkowsky, is on the record as despising neoreactionaries to the point of refusing to discuss their views. (As far as recent events, Alexander wrote a scathing review of Yarvin’s involvement in Trumpist politics[4] whose main thrust is that Yarvin has betrayed basically everything he advocated for.)
The story of the book’s conception also severely strains an assumption of good faith[5]: the author, Elizabeth Sandifer, explicitly says it was to a large extent inspired, sourced, and edited by David Gerard, a prominent contributor to RationalWiki and r/SneerClub (the “sneerers” mentioned in TFA) and Wikipedia administrator who after years of edit-warring got topic-banned from editing articles about Scott Alexander (Scott Siskind) for conflict of interest and defamation[6] (including adding links to the book as a source for statements on Wikipedia about links between rationalists and neoreaction). Elizabeth Sandifer herself got banned for doxxing a Wikipedia editor during Gerard's earlier edit war at the time of Manning's gender transition, for which Gerard was also sanctioned[7].
[1] https://slatestarcodex.com/2013/03/03/reactionary-philosophy...
[2] https://slatestarcodex.com/2013/10/20/the-anti-reactionary-f...
[3] https://slatestarcodex.com/2013/10/24/some-preliminary-respo...
[4] https://www.astralcodexten.com/p/moldbug-sold-out
[5] https://www.tracingwoodgrains.com/p/reliable-sources-how-wik...
[6] https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_no...
[7] https://en.wikipedia.org/wiki/Wikipedia:Arbitration/Requests...
I always find it interesting that when the topic of rationalists' fixation on neoreactionary topics comes into question, the primary defenses are that it's important to look at controversial ideas and that we shouldn't dismiss novel ideas because we don't like the group sharing them.
Yet as soon as the topic turns to criticisms of the rationalist community, we're supposed to ignore those ideas and instead fixate on the messenger, ignore their arguments, and focus on ad-hominem attacks that reduce their credibility.
It's no secret that Scott Alexander had a bit of a fixation on neoreactionary content for years. The leaked e-mails showed he believed there to be "gold" in some of their ideas and he enjoyed the extra traffic it brought to his blog. I know the rationalist community has been working hard to distance themselves from that era publicly, but dismissing that chapter of the history because it feels too much like a "smear" or because we're not supposed to like the author feels extremely hypocritical given the context.
Yet as soon as the topic turns to criticisms of the rationalist community, we're supposed to ignore those ideas and instead fixate on the messenger, ignore their arguments, and focus on ad-hominem attacks that reduce their credibility.
It's no secret that Scott Alexander had a bit of a fixation on neoreactionary content for years. The leaked e-mails showed he believed there to be "gold" in some of their ideas and he enjoyed the extra traffic it brought to his blog. I know the rationalist community has been working hard to distance themselves from that era publicly, but dismissing that chapter of the history because it feels too much like a "smear" or because we're not supposed to like the author feels extremely hypocritical given the context.
There are certain parts of the history of the rationalist movement that its enemies are orders of magnitude more "fixated" on than rationalists ever were, Neoreaction and the Basilisk being the biggest.
Part of evaluating unusual ideas is that you have to get really good at ignoring bad ones. So when somebody writes a book called "Neoreaction: a Basilisk" and claims that it's about rationality, I make a very simple expected-value calculation.
Part of evaluating unusual ideas is that you have to get really good at ignoring bad ones. So when somebody writes a book called "Neoreaction: a Basilisk" and claims that it's about rationality, I make a very simple expected-value calculation.
> The leaked e-mails
Curious to read these. Got a source?
I've always been very skeptical of Scott "Alexander" after he and his supporters tricked half of reddit into harassing some journalists for "doxxing" him when his identity was public knowledge seemingly because he just really didn't like the takes presented by the journalists. The way he refers to them like it was a hit piece targeting him reeked of conspiratorial and paranoid thinking.
edit:
https://www.nytimes.com/2021/02/13/technology/slate-star-cod...
Curious to read these. Got a source?
I've always been very skeptical of Scott "Alexander" after he and his supporters tricked half of reddit into harassing some journalists for "doxxing" him when his identity was public knowledge seemingly because he just really didn't like the takes presented by the journalists. The way he refers to them like it was a hit piece targeting him reeked of conspiratorial and paranoid thinking.
edit:
https://www.nytimes.com/2021/02/13/technology/slate-star-cod...
> when his identity was public knowledge
The details are important here. His identity was "public knowledge" in the sense that regular readers of his blog could sometimes find links to his previous blog, and somewhere on that previous blog he mentioned his name. So many of his long-term readers knew.
But in the opposite direction -- if all you knew was Scott's full name, and you did a Google search -- there was no connection to the blog. You could find his professional web pages, and that was it.
What the NYT journalists threatened was to make a #1 search result for his full name that would expose his private life and his pseudonymous blog to all potential patients trying to find out some information about their doctor. Which would practically cost him his job.
And, ultimately, Scott did lose his job. The fact that writing on Substack turned out to be more profitable than his former job was a lucky coincidence.
The details are important here. His identity was "public knowledge" in the sense that regular readers of his blog could sometimes find links to his previous blog, and somewhere on that previous blog he mentioned his name. So many of his long-term readers knew.
But in the opposite direction -- if all you knew was Scott's full name, and you did a Google search -- there was no connection to the blog. You could find his professional web pages, and that was it.
What the NYT journalists threatened was to make a #1 search result for his full name that would expose his private life and his pseudonymous blog to all potential patients trying to find out some information about their doctor. Which would practically cost him his job.
And, ultimately, Scott did lose his job. The fact that writing on Substack turned out to be more profitable than his former job was a lucky coincidence.
If the stakes were so high he shouldn't have been so careless with using his real name. As a professional psychiatrist it is 100% on him to make sure these situations aren't a possibility and if that isn't possible- it's really not that big of a deal to simply not blog about your deeply unrestrained and potentially offensive opinions about [insert culture war here].
This is not the fault of the nytimes and given the success of his blog, it absolutely would have happened eventually. It is frankly irresponsible on his part. He chose that profession and with it comes certain sacrifices made for the wellbeing of his patients.
Further he went on crazy rants about how the article was a hit piece which is deeply dramatic and maybe even a little egotistical. He's not nearly as important as he thinks - and the nytimes piece covered him in a fairly neutral way from my perspective.
You should read it if you haven't. It's an enlightening piece about the burgeoning semi-conservative movement masquerading as pseudo liberalism amongst so called thought leaders in silicon valley and tech more generally.
This is not the fault of the nytimes and given the success of his blog, it absolutely would have happened eventually. It is frankly irresponsible on his part. He chose that profession and with it comes certain sacrifices made for the wellbeing of his patients.
Further he went on crazy rants about how the article was a hit piece which is deeply dramatic and maybe even a little egotistical. He's not nearly as important as he thinks - and the nytimes piece covered him in a fairly neutral way from my perspective.
You should read it if you haven't. It's an enlightening piece about the burgeoning semi-conservative movement masquerading as pseudo liberalism amongst so called thought leaders in silicon valley and tech more generally.
> the nytimes piece covered him in a fairly neutral way from my perspective.
Really? I have read the article, and I basically agree with https://www.astralcodexten.com/p/statement-on-new-york-times... and https://scottaaronson.blog/?p=5310
Really? I have read the article, and I basically agree with https://www.astralcodexten.com/p/statement-on-new-york-times... and https://scottaaronson.blog/?p=5310
> when the topic of rationalists' fixation on neoreactionary topics comes into question, the primary defenses are that it's important to look at controversial ideas and that we shouldn't dismiss novel ideas because we don't like the group sharing them.
No. Rationalists do say that it's important to do those things, because that's true. But it is not a defense of a "fixation on neoreactionary topics", because there is no such fixation. It only comes across as a fixation to people who are unwilling to even understand what they are denigrating.
You will note that Scott Alexander is heavily critical of neoreaction.
> Yet as soon as the topic turns to criticisms of the rationalist community, we're supposed to ignore those ideas and instead fixate on the messenger, ignore their arguments, and focus on ad-hominem attacks that reduce their credibility.
No. Nobody said that those criticism should be ignored. What was said is that those criticism are invalid, because they are. It is not ad-hominem against Sandifer to point out that Sandifer is trying to insinuate untrue things about Alexander. It is simply observing reality. Sandifer attempts to describe Alexander, Yudkowsky et. al. as supportive of neoreactionary thought. In reality, Alexander, Yudkowsky et. al. are strongly-critical-at-best of neoreactionary thought.
> The leaked e-mails showed he believed there to be "gold" in some of their ideas
This is clutching at straws. Alexander wrote https://slatestarcodex.com/2013/10/20/the-anti-reactionary-f... , in 2013.
You are engaging in the same kind of semantic games that Sandifer does. Please stop.
No. Rationalists do say that it's important to do those things, because that's true. But it is not a defense of a "fixation on neoreactionary topics", because there is no such fixation. It only comes across as a fixation to people who are unwilling to even understand what they are denigrating.
You will note that Scott Alexander is heavily critical of neoreaction.
> Yet as soon as the topic turns to criticisms of the rationalist community, we're supposed to ignore those ideas and instead fixate on the messenger, ignore their arguments, and focus on ad-hominem attacks that reduce their credibility.
No. Nobody said that those criticism should be ignored. What was said is that those criticism are invalid, because they are. It is not ad-hominem against Sandifer to point out that Sandifer is trying to insinuate untrue things about Alexander. It is simply observing reality. Sandifer attempts to describe Alexander, Yudkowsky et. al. as supportive of neoreactionary thought. In reality, Alexander, Yudkowsky et. al. are strongly-critical-at-best of neoreactionary thought.
> The leaked e-mails showed he believed there to be "gold" in some of their ideas
This is clutching at straws. Alexander wrote https://slatestarcodex.com/2013/10/20/the-anti-reactionary-f... , in 2013.
You are engaging in the same kind of semantic games that Sandifer does. Please stop.
The party line has shifted comrade. This year with posts on Richard Lynn etc. Scott Alexander is now saying in public the same thing he said in a private email: that he thinks race pseudoscientists and neoreactionaries are brilliant and precious and as many people as possible need to read the best 1% of their ideas. He is no longer pretending that he thinks they have nothing to offer and just has them in his blogroll because ?
The private email from 2014 explained how he hoped people would respond to the anti-neoreactionary FAQ, and his posts this year are 100% consistent with that.
The private email from 2014 explained how he hoped people would respond to the anti-neoreactionary FAQ, and his posts this year are 100% consistent with that.
[deleted]
The "neoreactionary movement" is definitely not dominant
The irony here is the Rationalist community are made up of the ones who weren't observant enough to pick that "identifying as a Rationalist" is generally not a rational decision.
From what I’ve seen it’s a mix of that, some who avoid the issue, and some who do it intentionally even though they don’t really believe it.
I would love a link to anything that could convince me the Rationalist community isn't just a bunch of hoo-haw.
I read the first third of HPMOR. I stopped because I found the writing poor, but more importantly, it didn't "open my mind" to any higher-order way of rationalist thinking. My takeaway was "Yup, the original HP story was full of inconsistencies and stupidities, and you get a different story if the characters were actually smart."
I've read a bunch of EY essays and a lot of lesswrong posts, trying to figure out what is the mind-shattering idea.
* The map is not the territory --> of course it isn't.
* Update your beliefs based on evidence --> who disagrees with this? (with exception on religion)
* People are biased and we need to overcome that --> another obvious statement
* Make decisions based on evidence and towards your desired outcomes --> thanks for the tip?
Seems to me this whole philosophy can be captured in about half page of notes, which most people would nod and say "yup, makes sense."
I read the first third of HPMOR. I stopped because I found the writing poor, but more importantly, it didn't "open my mind" to any higher-order way of rationalist thinking. My takeaway was "Yup, the original HP story was full of inconsistencies and stupidities, and you get a different story if the characters were actually smart."
I've read a bunch of EY essays and a lot of lesswrong posts, trying to figure out what is the mind-shattering idea.
* The map is not the territory --> of course it isn't.
* Update your beliefs based on evidence --> who disagrees with this? (with exception on religion)
* People are biased and we need to overcome that --> another obvious statement
* Make decisions based on evidence and towards your desired outcomes --> thanks for the tip?
Seems to me this whole philosophy can be captured in about half page of notes, which most people would nod and say "yup, makes sense."
Self-help books are useful, not because they have "mind-shattering ideas", but because they collect a lot of self-consistent and reasonably coherent advice into the same place.
In this same way, the rationalist knowledge seeking strategies are not "mind-shattering" but simply reasonable. It presents a set of rules to follow to be more effective in the world around you.
The parts of rationalism that stretch past the half page of notes mainly concern all the downstream conclusions that pop up from this reasonable set of epistemological rules.
In this same way, the rationalist knowledge seeking strategies are not "mind-shattering" but simply reasonable. It presents a set of rules to follow to be more effective in the world around you.
The parts of rationalism that stretch past the half page of notes mainly concern all the downstream conclusions that pop up from this reasonable set of epistemological rules.
> Update your beliefs based on evidence --> who disagrees with this?
Like, dozens of comments in this thread?
For example, people expressing strong opinions on what Effective Altruism is actually about, when https://www.givewell.org/charities/top-charities is just one google search away... but why would anyone bother checking before they post a strong opinion?
The #1 comment says that the rationality community is about "trying to reason about things from first principle", when if fact it is the opposite.
A commenter links a post by Scott Alexander and claims that Scott predicted something and was wrong, when if fact in the linked article Scott says he gives it a probability 30% (which means he gives probability 70% to that thing not happening). Another commenter defends that as a "perfectly reasonable but critical comment".
And hey, compared to most of the internet, HN is the smart place, and the local discussion norms are better than average. But it still doesn't seem like people here actually care about being, uhm, less wrong about things, even ones that are relatively trivial to figure out.
So basically, the mind-shattering idea is to build a community that actually works like that (well, most of the time). Where providing evidence gets upvoted, and unsubstantiated accusations that turn out to be wrong get downvoted, and a few more things like this.
Plus there is the idea of trying to express your beliefs as probabilities, using actual numbers. That's why EY cannot stop talking about the Bayes' Theorem. Yes, people actually do that.
Like, dozens of comments in this thread?
For example, people expressing strong opinions on what Effective Altruism is actually about, when https://www.givewell.org/charities/top-charities is just one google search away... but why would anyone bother checking before they post a strong opinion?
The #1 comment says that the rationality community is about "trying to reason about things from first principle", when if fact it is the opposite.
A commenter links a post by Scott Alexander and claims that Scott predicted something and was wrong, when if fact in the linked article Scott says he gives it a probability 30% (which means he gives probability 70% to that thing not happening). Another commenter defends that as a "perfectly reasonable but critical comment".
And hey, compared to most of the internet, HN is the smart place, and the local discussion norms are better than average. But it still doesn't seem like people here actually care about being, uhm, less wrong about things, even ones that are relatively trivial to figure out.
So basically, the mind-shattering idea is to build a community that actually works like that (well, most of the time). Where providing evidence gets upvoted, and unsubstantiated accusations that turn out to be wrong get downvoted, and a few more things like this.
Plus there is the idea of trying to express your beliefs as probabilities, using actual numbers. That's why EY cannot stop talking about the Bayes' Theorem. Yes, people actually do that.
> The #1 comment says that the rationality community is about "trying to reason about things from first principle", when if fact it is the opposite.
Oh? Eliezer Yudkowsky (the most prominent Rationalist) bragged about how he was able to figure out AI was dangerous (the most stark Rationalist claim) from "the null string as input."[1]
[1] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
Oh? Eliezer Yudkowsky (the most prominent Rationalist) bragged about how he was able to figure out AI was dangerous (the most stark Rationalist claim) from "the null string as input."[1]
[1] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
Well that was a whole thing. I especially liked the existential threat of Cade Metz. But ultimately, I think the great oracle of Chicago got this whole thing right when he said:
-Ism's in my opinion are not good. A person should not believe in an -ism, he should believe in himself. I quote John Lennon, "I don't believe in Beatles, I just believe in me." Good point there. After all, he was the walrus. I could be the walrus. I'd still have to bum rides off people.
-Ism's in my opinion are not good. A person should not believe in an -ism, he should believe in himself. I quote John Lennon, "I don't believe in Beatles, I just believe in me." Good point there. After all, he was the walrus. I could be the walrus. I'd still have to bum rides off people.
> I especially liked the existential threat of Cade Metz.
I am perpetually fascinated by the way rationalists love to dismiss critics by pointing out that they met some people in person and they seemed nice.
It's such a bizarre meme.
Curtis Yarvin went to one of the "Vibecamp" rationalist gatherings, was nice to some prominent Twitter rationalists, and now they are ardent defenders of him on Twitter. Their entire argument is "I met him and he was nice".
It's mind boggling that the rationalist part of their philosophy goes out the window as soon as the lines are drawn between in-group and out-group.
Bringing up Cade Metz is a perennial favorite signal because of how effectively they turned it into a "you're either with us or against us" battle, completely ignoring any valid arguments Cade Metz may have been brought to the table. Then you look at how they treat Neoreactionaries and how we're supposed to look past our disdain for them and focus on the possible good things in their arguments, and you realize maybe this entire movement isn't really about truth-seeking as much as they think it is.
I am perpetually fascinated by the way rationalists love to dismiss critics by pointing out that they met some people in person and they seemed nice.
It's such a bizarre meme.
Curtis Yarvin went to one of the "Vibecamp" rationalist gatherings, was nice to some prominent Twitter rationalists, and now they are ardent defenders of him on Twitter. Their entire argument is "I met him and he was nice".
It's mind boggling that the rationalist part of their philosophy goes out the window as soon as the lines are drawn between in-group and out-group.
Bringing up Cade Metz is a perennial favorite signal because of how effectively they turned it into a "you're either with us or against us" battle, completely ignoring any valid arguments Cade Metz may have been brought to the table. Then you look at how they treat Neoreactionaries and how we're supposed to look past our disdain for them and focus on the possible good things in their arguments, and you realize maybe this entire movement isn't really about truth-seeking as much as they think it is.
I'm hyperfixated on the idea that they're angry at Metz for "outing" Scott Alexander, who published some of his best-known posts under his own name.
Isn't "Scott Alexander" his first and middle name? Did he actually use his last name on his blog before the NYT published that piece?
Yes.
[Citation needed]
Late to respond, but here's one such link: https://web.archive.org/web/20130118213127/https://www.overc...
Title of the post is "Hail Scott Siskind" from Robin Hanson's blog Overcoming Bias. This blog was a central point of the early rationalist movement.
Notably, posts like this were scrubbed some time near the New York Times debacle. There was an active effort to scrub mentions of "Siskind" from rationalist adjacent blogs around that time.
It was amazingly effective at rewriting the history. You won't find any mention of the activities to scrub "Siskind" from the rationalist blogs, though.
Title of the post is "Hail Scott Siskind" from Robin Hanson's blog Overcoming Bias. This blog was a central point of the early rationalist movement.
Notably, posts like this were scrubbed some time near the New York Times debacle. There was an active effort to scrub mentions of "Siskind" from rationalist adjacent blogs around that time.
It was amazingly effective at rewriting the history. You won't find any mention of the activities to scrub "Siskind" from the rationalist blogs, though.
Tricky to do without giving you an archive link of a post with his last name on it, which I'm disinclined to do. Here, in this community, with this community's norms, I'm happy to oblige him. But here is not the real world.
Come on; Scott Alexander Siskind now has his real name published in New York Times, RationalWiki, and Wikipedia, since early 2021. You can find this all on the first page of Google results (exactly as he predicted). He even mentioned his name in the first article on his Substack blog, but then returned to the pen name his readers are more familiar with.
And he has already lost the job he was trying to protect back then. Now AFAIK his main source of income are subscriptions on Substack, and obviously those are not threatened in any way by having his name mentioned one more time.
Please let's respect each other's intelligence by not playing the game of pretending that posting Scott's full name before 2021 is the same as posting Scott's full name after 2021.
So, do you have examples of "some of his best-known posts published under his own name" before 2021?
And he has already lost the job he was trying to protect back then. Now AFAIK his main source of income are subscriptions on Substack, and obviously those are not threatened in any way by having his name mentioned one more time.
Please let's respect each other's intelligence by not playing the game of pretending that posting Scott's full name before 2021 is the same as posting Scott's full name after 2021.
So, do you have examples of "some of his best-known posts published under his own name" before 2021?
Yes. This isn't a controversial point. It was very much part of the conversation at the time; just go read the threads.
I checked using Google search, and Scott's full name does not appear even once on https://slatestarcodex.com , his pre-2021 blog.
Is this information surprising for you?
Is this information surprising for you?
He had a blog before Slate Star Codex. The impression I have is that I know more about this than you, and I'm not invested in this discussion enough to educate you adversarially, so I think it makes the most sense to just leave this subthread where it lies.
Your persistent refusal to acknowledge that Scott did not want to go from a world where his patients Googling his real name did _not_ immediately lead to his blog, to a world where it _did_ immediately lead to his blog, and that was his primary (and valid) objection to having his real first and last name put into print, is baffling.
My response to that is: it's good to want things. He doesn't get to ask the world to forget his name. I actually talked to therapists when this whole story broke out, and none of them said this "patients must not be able to Google your blog" thing was an actual thing. People just believe it because they like Scott Alexander and believe whatever he tells them.
> My response to that is: it's good to want things. He doesn't get to ask the world to forget his name.
Is this anything other than a naked assertion of force, that might makes right? He couldn't stop it, therefore it's fine that it happened to him? (Also, it's extremely... something... to describe "he asked a journalist to not print his name on the front page of the NYT" as "asking the world to forget his name", as if his real name was already the primary referrent by which he wielded his influence, and he wanted to shield that power from scrutiny. This was obviously not the case, and is _still_ not the case despite the article, which is why nobody has actually made a compelling argument for why including his name in the article was _good_ rather than _something Cade Metz had the power to do_. I in fact don't particularly think that Cade Metz did it to deliberately hurt Scott, I just think he's a blankface who didn't care that his usual modus operandi would sometimes hurt people for no good reason and was unable to step out of his frame enough to actually check whether what he was doing made any sense, in that instance.)
> I actually talked to therapists when this whole story broke out, and none of them said this "patients must not be able to Google your blog" thing was an actual thing. People just believe it because they like Scott Alexander and believe whatever he tells them.
That you describe it as "patients must not be able to Google your blog" makes me not particularly trust the reports of those therapists. I, too, talked to some therapists, who thought that Scott's concerns were reasonable. Not that there was an overriding professional duty, sure, but that wasn't the claim, either. I dunno, man. The attitude you have towards this really seems like, "well, getting slapped isn't that bad, and you're not strong enough to stop him... maybe stop complaining?" What good thing happened when Cade Metz put his name in print? If you want to adopt a principled stance against pseudonymous writing online, do that. But don't pretend that Scott's failure to keep a pristine separation between his real name and his entire history of online writing somehow makes it so that the NYT printing his name is merely the maintaining the status quo ("ask the world to forget his name"), rather than dramatically expanding the circle of people for whom his identity was deanonymized.
Is this anything other than a naked assertion of force, that might makes right? He couldn't stop it, therefore it's fine that it happened to him? (Also, it's extremely... something... to describe "he asked a journalist to not print his name on the front page of the NYT" as "asking the world to forget his name", as if his real name was already the primary referrent by which he wielded his influence, and he wanted to shield that power from scrutiny. This was obviously not the case, and is _still_ not the case despite the article, which is why nobody has actually made a compelling argument for why including his name in the article was _good_ rather than _something Cade Metz had the power to do_. I in fact don't particularly think that Cade Metz did it to deliberately hurt Scott, I just think he's a blankface who didn't care that his usual modus operandi would sometimes hurt people for no good reason and was unable to step out of his frame enough to actually check whether what he was doing made any sense, in that instance.)
> I actually talked to therapists when this whole story broke out, and none of them said this "patients must not be able to Google your blog" thing was an actual thing. People just believe it because they like Scott Alexander and believe whatever he tells them.
That you describe it as "patients must not be able to Google your blog" makes me not particularly trust the reports of those therapists. I, too, talked to some therapists, who thought that Scott's concerns were reasonable. Not that there was an overriding professional duty, sure, but that wasn't the claim, either. I dunno, man. The attitude you have towards this really seems like, "well, getting slapped isn't that bad, and you're not strong enough to stop him... maybe stop complaining?" What good thing happened when Cade Metz put his name in print? If you want to adopt a principled stance against pseudonymous writing online, do that. But don't pretend that Scott's failure to keep a pristine separation between his real name and his entire history of online writing somehow makes it so that the NYT printing his name is merely the maintaining the status quo ("ask the world to forget his name"), rather than dramatically expanding the circle of people for whom his identity was deanonymized.
If it's an assertion of force, it's by Scott against the reporter, who needed only Google to find Scott's name. People don't get to demand that reporters un-know common knowledge just because publicity is unwelcome.
His reaction to this incident as being a sort of targeted hit piece towards him specifically was utter paranoid nonsense and just really made him look full of shit from an outside perspective. Further, authors of the piece received countless death threats from a mob of angry redditors with no real insight other than "someone got doxxed?!?! HOW DARE THEY!!!" (a narrative which I personally fell victim to before reading the actual article). Scott did nothing to remedy this situation and perhaps even fanned the flames a good deal.
More people should read the article, if anything because it provides interesting insight into Sam Altman's shady behavior.
https://www.nytimes.com/2021/02/13/technology/slate-star-cod...
More people should read the article, if anything because it provides interesting insight into Sam Altman's shady behavior.
https://www.nytimes.com/2021/02/13/technology/slate-star-cod...
Maybe mr metz could have written under a pseudonym in order to avoid the very serious threats on his life
Have you ever had random people harass you and threaten violence towards you in heavy numbers? No amount of resolve will keep that from being a deeply stressful situation.
I don't doubt it's not fun, but I also don't think that either you or Mr Metz really give a shit. Rather, you like to bandy about this harassment as proof that your opponents are evil (of course, when it's your opponents being harassed, then that's a sort of divine punishment, and hence also proof that they're evil). If it were any other way, then you wouldn't particularly bring it up in reference to scott, since there isn't much of anything scott can do about other people committing harassment. Furthermore, you might express a degree of sympathy for his efforts to avoid harassment by not having his dox published in the nyt, rather than indignantly expounding on your god given right to publish whatever you feel like.
> Ism's in my opinion are not good. A person should not believe in an -ism, he should believe in himself
There's an -ism for that.
Actually, a few different ones depending on the exact angle you look at the it from: solipsism, narcissism,...
There's an -ism for that.
Actually, a few different ones depending on the exact angle you look at the it from: solipsism, narcissism,...
> There's an -ism for that
It's Buddhism.
https://en.wikipedia.org/wiki/Anattā
> Actually, a few different ones depending on the exact angle you look at the it from: solipsism, narcissism,...
That is indeed a problem with it. The Buddhist solution is to make you promise not to do that.
https://en.wikipedia.org/wiki/Bodhicitta
And the (well, a) term for the entire problem is "non-dual awareness".
It's Buddhism.
https://en.wikipedia.org/wiki/Anattā
> Actually, a few different ones depending on the exact angle you look at the it from: solipsism, narcissism,...
That is indeed a problem with it. The Buddhist solution is to make you promise not to do that.
https://en.wikipedia.org/wiki/Bodhicitta
And the (well, a) term for the entire problem is "non-dual awareness".
Ideology is at its most powerful when people think it doesn't exist.
I genuinely worry that the Rationalists are actually Fascist Anarchists who don’t own cars.
I once was interested in a woman who was really into the effective altruism/rationalism crowd. I went to a few meetings with her but my inner contrarian didn't like it.
Took me a few years to realize how cultish it all felt and that I am somewhat happy my edgy atheist contrarian personality overwrote my dicks thinking with that crowd.
Took me a few years to realize how cultish it all felt and that I am somewhat happy my edgy atheist contrarian personality overwrote my dicks thinking with that crowd.
I feel like I'm witnessing something that Adam Curtis would cover in the last part of The Century of Self, in real time.
There was always an underlying Randian impulse to the EA crowd - as if we could solve any issue if we just get the right minds onto tackling the problem. The black-and-white thinking, group think, hero worship and charicaturist literature are all there.
I always wondered is it her direct influence, or is it just that those characteristics naturally "go together".
Some of the comments here remind me of online commentary about some place called "the orange site". Always wondered who they were talking about...
Honestly, as a daily reader and oftentimes commenter on the dreaded orange site, I've seen a lot of criticism of its denizens, very often along the lines of -
"Computer people who think that because they're smart in one area they have useful opinions on anything else, holding forth with great certainty about stuff they have zero undertanding or insight into"
And you know what, I think they're right. The rest of you are always doing that sort of thing!
(/s, if it's necessary...)
"Computer people who think that because they're smart in one area they have useful opinions on anything else, holding forth with great certainty about stuff they have zero undertanding or insight into"
And you know what, I think they're right. The rest of you are always doing that sort of thing!
(/s, if it's necessary...)
Can't stand that place. Those people are all so sure that they're right about everything.
He already had a rationalist “coming out” like ages ago. Dude just make up your mind
https://scottaaronson.blog/?p=2537
https://scottaaronson.blog/?p=2537
While this was an interesting and enjoyable read, it doesn't seem to be a “rationalist ‘coming out’”. On the contrary, he's just saying he would have liked going to a ‘rationalist’ meeting.
The last paragraph discusses how he's resisted the label and then he closes with “the rationalists have walked the walk and rationaled the rational, and thus they’ve given me no choice but to stand up and be counted as one of them.”
He’s clearly identifying as a rationalist there
He’s clearly identifying as a rationalist there
They call themselves rationalist, yet they don't have very rational opinions if you ask them about scientific racism [1]
[1] https://www.astralcodexten.com/p/how-to-stop-worrying-and-le...
[1] https://www.astralcodexten.com/p/how-to-stop-worrying-and-le...
I am not sure precisely it not very rational about that link. Did you have a specific point you were trying to make with it?
Yes, that they're not "rational".
If you take a look at the biodiversity survey here https://reflectivealtruism.com/2024/12/27/human-biodiversity...
1/3 of the users at acx actually support flawed scientific theories that would explain iq on a scientific basis. The Lynn study on iq is also quite flawed https://en.m.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations
If you want to read about human biodiversity, https://en.m.wikipedia.org/wiki/Human_Biodiversity_Institute
As I said, it's not very rational of them to support such theories. And of course as you scratch the surface, it's the old 20th century racist theories, and of course those theories are supported by (mostly white men, if I had to guess) people claiming to be rational
If you take a look at the biodiversity survey here https://reflectivealtruism.com/2024/12/27/human-biodiversity...
1/3 of the users at acx actually support flawed scientific theories that would explain iq on a scientific basis. The Lynn study on iq is also quite flawed https://en.m.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations
If you want to read about human biodiversity, https://en.m.wikipedia.org/wiki/Human_Biodiversity_Institute
As I said, it's not very rational of them to support such theories. And of course as you scratch the surface, it's the old 20th century racist theories, and of course those theories are supported by (mostly white men, if I had to guess) people claiming to be rational
Human ethnic groups are measurably different in genetic terms, as based on single nucleotide polymorphisms and allelic frequency. There are multiple PCA plots of the 1000 Genomes dataset which show clear cluster separation based on ancestry:
https://www.researchgate.net/figure/Example-Ancestry-PCA-plo...
We know ethnic groups vary in terms of height, hair color, eye color, melanin, bone density, sprinting ability, lactose tolerance, propensity to diseases like sickle cell anemia, Tay-Sachs, stomach cancer, alcoholism risk, etc. Certain medications need to be dosed differently for different ethnic groups due to the frequency of certain gene variants, e.g. Carbamazepine, Warfarin, Allopurinol.
The fixation index (Fst) quantifies the level of genetic variation between groups, a value of 0 means no differentiation, and 1 is maximal. A 2012 study based on SNPs found that Finns and Swedes have a Fst value of 0.0050-0.0110, Chinese and Europeans at 0.110, and Japanese and Yoruba at 0.190.
https://pmc.ncbi.nlm.nih.gov/articles/PMC2675054/
A 1994 study based on 120 alleles found the two most distant groups were Mbuti pygmies and Papua New Guineans at a Fst of 0.4573.
https://en.wikipedia.org/wiki/File:Full_Fst_Average.png
In genome wide association studies, polygenic score have been developed to find thousands of gene variants linked to phenotypes like spatial and verbal intelligence, memory, and processing speed. The distribution of these gene variants is not uniform across ethnic groups.
Given that we know there are genetic differences between groups, and observable variation, it stands to reason that there could be a genetic component for variation in intelligence between groups. It would be dogmatic to a priori claim there is absolutely no genetic component, and pretty obviously motivated out of the fear that inequality is much more intractable than commonly believed.
https://www.researchgate.net/figure/Example-Ancestry-PCA-plo...
We know ethnic groups vary in terms of height, hair color, eye color, melanin, bone density, sprinting ability, lactose tolerance, propensity to diseases like sickle cell anemia, Tay-Sachs, stomach cancer, alcoholism risk, etc. Certain medications need to be dosed differently for different ethnic groups due to the frequency of certain gene variants, e.g. Carbamazepine, Warfarin, Allopurinol.
The fixation index (Fst) quantifies the level of genetic variation between groups, a value of 0 means no differentiation, and 1 is maximal. A 2012 study based on SNPs found that Finns and Swedes have a Fst value of 0.0050-0.0110, Chinese and Europeans at 0.110, and Japanese and Yoruba at 0.190.
https://pmc.ncbi.nlm.nih.gov/articles/PMC2675054/
A 1994 study based on 120 alleles found the two most distant groups were Mbuti pygmies and Papua New Guineans at a Fst of 0.4573.
https://en.wikipedia.org/wiki/File:Full_Fst_Average.png
In genome wide association studies, polygenic score have been developed to find thousands of gene variants linked to phenotypes like spatial and verbal intelligence, memory, and processing speed. The distribution of these gene variants is not uniform across ethnic groups.
Given that we know there are genetic differences between groups, and observable variation, it stands to reason that there could be a genetic component for variation in intelligence between groups. It would be dogmatic to a priori claim there is absolutely no genetic component, and pretty obviously motivated out of the fear that inequality is much more intractable than commonly believed.
…but that just sounds like scientific racism.
Rather than judging an individual on their actual intelligence, these kinds of statistical trends allow you to justify judging an individual based on their race, because you feel you can credibly claim that race is an acceptable proxy for their genome, is an acceptable proxy for their intelligence.
Or for their trustworthiness, or creativity, or sexuality, or dutifulness, or compassion, or aggressiveness, or alacrity, or humility, etc etc.
When you treat a person like a people, that’s still prejudice.
Rather than judging an individual on their actual intelligence, these kinds of statistical trends allow you to justify judging an individual based on their race, because you feel you can credibly claim that race is an acceptable proxy for their genome, is an acceptable proxy for their intelligence.
Or for their trustworthiness, or creativity, or sexuality, or dutifulness, or compassion, or aggressiveness, or alacrity, or humility, etc etc.
When you treat a person like a people, that’s still prejudice.
I don't know what you're trying to say. That there is no race IQ difference? That we should not determine if there is one? How does one act on the feelings you've expressed?
Well, the rational thing is obviously to be scared of what ideas sound like.
> Rather than judging an individual on their actual intelligence
Actual intelligence is hard to know! However, lots of factors allow you to make a rapid initial estimate of their actual intelligence, which you can then refine as required.
(When the factors include apparent genetic heritage, this is called "racism" and society doesn't like it. But that doesn't mean it doesn't work, just that you can get fired and banned for doing it.)
((This is of course why we must allow IQ tests for hiring; then there's no need to pay attention to skin color, so liberals should be all for it.))
> Rather than judging an individual on their actual intelligence
Actual intelligence is hard to know! However, lots of factors allow you to make a rapid initial estimate of their actual intelligence, which you can then refine as required.
(When the factors include apparent genetic heritage, this is called "racism" and society doesn't like it. But that doesn't mean it doesn't work, just that you can get fired and banned for doing it.)
((This is of course why we must allow IQ tests for hiring; then there's no need to pay attention to skin color, so liberals should be all for it.))
> Well, the rational thing is obviously to be scared of what ideas sound like.
Yes, actually. If an idea sounds like it can be used to commit crimes against humanity, you should pause. You should reassess said idea multiple times. You should be skeptical. You shouldn't ignore that feeling.
What a lot of people are missing is intent - the human element. Why were these studies conducted? Who conducted them?
If someone insane conducts a study then yes - that is absolutely grounds to be skeptical of said study. It's perfectly rationale. If extremely racist people produce studies which just so happen to be racist, we should take a step back and go "hmm".
Being right or being correct is one thing, but it's not absolutely valuable. The end-result and how "bad" it is also matters, and often times it matters more. And, elephant in the room, nobody actually knows if they're right. Making logical conclusions isn't so, because you are forced to make thousands of assumptions.
You might be right, you might not be. Let's all have some humility.
Yes, actually. If an idea sounds like it can be used to commit crimes against humanity, you should pause. You should reassess said idea multiple times. You should be skeptical. You shouldn't ignore that feeling.
What a lot of people are missing is intent - the human element. Why were these studies conducted? Who conducted them?
If someone insane conducts a study then yes - that is absolutely grounds to be skeptical of said study. It's perfectly rationale. If extremely racist people produce studies which just so happen to be racist, we should take a step back and go "hmm".
Being right or being correct is one thing, but it's not absolutely valuable. The end-result and how "bad" it is also matters, and often times it matters more. And, elephant in the room, nobody actually knows if they're right. Making logical conclusions isn't so, because you are forced to make thousands of assumptions.
You might be right, you might not be. Let's all have some humility.
The assertion that "actual intelligence is hard to know" followed almost immediately by "apparent genetic heritage" is what's wrong with your opinion. And no, it doesn't work -- at least it doesn't work for identifying intelligence. It just works for selecting people who appeal to your bias.
IQ tests are not actual measurements of anything; this is both because nobody has a rigorous working definition of intelligence and because nobody's figured out a universal method of measuring achievement of what insufficient definitions we have. Their proponents are more interested in pigeonholing people than actually measuring anything anyway.
And as a hiring manager, I'd hire an idiot who is good at the job over a genius who isn't.
IQ tests are not actual measurements of anything; this is both because nobody has a rigorous working definition of intelligence and because nobody's figured out a universal method of measuring achievement of what insufficient definitions we have. Their proponents are more interested in pigeonholing people than actually measuring anything anyway.
And as a hiring manager, I'd hire an idiot who is good at the job over a genius who isn't.
Actual intelligence is hard to know precisely/accurately/confidently, is what I meant, my bad. It's not hard to place a guess, and there's nothing wrong with it either so long as you're open for refinement. Your brain does it automatically in the first few seconds of seeing somebody, you can't even prevent it, and skin color is by no means the largest factor in it.
IQ as a metric is correlated with almost every life outcome. It's one of the most reliable metrics in psychology.
As a hiring manager, if you think an idiot can be good at the job, you either hire for an undemanding job or I'm not sure if you're good at yours.
IQ as a metric is correlated with almost every life outcome. It's one of the most reliable metrics in psychology.
As a hiring manager, if you think an idiot can be good at the job, you either hire for an undemanding job or I'm not sure if you're good at yours.
That multiple correlation is just more evidence the test is flawed. No qualified psychologist would call it reliable, and that's been the case for some years now.
Not all work is knowledge work. You might want to broaden your horizons.
Not all work is knowledge work. You might want to broaden your horizons.
SES, personality, and education are also correlated with life outcomes. Craniometry used to be one of the most reliable metrics in psychology.
Sure, I'm not saying IQ is everything. Do you have articles on craniometry actually being validated by studies? That sounds interesting.
SES, education, and personality are intimately interrelated with "IQ". Once you control for those, what's the leftover signal, and how do you know?
I mean, if IQ is real then SES and education are plausibly strongly caused by it. In that case, controlling for SES and education and then saying "there's not much signal left, thus IQ is bunk" just means that you've basically renamed IQ into "the implicit thing that determines SES and education, which I will not calculate directly." With such related properties it becomes hard to determine causation in general. The obvious test would be to improve education and see if IQ shifts. I don't have a study for this on hand, but I expect (just to register my prediction, not to make an argument) that if I look I'll find that the signal is weak that direction.
Your treatment of IQ is ridiculous. Give me access to a child for seven months, and I can increase their IQ score by 20 (and probably make myself an enemy in the process: IQ test drills are one of the dullest activities, since you can't even switch your brain off while doing them).
Intelligence is not a single axis thing. IQ test results are significantly influenced by socioeconomic factors. "Actual intelligence is hard to know" because it doesn't exist.
I have never yet known scientific racism to produce true results. I have known a lot of people to say the sorts of things you're saying: evidence-free claims that racism is fine so long as you're doing the Good Racism that Actually Works™, I Promise, This Time It's Not Prejudice Because It's Justified®.
No candidate genetic correlate of the g factor has ever replicated. That should be a massive flashing warning sign that – rather than having identified an elusive fact about reality that just so happens to not appear in any rigorous study – maybe you're falling afoul of the same in-group/out-group bias as nearly every group of humans since records begin.
Since I have no reason to believe your heuristic is accurate, we can stop there. However, to further underline that you're not thinking rationally: even if blue people were (on average) 2× as capable at spacial rotation-based office jobs than green people, it still wouldn't be a good idea to start with the skin colour prior and update from there, because that would lead to the creation of caste systems, which hinder social mobility. Even if scientific racism worked (which it hasn't to date!), the rational approach would still be to judge people on their own merits.
If you find it hard to assess the competence of your subordinates, to the point where you're resorting to population-level stereotypes to make hiring decisions, you're an incompetent manager and should find another job.
Intelligence is not a single axis thing. IQ test results are significantly influenced by socioeconomic factors. "Actual intelligence is hard to know" because it doesn't exist.
I have never yet known scientific racism to produce true results. I have known a lot of people to say the sorts of things you're saying: evidence-free claims that racism is fine so long as you're doing the Good Racism that Actually Works™, I Promise, This Time It's Not Prejudice Because It's Justified®.
No candidate genetic correlate of the g factor has ever replicated. That should be a massive flashing warning sign that – rather than having identified an elusive fact about reality that just so happens to not appear in any rigorous study – maybe you're falling afoul of the same in-group/out-group bias as nearly every group of humans since records begin.
Since I have no reason to believe your heuristic is accurate, we can stop there. However, to further underline that you're not thinking rationally: even if blue people were (on average) 2× as capable at spacial rotation-based office jobs than green people, it still wouldn't be a good idea to start with the skin colour prior and update from there, because that would lead to the creation of caste systems, which hinder social mobility. Even if scientific racism worked (which it hasn't to date!), the rational approach would still be to judge people on their own merits.
If you find it hard to assess the competence of your subordinates, to the point where you're resorting to population-level stereotypes to make hiring decisions, you're an incompetent manager and should find another job.
> Your treatment of IQ is ridiculous. Give me access to a child for seven months, and I can increase their IQ score by 20
That would be remarkable! Do you have a write-up/preprint on your protocol?
That would be remarkable! Do you have a write-up/preprint on your protocol?
There's nothing remarkable about it:
• Drill.
• Goto step 1.
Does this make the child "more intelligent"? Not in any meaningful way! But they get better at IQ tests.
It's a fairly common protocol. I can hardly be said to have invented it: I was put through it. (Sure, I came up with a few tricks for solving IQ-type problems that weren't in the instruction books, but those tricks too can be taught.)
I really don't understand why people think IQ test results are meaningful. They're among the most obvious cases of Goodhart's law that I know. Make up a sport that most kids won't have practised before, measure performance, and probably that's about as correlated with the (fictitious) "g factor" as IQ tests are.
• Drill.
• Goto step 1.
Does this make the child "more intelligent"? Not in any meaningful way! But they get better at IQ tests.
It's a fairly common protocol. I can hardly be said to have invented it: I was put through it. (Sure, I came up with a few tricks for solving IQ-type problems that weren't in the instruction books, but those tricks too can be taught.)
I really don't understand why people think IQ test results are meaningful. They're among the most obvious cases of Goodhart's law that I know. Make up a sport that most kids won't have practised before, measure performance, and probably that's about as correlated with the (fictitious) "g factor" as IQ tests are.
I mean, have you actually done this and have you documented your results? Preferably with a placebo and double blind?
The problem with "I've gone through this" is it's hard to analyze the counterfactual.
The problem with "I've gone through this" is it's hard to analyze the counterfactual.
I'm not sure how you'd run a double-blind experiment on this. You can single-blind the experimenters, but the participants are always going to know whether they've been drilling IQ tests.
Your point about counterfactuals is good, but… subjectively, I ended up with a better understanding of IQ test genre conventions (which is also why I bang on so much about "culturally-specific": they really are). My speed at solving the problems doubled or tripled, and my accuracy went from 80%-ish to near 100%. This did not translate to any improvements to my real-life skill at anything (although, I suppose it might've generalised a bit to other multiple-choice exams). I've got a lot more evidence to analyse than just an n=1 scatter plot.
Your point about counterfactuals is good, but… subjectively, I ended up with a better understanding of IQ test genre conventions (which is also why I bang on so much about "culturally-specific": they really are). My speed at solving the problems doubled or tripled, and my accuracy went from 80%-ish to near 100%. This did not translate to any improvements to my real-life skill at anything (although, I suppose it might've generalised a bit to other multiple-choice exams). I've got a lot more evidence to analyse than just an n=1 scatter plot.
> the participants are always going to know whether they've been drilling IQ tests.
I'm not asking you to actually do this, but the participants (and experimenters!) don't necessarily have to know what you're testing. Maybe get one to drill IQ tests, one to drill Latin, one to drill chess and one to drill the piano.
Does your ability extend to IQ tests with other patterns? Also, does it extend to logic puzzles?
I'm not asking you to actually do this, but the participants (and experimenters!) don't necessarily have to know what you're testing. Maybe get one to drill IQ tests, one to drill Latin, one to drill chess and one to drill the piano.
Does your ability extend to IQ tests with other patterns? Also, does it extend to logic puzzles?
From time to time I see a press release to the effect that some movie star (Alyssa Milano was one) got an IQ score around the max of Raven's Progressive Matrices. I bet somewhere there is a psychologist who will coach you on it, across the street one will test you, and on the next block a PR agency that will make the press release.
I mean, there aren't that many questions on Raven, you could memorize them all, particularly if you've got the kind of intelligence that actors have -- being able to memorize your lines. (And that's something, I have a 1950-ish book about the television industry that makes a point that people expect performers to be a "quick study", you'd better know your lines really well and not have to be told twice that you are expected to do this or that. That's different from, say, being able to solve really complex math problems.)
I mean, there aren't that many questions on Raven, you could memorize them all, particularly if you've got the kind of intelligence that actors have -- being able to memorize your lines. (And that's something, I have a 1950-ish book about the television industry that makes a point that people expect performers to be a "quick study", you'd better know your lines really well and not have to be told twice that you are expected to do this or that. That's different from, say, being able to solve really complex math problems.)
Isn't the point of Raven that you can synthesize variants at will? Or am I misunderstanding?
I'd consider it well plausible that top movie stars are also very smart.
I'd consider it well plausible that top movie stars are also very smart.
We do allow IQ tests for hiring. That we don't is a super-pernicious online myth. Some of the largest companies in America, with extremely deep pockets for employment law plaintiffs attorneys to come after, do in fact use IQ tests. Most companies don't, though, because they don't work.
Huh. Interesting, thank you, I didn't know that!
(The "they don't work" thing is my editorial gloss; the rest of it though is I think objectively verifiable.)
Um, I asked Grok and aren't IQ tests soft-outlawed under disparate impact? As opposed to skill qualifications, it's hard to prove that, say, you need to be "intelligent" to work at a certain job. You would end up having to argue that intelligence even exists in court, which I can see why companies would try to avoid- ie. disparate impact creates a reversed weight of evidence. If your test has disparate impact, you now have to prove that it's necessary, creating a chicken-and-egg problem.
Nope. It's trivially easy to pull up (a small minority of) Fortune 500 companies that use IQ tests in their hiring processes. The companies that offer these tests brag about the companies that use them. Whatever Grok thinks about this doesn't really much matter in the face of that evidence.
I think they use "aptitude tests" or "personality tests" that are at least packaged up to look relevant to the job, not direct naked IQ tests? I can't offhand find companies using actual IQ tests in hiring.
And if you're saying "well those are just repackaged IQ tests, so doesn't it count", then 1. it sure seems like IQ tests are illegal then, but 2. it also seems like they're so useful that companies are trying to smuggle them in anyway?
And if you're saying "well those are just repackaged IQ tests, so doesn't it count", then 1. it sure seems like IQ tests are illegal then, but 2. it also seems like they're so useful that companies are trying to smuggle them in anyway?
>> That we don't is a super-pernicious online myth.
> Um, I asked Grok and
Where does Grok's training set come from?
> Um, I asked Grok and
Where does Grok's training set come from?
>Some of the largest companies in America . . . do in fact use IQ tests.
Can you name one, please?
I saw your claim in this thread that the companies that make money by supplying these test to employers brag about how many large employers use the tests, but plenty of people brag falsely when bragging will tend to increase their revenue.
Can you name one, please?
I saw your claim in this thread that the companies that make money by supplying these test to employers brag about how many large employers use the tests, but plenty of people brag falsely when bragging will tend to increase their revenue.
You can literally just go to Wonderlic and look at the logo crawl on their front page. It's not a secret.
GWAS results are so inhospitable to race/IQ talking points that race/IQ people have for the last several years been slagging GWAS, and molecular genomics more generally, and saying the twin studies work was the right approach all along. It's a whole thing.
As we all know, genetics and evolution only apply from the neck down.
True, nature is egalitarian although only intracranialy.
The Lynn IQ data isn't so much "flawed" as it is "fraudulent". There has never been a global comparative study of IQ; Lynn exploited the fact that IQ is a common diagnostic tool, even in places that don't fund a lot of social science research and thus don't conduct their own internal cross-sectional studies, and so ended up comparing diagnostic tests done at hospitals in the developing world with research studies done of volunteers in the developed world.
Nothing about the article you posted in your first comment seems racist. You could argue that believing in the conclusions of Richard Lynn’s work makes someone racist, but to support that claim, you’d need to show that those who believe it do so out of willful ignorance of evidence that his science is flawed.
Scott itself makes a point of the study being debated. It's not. It's not debated. It's pseudo science,or "science" made with so many questionable points that it's hard to call it "science". He links to a magazine article written by a researcher that has been fired, not surprisingly, for his pseudo scientific stances on racism https://en.m.wikipedia.org/wiki/Noah_Carl
Saying in 2025 that the study is still debated is not only racist, but dishonest as well. It's not debated, it's junk
Saying in 2025 that the study is still debated is not only racist, but dishonest as well. It's not debated, it's junk
It is debated: just not by serious scholars or academics. (Which doesn't necessarily make it wrong; but "scientific racism is bunk, and its proponents are persuasive" is a model whose high predictive power has served me well, so I believe it's wrong regardless.)
A lot of "rationalists" of this kind are very poorly informed about statistical methodology, a condition they inherit from reading papers written in these pseudoscientific fields about people likewise very poorly informed.
This is a pathology that has not really been addressed in the large, anywhere, really. Very few in the applied sciences who understand statistical methodology, "leave their areas" -- and many areas that require it, would disappear if it entered.
This is a pathology that has not really been addressed in the large, anywhere, really. Very few in the applied sciences who understand statistical methodology, "leave their areas" -- and many areas that require it, would disappear if it entered.
I don’t know where you get this idea, at least about Scott. He has written extensively about flaws in science and misuse of statistics. My favorite is the one where he cries at the end “ Science! YOU WERE THE CHOSEN ONE! It was said that you would destroy reliance on biased experts, not join them!”
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
You're correct to observe that SSC raises many issues with statistical methodology. But the whole field psychometrics, and its "Rationalist" fans, proceed as if they havent read: https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-rev...
I agree, I had to read things for an ethics course in IT in uni that read more like science fiction than actual science. Anyway, my point is that it feels pretentious - very pretentious, and I'm being kind with words - to support such pseudo scientific theories and call itself rationalist. Especially when these teories can be debunked just by reading the related Wikipedia page
More charitably, it is really, really hard to tell the difference between a crank kicked out of a field for being a crank, and an earnest researcher being persecuted for not towing the political line, without being an expert in the field in question and familiar with the power structures involved.
A lot of people who like to think of themselves as skeptical could also be categorized as contrarian -- they are skeptical of institutions, and if someone is outside an institution, that automatically gives them a certain credibility.
There are three or four logical fallacies in the mix, and if you throw in confirmation bias because what the one side says appeals to your own prior beliefs, it is really, really easy to convince yourself that you're the steely-eyed rationalist perceiving the world correctly while everyone else is deluded by their biases.
A lot of people who like to think of themselves as skeptical could also be categorized as contrarian -- they are skeptical of institutions, and if someone is outside an institution, that automatically gives them a certain credibility.
There are three or four logical fallacies in the mix, and if you throw in confirmation bias because what the one side says appeals to your own prior beliefs, it is really, really easy to convince yourself that you're the steely-eyed rationalist perceiving the world correctly while everyone else is deluded by their biases.
In that essay Scott Alexander more or less says "so Richard Lynn made up numbers about how stupid black and brown people are, but we all know he was right if those mean scientists just let us collect the data to prove it." The level of thinking most of us moved past in high school, and he is a MD who sees himself as a Public Intellectual! More evidence that thinking too much about IQ makes people stupid.
[deleted]
These kinds of propositions are determined by history, not by declaration.
Espouse your beliefs, participate in certain circles if you want, but avoid labels unless you intend to do ideological battle with other label-bearers.
Espouse your beliefs, participate in certain circles if you want, but avoid labels unless you intend to do ideological battle with other label-bearers.
Bleh, labels can be restrictive, but guess what labels can also be? Useful.
>These kinds of propositions are determined by history, not by declaration.
A single failed prediction should revoke the label.
The ideal rational person should be pyrrhonian skeptic, or at a minimum a bayesian epistemologist.
A single failed prediction should revoke the label.
The ideal rational person should be pyrrhonian skeptic, or at a minimum a bayesian epistemologist.
[deleted]
Just to confirm, this is about:
https://en.wikipedia.org/wiki/Rationalist_community
and not:
https://en.wikipedia.org/wiki/Rationalism
right?
https://en.wikipedia.org/wiki/Rationalist_community
and not:
https://en.wikipedia.org/wiki/Rationalism
right?
Absolutely everybody names it wrong. The movement is called rationality or "LessWrong-style rationality", explicitly to differentiate it from rationalism the philosophy; rationality is actually in the empirical tradition.
But the words are too close together, so this is about as lost a battle as "hacker".
But the words are too close together, so this is about as lost a battle as "hacker".
I don't think "rationality" is a good name for the movement, for the same reason as I wish "effective altruism" had picked a different name: it conflates the goal with the achievement of the goal. A rationalist (in the Yudkowsky sense) is someone who is trying to be rational, in a particular way. But "rationality" means actually being rational.
I don't think it's actually true that rationalists-in-this-sense commonly use "rationality" to refer to the movement, though they do often use it to refer to what the movement is trying to do.
I don't think it's actually true that rationalists-in-this-sense commonly use "rationality" to refer to the movement, though they do often use it to refer to what the movement is trying to do.
Yeah, that's why they famously say they're an "aspiring rationalist." But I don't think there's anything wrong with setting a target, even if it's unreachable.
No, absolutely nothing wrong with setting ambitious and unreachable targets. Reach for the stars and you might at least get to the moon, etc. But if you say "I'm an effective altruist" then you are claiming to actually be effective and if you say "our movement is Rationality" then you are claiming to be paradigmatic examples of actually being rational, and I think that's a bad idea because it should be possible to say "I'm part of such-and-such a movement" without making that kind of claim.
Oh yeah, I don't think anyone should claim to be effective or rational, per se. Fwiw the original Sequences make it very clear that a rationalist is supposed to be somebody who pursues rationality, rather than possessing it.
So you say it should be possible to avoid making this claim. I agree, and I believe Eliezer tried! Unfortunately, it was attributed to him anyway.
So you say it should be possible to avoid making this claim. I agree, and I believe Eliezer tried! Unfortunately, it was attributed to him anyway.
Right. And I think that using "Rationality" as a name for the movement as opposed to its goals does amount to making that claim, which is one reason why (so far as I can see) EY never actually uses the word in quite that way and I don't think others should either.
Along these lines I am sort of skimming articles/blogs/websites about Lightcone, LessWrong, etc, and I am still struggling with the question...what do they DO?
Look, it's just an internet community of people who write blog posts and discuss their interests on web forums.
Asking "What do they do?" is like asking "What do Hackernewsers do?"
It's not exactly a coherent question. Rationalists are a somewhat tighter group, but in the end the point stands. They write and discuss their common interests, e.g. the progress of AI, psychiatry stuff, bayesianism, thought experiments, etc.
Asking "What do they do?" is like asking "What do Hackernewsers do?"
It's not exactly a coherent question. Rationalists are a somewhat tighter group, but in the end the point stands. They write and discuss their common interests, e.g. the progress of AI, psychiatry stuff, bayesianism, thought experiments, etc.
Twenty years or so ago, Eliezer Yudkowsky, a former proto-accelerationalist, realized that superintelligence was probably coming, was deeply unsafe, and that we should do something about that. Because he had a very hard time convincing people of this to him obvious fact, he first wrote a very good blog about human reason, philosophy and AI, in order to fix whatever was going wrong in people's heads that caused them to not understand that superintelligence was coming and so on. The group of people who read, commented on and contributed to this blog are called the rationalists.
(You're hearing about them now because these days it looks a lot more plausible than in 2007 that Eliezer was right about superintelligence, so the group of people who've beat the drum about this for over a decade now form the natural nexus around which the current iteration of project "we should do something about unsafe superintelligence" is congealing.)
(You're hearing about them now because these days it looks a lot more plausible than in 2007 that Eliezer was right about superintelligence, so the group of people who've beat the drum about this for over a decade now form the natural nexus around which the current iteration of project "we should do something about unsafe superintelligence" is congealing.)
> hat superintelligence was probably coming, was deeply unsafe
Well, he was right about that. Pretty much all the details were wrong, but you can't expect that much so it's fine.
The problem is that it's philosophically confused. Many things are "deeply unsafe", the main example being driving or being anywhere near someone driving a car. And yet it turns out to matter a lot less, and matter in different ways, than you'd expect if you just thought about it.
Also see those signs everywhere in California telling you that everything gives you cancer. It's true, but they should be reminding you to wear sunscreen.
Well, he was right about that. Pretty much all the details were wrong, but you can't expect that much so it's fine.
The problem is that it's philosophically confused. Many things are "deeply unsafe", the main example being driving or being anywhere near someone driving a car. And yet it turns out to matter a lot less, and matter in different ways, than you'd expect if you just thought about it.
Also see those signs everywhere in California telling you that everything gives you cancer. It's true, but they should be reminding you to wear sunscreen.
I don't know - the level of seriousness they discuss w.r.t. alignment issues just seem so out of touch with the realities of large language models and the notion of a super intelligence being "closer than ever" gives way too much credit to the capabilities (or lack there of) of LLM's.
A lot of it seems rooted in Asimov-inspired, stimulant-fueled philosophizing than any kind of empirical or grounded observations.
A lot of it seems rooted in Asimov-inspired, stimulant-fueled philosophizing than any kind of empirical or grounded observations.
The ability of humans to get used to progress is amazing to me. We have a computer system that you can describe, in conversational English, the requirements for a program, including style, conventions and programming language, and this system will then synthesize this program from thin air, even if it's a novel idea with novel requirements, it will autonomously test it, make improvements and take your feedback into account. It will do this using a vast (if imperfect) knowledge of programming frameworks, far outstripping in breadth any individual human programmer.
And knowing this, you think that the only reason we could have to expect to create intelligence in a machine, even surpassing a human... is "Asimov-inspired, stimulant-fueled philosophizing"? That seems deeply unserious to me.
And knowing this, you think that the only reason we could have to expect to create intelligence in a machine, even surpassing a human... is "Asimov-inspired, stimulant-fueled philosophizing"? That seems deeply unserious to me.
I find LLM's just as awe inspiring as you. But you and I both will find them underwhelming eventually. This is the history of the field going all the way back to Turing.
At any rate, my point remains. The flaws inherent to the current deep learning regime _absolutely_ disqualify them as being capable of any sort of rapid takeoff/escalation (a la paperclip optimization) that the rationalist community is likely referring to when they say super intelligence or ASI.
Sorry about the "asimovian" comment - you'd be correct to call it an exaggeration and somewhat toxic.
At any rate, my point remains. The flaws inherent to the current deep learning regime _absolutely_ disqualify them as being capable of any sort of rapid takeoff/escalation (a la paperclip optimization) that the rationalist community is likely referring to when they say super intelligence or ASI.
Sorry about the "asimovian" comment - you'd be correct to call it an exaggeration and somewhat toxic.
I mean, I find them underwhelming today. I just also haven't been given any convincing evidence that the technology is anywhere close to tapped out yet, and lots of evidence that it isn't.
Hang out and talk
* 20 somethings who are clearly on spectrum
* Group are "special"
* Centered around a charismatic leader
* Weird sex stuff
Guys we have a cult!
* Group are "special"
* Centered around a charismatic leader
* Weird sex stuff
Guys we have a cult!
These are the people who came up with Roko's Basilisk, Effective Altruism and spawned the Zizians. I think Robert Evans described them not as a cult but as a cult incubator, or something along those lines.
Also:
* Communal living
* Sacred texts & knowledge
* Doomsday predictions
* Guru/prophet lives on the largesse of followers
It's rich for a group that claims to reason based on priors to completely ignore that they possess all the major defining characteristics of a cult.
* Communal living
* Sacred texts & knowledge
* Doomsday predictions
* Guru/prophet lives on the largesse of followers
It's rich for a group that claims to reason based on priors to completely ignore that they possess all the major defining characteristics of a cult.
Below is a list of cult markers I posted here the last time this came up. It recaps some of the bullet points above.
1. Apocalyptic world view.
2. Charismatic and/or exploitative leader.
3. Insularity.
4. Esoteric jargon.
5. Lack of transparency or accountability (often about finances or governance).
6. Communal living arrangements.
7. Sexual mores outside social norms, especially around the leader.
8. Schismatic offshoots.
9. Outsized appeal and/or outreach to the socially vulnerable.
1. Apocalyptic world view.
2. Charismatic and/or exploitative leader.
3. Insularity.
4. Esoteric jargon.
5. Lack of transparency or accountability (often about finances or governance).
6. Communal living arrangements.
7. Sexual mores outside social norms, especially around the leader.
8. Schismatic offshoots.
9. Outsized appeal and/or outreach to the socially vulnerable.
They seem to have a lot in common with the People's Front of Judea (or the Judian People's Front, for that matter).
Also, mandatory scriptures and group hiouses.
so many of the people i’ve read in these rationalist groups sound like they need a hug and therapy
https://en.wikipedia.org/wiki/Rationalist_community
"In particular, several women in the community have made allegations of sexual misconduct, including abuse and harassment, which they describe as pervasive and condoned."
There's weird sex stuff, logically, it's a cult.
"In particular, several women in the community have made allegations of sexual misconduct, including abuse and harassment, which they describe as pervasive and condoned."
There's weird sex stuff, logically, it's a cult.
Most weird sex stuff takes place outside of cults, so that doesn't follow.
There's weird sex stuff, logically, it's likely not a cult.
My eyes started to glaze over after a bit; so what I'm getting here is there a group that calls themselves "Rationalists," but in just about every externally meaningful sense, they're smelling like -- perhaps not a cult, but certainly a lot of weird insider/outsider talk that feels far from rational?
Capital r-Rationalism definitely bleeds into cult-like behaviour, even if they haven’t necessarily realised that they’re radicalising themselves.
They’ve already had a splinter rationalist group go full cult, right up to & including the consequent murders & shoot-out with the cops flameout: https://en.wikipedia.org/wiki/Zizians
They’ve already had a splinter rationalist group go full cult, right up to & including the consequent murders & shoot-out with the cops flameout: https://en.wikipedia.org/wiki/Zizians
It's encouraging to hear that behind all the internet noise, the real-life community is thriving and full of people earnestly trying to build a better future
The main things I don’t like about rationalism are aesthetic (the name sucks and misusing the language of Bayesian probability is annoying). Sounds like they are a thoughtful and nice bunch otherwise(?).
Reading this made me realize that rationality isn’t really about joining a group. It’s more about whether you care about how you think. No matter how solid the logic is, if the starting assumptions are off, it doesn’t help much. Reality is often messier than any model we build.
How do you decide when to trust the model vs trust your instincts?
[deleted]
[flagged]
Please don't comment like this on Hacker News. It breaks several guidelines, most notably these ones:
Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
When disagreeing, please reply to the argument instead of calling names.
Please don't fulminate. Please don't sneer...
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
https://news.ycombinator.com/newsguidelines.html
Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
When disagreeing, please reply to the argument instead of calling names.
Please don't fulminate. Please don't sneer...
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
https://news.ycombinator.com/newsguidelines.html
> clever enough to always be right
Oh, see here's the secret. Lots of people THINK they are always right. Nobody is.
The problem is you can read a lot of books, study a lot of philosophy, practice a lot of debate. None of that will cause you to be right when you are wrong. It will, however, make it easier for you to sell your wrong position to others. It also makes it easier for you to fool yourself and others into believing you're uniquely clever.
Oh, see here's the secret. Lots of people THINK they are always right. Nobody is.
The problem is you can read a lot of books, study a lot of philosophy, practice a lot of debate. None of that will cause you to be right when you are wrong. It will, however, make it easier for you to sell your wrong position to others. It also makes it easier for you to fool yourself and others into believing you're uniquely clever.
Sometimes the meta-skill of how you come across while being right is just as important as the correctness itself…
I don't see how that's any more "wanker" then this famous saying by Socrates's; Western thought is wankers all the way down.
> Although I do not suppose that either of us knows anything really beautiful and good, I am better off than he is – for he knows nothing, and thinks he knows. I neither know nor think I know.
> Although I do not suppose that either of us knows anything really beautiful and good, I am better off than he is – for he knows nothing, and thinks he knows. I neither know nor think I know.
“I don’t like how they said it” and “I don’t like how this made me feel” is the aspect of the human brain that has given us Trump. As long as the idea that “how you feel about it” is a basis for any decision making, the world will continue to be fucked. The authors audience largely understand that “this made me feel” is an indication that introspection is required, and not an indication that the author should be ignored.
It's a coping mechanism for autists, mainly.
Rationalists as a movement remind me of the individuals who claim to be serious about history but are only interested in a very, VERY specific set of six years in one very specific part of the world.
And boy are they extremely interested in ONLY those six years.
And boy are they extremely interested in ONLY those six years.
It's weird that "being interested in philosophy" is like... a movement. My background is in philosophy, but the rationalist vs nonrationalist debate seems like an undergraduate class dispute.
My old roommate worked for Open Phil, and was obsessed with AI Safety and really into Bitcoin. I never was. We still had interesting arguments about it all the time. Most of the time we just argued until we got to the axioms we disagreed on, and that was that.
You don't have to agree with the Rationalist™ perspective to apply philosophically rigorous thinking. You can be friends and allies with them without agreeing with all their views. There are strong arguments for why frequentism may be more applicable than bayesianism in different domains. Or why transhumanism is a pipe dream. They are still conversations that are worthwhile as long as you're not so confident in your position that you think you might learn something.
My old roommate worked for Open Phil, and was obsessed with AI Safety and really into Bitcoin. I never was. We still had interesting arguments about it all the time. Most of the time we just argued until we got to the axioms we disagreed on, and that was that.
You don't have to agree with the Rationalist™ perspective to apply philosophically rigorous thinking. You can be friends and allies with them without agreeing with all their views. There are strong arguments for why frequentism may be more applicable than bayesianism in different domains. Or why transhumanism is a pipe dream. They are still conversations that are worthwhile as long as you're not so confident in your position that you think you might learn something.
> It's weird that "being interested in philosophy" is like... a movement. My background is in philosophy, but the rationalist vs nonrationalist debate seems like an undergraduate class dispute.
Bring up the rationalist community within academic philosophy circles and you'll get a lot of groans.
The fun part about rationalists is that they like to go back to first principles and rediscover basics. The less fun part is that they'll ignore all of the existing work and pretend they're going to figure it all out themselves, often with weird results.
This leaves philosophy people endlessly frustrated as the rationalists write long essays about really basic philosophy concepts as if they're breaking new ground, while ignoring a lot of very interesting work that could have made the topic much more interesting to discuss.
Bring up the rationalist community within academic philosophy circles and you'll get a lot of groans.
The fun part about rationalists is that they like to go back to first principles and rediscover basics. The less fun part is that they'll ignore all of the existing work and pretend they're going to figure it all out themselves, often with weird results.
This leaves philosophy people endlessly frustrated as the rationalists write long essays about really basic philosophy concepts as if they're breaking new ground, while ignoring a lot of very interesting work that could have made the topic much more interesting to discuss.
Rationalists are constantly falling into ditches that actual philosophers crawled out of centuries ago. But what's even more exasperating is that they do it in tendentious disquisitions that take thousands of wasted words to get.. to.. the.. everloving.. point.
> they do it in tendentious disquisitions that take thousands of wasted words to get.. to.. the.. everloving.. point.
Right, and "actual philosophers" like Sartre and Heidegger _never_ did that. Ever.
"Being and Nothingness" and "Being and Time" are both short enough to fit into a couple tweets, right?
</irony>
Right, and "actual philosophers" like Sartre and Heidegger _never_ did that. Ever.
"Being and Nothingness" and "Being and Time" are both short enough to fit into a couple tweets, right?
</irony>
Thanks for reminding me that I left out their unjustified sense of significance and self-importance, thinking that their sophomoric ramblings are somehow comparable to Sartre's or Heidegger's writings.
I mean, I see the parallels. Heidegger was a Nazi, similar to how LWers are racist/eugenicists/Zionists.
It's a lot like how every "interesting"/pop scientific field (think quantum) has a niche community of Free Thinkers that the "establishment" just won't listen to. If you hear from the actual core scientific community you'll find out that they just already went over this 40-400 years ago and nothing being argued as groundbreaking is new or groundbreaking like it's being pitched as. That indifference can feel like rejection to the highly-excited outsiders which can develop into animosity which just further isolates them.
I mean, I don't disagree with you there. Even within academic philosophy circles, you'll get groans when one sect is discussing things with another sect. Lord knows ancient philosophy academics and analytic philosophy academics are going to get headaches just trying to hold a conversation... and we're not even including continental school.
My point is that, yes, while it may be a bit annoying in general (lord knows how many times I rolled my eyes at my old roommate talking about trans-humanism), the idea that this Rationalist™ movement "thinking about things philosophically" is controversial is just weird. That they seem to care about a philosophical approach to thinking about things, and maybe didn't get degrees and maybe don't understand much background while forming their own little school, seems as unremarkable is it is uncontroversial.
My point is that, yes, while it may be a bit annoying in general (lord knows how many times I rolled my eyes at my old roommate talking about trans-humanism), the idea that this Rationalist™ movement "thinking about things philosophically" is controversial is just weird. That they seem to care about a philosophical approach to thinking about things, and maybe didn't get degrees and maybe don't understand much background while forming their own little school, seems as unremarkable is it is uncontroversial.
Never heard of the man, but that was a fun read. And it looks like a fun club to be part of. Until in becomes unbearable perhaps. Also raises the chances to get invited to birthday orgies..? Perhaps I should have stayed a in academia..
> Until in becomes unbearable perhaps
Until?
Until?
>"frankly, that they gave off some (not all) of the vibes of a cult, with Eliezer as guru. Eliezer writes in parables and koans. He teaches that the fate of life on earth hangs in the balance, that the select few who understand the stakes have the terrible burden of steering the future"
One of the funniest and most accurate turns of phrases in my mind is Charles Stross' characterization of rationalists as "duck typed Evangelicals". I've come to the conclusion that American atheists just don't exist, in particular Californians. Five minutes after they leave organized religion they're in a techno cult that fuses chosen people myths, their version of the Book of Revelation, gnosticism and what have you.
I used to work abroad in Shenzhen for a few years and despite meeting countless of people as interested in and obsessed with technology, if not more than the people mentioned in this blogpost, there's just no corellary to this. There's no millenarian obsession over machines taking over the world, bizarre trust in rationalism or cult like compounds full of socially isolated new age prophets.
One of the funniest and most accurate turns of phrases in my mind is Charles Stross' characterization of rationalists as "duck typed Evangelicals". I've come to the conclusion that American atheists just don't exist, in particular Californians. Five minutes after they leave organized religion they're in a techno cult that fuses chosen people myths, their version of the Book of Revelation, gnosticism and what have you.
I used to work abroad in Shenzhen for a few years and despite meeting countless of people as interested in and obsessed with technology, if not more than the people mentioned in this blogpost, there's just no corellary to this. There's no millenarian obsession over machines taking over the world, bizarre trust in rationalism or cult like compounds full of socially isolated new age prophets.
This also related to that earlier bit:
> I also found them bizarrely, inexplicably obsessed with the question of whether AI would soon become superhumanly powerful and change the basic conditions of life on earth, and with how to make the AI transition go well. Why that, as opposed to all the other sci-fi scenarios one could worry about, not to mention all the nearer-term risks to humanity?
The reason they landed on a not-so-rational risk to humanity is because it fulfilled the psycho-social need to have a "terrible burden" that binds the group together.
It's one of the reasons religious groups will get caught up on The Rapture or whatever, instead of eradicating poverty.
> I also found them bizarrely, inexplicably obsessed with the question of whether AI would soon become superhumanly powerful and change the basic conditions of life on earth, and with how to make the AI transition go well. Why that, as opposed to all the other sci-fi scenarios one could worry about, not to mention all the nearer-term risks to humanity?
The reason they landed on a not-so-rational risk to humanity is because it fulfilled the psycho-social need to have a "terrible burden" that binds the group together.
It's one of the reasons religious groups will get caught up on The Rapture or whatever, instead of eradicating poverty.
"I have come out as a smart good thinking person, who knew"
>liberal zionist
hmmmm
>liberal zionist
hmmmm
As others have pointed out about the movement, it's for people who think they are smart and doing good things. It's just fitting that he is a Zionist who think he is a good person.
Scott Aaronson, the man who turned scrupulosity into a weapon against his own psyche is a capital R rationalist?
Yeah, this surprises absolutely nobody.
Yeah, this surprises absolutely nobody.
"YOUR ATTENTION PLEASE: I have now joined the club everyone assumed I was already a member of."
It's his personal blog, the only people whose attention he's asking for are the people choosing to wander over there to see what he's up to.
Not his fault that people deemed it interesting enough to upvote to the front page of HN.
Not his fault that people deemed it interesting enough to upvote to the front page of HN.
One of my many problems with rationalism is that it generally fails to acknowledge it's fundamentally religious character while pronouncing itself superior to all other religions.
"I’m still a computer scientist, an academic, a straight-ticket Democratic voter, a liberal Zionist, a Jew, etc. (all identities, incidentally, well-enough represented at LessOnline that I don’t even think I was the unique attendee in the intersection of them all)"
Not incidental!
Not incidental!
Sounds like they hear only themselves? There is a common type of sophisticated thinkers who have trained their intellectual muscle to a remarkable degree and got stuck there, refusing to see that intellect isn't the capstone of life. Their mind, ears and mouth quickly form a closed loop that makes them hear only what they themselves say. When this loop strengthens, this thinker becomes a dogmatic cult leader with a sophisticated, but dimly lit inner world that can, nevertheless impress smaller minds. Such inner worlds are always like mazes, with lots of rooms and corridors, all dimly lit, without exits and with their impressively developed mind roaming along these corridors like the Minotaur.
I used to snicker at these guys, but I realized I'm not being humble or to be more theologically minded: gracious.
Recognizing we all take a step of faith to move outside of solipsism into a relationship with others should humble us.
Recognizing we all take a step of faith to move outside of solipsism into a relationship with others should humble us.
Are you really concerned about the consequences of AI?
Then, empower UN-like organizations to oversee the use of technology - like the United Nations Atomic Energy Commission.
And, if you're even further concerned, put in place mechanisms that guarantee that the productivity gains, yields, and GDP increases obtained via the new technology of AI are distributed and enjoyed by all of the living population with a minimum of fairness.
For some reason, specially this last bit, doesn't really fly with our friends, The Rationalists. I wonder why...
Then, empower UN-like organizations to oversee the use of technology - like the United Nations Atomic Energy Commission.
And, if you're even further concerned, put in place mechanisms that guarantee that the productivity gains, yields, and GDP increases obtained via the new technology of AI are distributed and enjoyed by all of the living population with a minimum of fairness.
For some reason, specially this last bit, doesn't really fly with our friends, The Rationalists. I wonder why...
I don't understand the urge for Americans to name things the opposite of what they are.
> “You’re Scott Aaronson?! The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right, but who sustains an unreasonable amount of psychic damage in the process?”
Give me strength. So much hubris with these guys (and they’re almost always guys).
I would have assumed that a rationalist would look for truth and not correctness.
Oh wait, it’s all just a smokescreen for know-it-alls to show you how smart they are.
Give me strength. So much hubris with these guys (and they’re almost always guys).
I would have assumed that a rationalist would look for truth and not correctness.
Oh wait, it’s all just a smokescreen for know-it-alls to show you how smart they are.
Pretty sure that's meant to be taken light-heartedly. You know, as a joke.
It's not.
That's exactly what Rationalism(tm) is.
The basic trope is showing off how smart you are and what I like to call "intellectual edgelording." The latter is basically a fetish for contrarianism. The big flex is to take a very contrarian position -- according to what one imagines is the prevailing view -- and then defend it in the most creative way possible.
Intellectual edgelording gives us shit like neoreaction ("monarchy is good actually" -- what a contrarian flex!), timeless decision theory, and wild-ass shit like the Zizians, effective altruists thinking running a crypto scam is the best path to maximizing their utility, etc.
Whether an idea is contrarian or not is unrelated to whether it's a good idea or not. I think the fetish for contrarianism might have started with VCs playing public intellectual, since as a VC you make the big bucks when you make a contrarian bet that pays off. But I think this is an out-of-context misapplication of a lesson from investing to the sphere of scientific and philosophical truth. Believing a lot of shitty ideas in the hopes of finding gems is a good way to drive yourself bonkers. "So I believe in the flat Earth, vaccines cause autism, and loop quantum gravity, so I figure one big win this portfolio makes me a genius!"
Then there's the cults. I think this stuff is to Silicon Valley and tech what Scientology is to Hollywood and the film and music industries.
The basic trope is showing off how smart you are and what I like to call "intellectual edgelording." The latter is basically a fetish for contrarianism. The big flex is to take a very contrarian position -- according to what one imagines is the prevailing view -- and then defend it in the most creative way possible.
Intellectual edgelording gives us shit like neoreaction ("monarchy is good actually" -- what a contrarian flex!), timeless decision theory, and wild-ass shit like the Zizians, effective altruists thinking running a crypto scam is the best path to maximizing their utility, etc.
Whether an idea is contrarian or not is unrelated to whether it's a good idea or not. I think the fetish for contrarianism might have started with VCs playing public intellectual, since as a VC you make the big bucks when you make a contrarian bet that pays off. But I think this is an out-of-context misapplication of a lesson from investing to the sphere of scientific and philosophical truth. Believing a lot of shitty ideas in the hopes of finding gems is a good way to drive yourself bonkers. "So I believe in the flat Earth, vaccines cause autism, and loop quantum gravity, so I figure one big win this portfolio makes me a genius!"
Then there's the cults. I think this stuff is to Silicon Valley and tech what Scientology is to Hollywood and the film and music industries.
Thank you for finally making this make sense to me.
Another thing that's endemic in Rationalism is a kind of specialized variety of the Gish gallop.
It goes like this:
(1) Assert a set of priors (with emphasis on the word assert).
(2) Reason from those priors to some conclusion.
(3) Seamlessly, without skipping a beat, take that solution as valid because the reasoning appears consistent and make that part of a new set of priors.
(4) Repeat, or rather recurse since the new set of priors is built on previous iterations.
The entire concept of science is founded on the idea that you can't do that. You have to stop and touch grass, which in science means making observations or doing experiments if possible. You have to see if the conclusion you reached actually matches reality in any meaningful way. That's because reason alone is fragile. As any programmer knows, a single error or a single mistaken prior propagates and renders the entire tree invalid. Do this recursively and one error anywhere in this crystalline structure means you've built a gigantic tower of bullshit.
I compare it to the Gish gallop because of how enthusiastically they do it, and how by doing it so fast it becomes hard to try to argue against. You end up having to try to counter a firehose of Oh So Very Smart complicated exquisitely reasoned nonsense.
Or you can just, you know, conclude that this entire method of determining truth is invalid and throw the entire thing in the trash.
A good "razor" for this kind of thing is to judge it by its fruit. So far the fruit is AI hysteria, cults like the Zizians, neoreactionary political ideology, Sam Bankman Fried, etc. Has anything good or useful come from any of this?
It goes like this:
(1) Assert a set of priors (with emphasis on the word assert).
(2) Reason from those priors to some conclusion.
(3) Seamlessly, without skipping a beat, take that solution as valid because the reasoning appears consistent and make that part of a new set of priors.
(4) Repeat, or rather recurse since the new set of priors is built on previous iterations.
The entire concept of science is founded on the idea that you can't do that. You have to stop and touch grass, which in science means making observations or doing experiments if possible. You have to see if the conclusion you reached actually matches reality in any meaningful way. That's because reason alone is fragile. As any programmer knows, a single error or a single mistaken prior propagates and renders the entire tree invalid. Do this recursively and one error anywhere in this crystalline structure means you've built a gigantic tower of bullshit.
I compare it to the Gish gallop because of how enthusiastically they do it, and how by doing it so fast it becomes hard to try to argue against. You end up having to try to counter a firehose of Oh So Very Smart complicated exquisitely reasoned nonsense.
Or you can just, you know, conclude that this entire method of determining truth is invalid and throw the entire thing in the trash.
A good "razor" for this kind of thing is to judge it by its fruit. So far the fruit is AI hysteria, cults like the Zizians, neoreactionary political ideology, Sam Bankman Fried, etc. Has anything good or useful come from any of this?
Rationalists are better called Rationalizationists, really.
The problem with rationalism is we don't have language to express our thoughts formally enough nor a compiler to transform that language into something runnable (platonic AST) nor a machine capable of emulating reality.
Expecting rational thought to correspond to reality is like expecting a 6 million line program written in a hypothetical programming language invented in the 1700s to run bug free on a turing machine.
Tooling matters.
Expecting rational thought to correspond to reality is like expecting a 6 million line program written in a hypothetical programming language invented in the 1700s to run bug free on a turing machine.
Tooling matters.
The rationalist movement has been talking about this since the beginning, and has consistently come to the conclusion that of course the thing to do is admit to our own fallibility and try to do the best we can with our profound limitations.
(You didn’t explicitly say otherwise, so if my exasperation is misdirected then you have my apology in advance.)
(You didn’t explicitly say otherwise, so if my exasperation is misdirected then you have my apology in advance.)
No need to apologize, I am unashamedly an opponent of rationalism and I hold the entire movement and every one like it in history in withering contempt.
I was at Lighthaven that week. The weekend-long LessOnline event Scott references opened what LightHaven termed "Festival Season", with a summer camp organised for the following 5 week days, and a prediction market & forecasting conference called Manifest the following weekend.
I didn't attend LessOnline since I'm not active on LessWrong nor identify as a rationalist - but I did attended a GPU programming course in the "summer camp" portion of the week, and the Manifest conference (my primary interest).
My experience generally aligns with Scott's view, the community is friendly and welcoming, but I had one strange encounter. There was some time allocated to meet with other attendees at Manifest who resided in the same part of the world (not the bay area). I ended up surrounded by a group of 5-6 folks who appeared to be friends already, had been a part of the Rationalist movement for a few years, and had attended LessOnline the previous weekend. They spent most of the hour critiquing and comparing their "quality of conversations" at LessOnline with the less Rationalist-y, more prediction market & trading focused Manifest event. Completely unaware or unwelcoming of my presence as an outsider, they essentially came to the conclusion that a lot of the Manifest crowd were dummies and were - on average - "more wrong" than themselves. It was all very strange, cult-y, pseudo-intellectual, and lacking in self-awareness.
All that said, the experience at Summer Camp and Manifest was a net positive, but there is some credence to sneers aimed at the Rationalist community.
I didn't attend LessOnline since I'm not active on LessWrong nor identify as a rationalist - but I did attended a GPU programming course in the "summer camp" portion of the week, and the Manifest conference (my primary interest).
My experience generally aligns with Scott's view, the community is friendly and welcoming, but I had one strange encounter. There was some time allocated to meet with other attendees at Manifest who resided in the same part of the world (not the bay area). I ended up surrounded by a group of 5-6 folks who appeared to be friends already, had been a part of the Rationalist movement for a few years, and had attended LessOnline the previous weekend. They spent most of the hour critiquing and comparing their "quality of conversations" at LessOnline with the less Rationalist-y, more prediction market & trading focused Manifest event. Completely unaware or unwelcoming of my presence as an outsider, they essentially came to the conclusion that a lot of the Manifest crowd were dummies and were - on average - "more wrong" than themselves. It was all very strange, cult-y, pseudo-intellectual, and lacking in self-awareness.
All that said, the experience at Summer Camp and Manifest was a net positive, but there is some credence to sneers aimed at the Rationalist community.
I attended Manifest to meet some rationalists. I consider myself a post-rat. I also heard some comments that Manifest was less Rationalist-y than LessOnline but w/o the additional commentary on "dummies" (not meant to imply you didn't hear it).
I did find some rationalists too far down their "epistemological rabbit hole" to successfully unwind in one or two conversations but nevertheless many clever people. I still need some time to make post-rats out of them, though.
Affirming that it was a positive experience. I'm glad to have attended.
I did find some rationalists too far down their "epistemological rabbit hole" to successfully unwind in one or two conversations but nevertheless many clever people. I still need some time to make post-rats out of them, though.
Affirming that it was a positive experience. I'm glad to have attended.
Had to stop reading, everyone sounded so awful.
[deleted]
I think I'm missing something important.
My understanding of "Rationalists" is that they're followers of rationalism; that is, that truth can be understood only through intellectual deduction, rather than sensory experience.
I'm wondering if this is a _different_ kind of "Rationalist." Can someone explain?
My understanding of "Rationalists" is that they're followers of rationalism; that is, that truth can be understood only through intellectual deduction, rather than sensory experience.
I'm wondering if this is a _different_ kind of "Rationalist." Can someone explain?
It's a terrible name that collides with the older one you're thinking of
The easiest way to understand their usage of the term "rational" might be to think of it as the negation of the term "irrational" (where the latter refers mostly to cognitive biases). Not as a contrast with "empirical."
I read Scott Aaronson's blog and his papers and enjoy both. That said, he's a perfect example of a person who is a genius in one area, thinks this translates to expertise or intuition in other areas, and is often very, very wrong.
Human emotions guide reasoning: that's why we need Politics.
These people should have read "Descartes' Error" with more attention than they spent on Friedman and Hayek.
https://en.wikipedia.org/wiki/Descartes'_Error
These people should have read "Descartes' Error" with more attention than they spent on Friedman and Hayek.
https://en.wikipedia.org/wiki/Descartes'_Error
I'm yet to see "Effective Altruism" or (neo-)"Rationalism" that is not mostly the rationalization of exploiting less privileged people than them, and justifying behaving in anti-social, exploitative and extractionary ways.
Examples?
"I'm a Rationalist"
"Here are some labels I identify as"
So they arent rational enough to understand first principles don't objectively exist.
They were corrupted by words of old men, and have built a foundation of understanding on them. This isnt rationality, but rather Reason based.
I consider Instrumentalism and Bayesian epistemology to be the best we can get towards knowledge.
I'm going to be a bit blunt and not humble at all, this person is a philosophical inferior to myself. Their confidence is hubris. They haven't discovered epistemology. There isnt enough skepticism in their claims. They use black and white labels and black and white claims. I remember when I was confident like the author, but a few empirical pieces of evidence made me realize I was wrong.
"it is a habit of mankind to entrust to careless hope what they long for, and to use sovereign reason to thrust aside what they do not fancy."
"Here are some labels I identify as"
So they arent rational enough to understand first principles don't objectively exist.
They were corrupted by words of old men, and have built a foundation of understanding on them. This isnt rationality, but rather Reason based.
I consider Instrumentalism and Bayesian epistemology to be the best we can get towards knowledge.
I'm going to be a bit blunt and not humble at all, this person is a philosophical inferior to myself. Their confidence is hubris. They haven't discovered epistemology. There isnt enough skepticism in their claims. They use black and white labels and black and white claims. I remember when I was confident like the author, but a few empirical pieces of evidence made me realize I was wrong.
"it is a habit of mankind to entrust to careless hope what they long for, and to use sovereign reason to thrust aside what they do not fancy."
Ah, so it's like the Order of the October Star: certain people have simply realized that they are entitled to wear it. Or, rather, that they had always been entitled to wear it. Got it.
Since intuitive and non-rational thinking are demonstrably rational in the face of incomplete information, I guess we’re all rationalists. Or that’s how I’m rationalizing it, anyway.
All rationalists and effective altruists are bad people pretending to be good people for social capital.
The narcissism in this movement is insufferable. I hope the conditions for its existence will soon pass and give way to something kinder and more learned.
I'm so out of the loop. What is the new, special sense of Rationalist over what it might have meant to e.g. Descarte?
[deleted]
I cannot falsify the thesis that so-called "Rationalism" and "Effective Altruism" is - in practice - just folklore so that top-percenters don't pay taxes and feel good about themselves.
As the least worse solution to maximize Social Utility has long been invented: Democracy and political action.
As the least worse solution to maximize Social Utility has long been invented: Democracy and political action.
I have some doubts about the rationality of trying to avoid paying taxes by sending half of their salary to anti-malaria charities, but maybe those people just suck at math which is why they get paid so much.
Or maybe you are wrong.
Or maybe you are wrong.
Instead of waving the appeal to authority fallacy, try to do some reading up:
https://ips-dc.org/the-true-cost-of-billionaire-philanthropy...
https://ips-dc.org/the-true-cost-of-billionaire-philanthropy...
This is largely a cult, showing most of the red flags.
But if we put aside the narcissistic traits, lack of intellectual humility, religious undertones and (paradoxically) appeal to emotional responses with apocalyptic framing, the whole thing is still irrelevant BS.
They work in a vacuum, on either false or artificial premises with nothing to back their claims except long strings of syllogism.
This is not Science, no measurements, no experiments, no validation, zero value apart from maybe intellectual stimulation and socialisation for nerds with too much free time…
But if we put aside the narcissistic traits, lack of intellectual humility, religious undertones and (paradoxically) appeal to emotional responses with apocalyptic framing, the whole thing is still irrelevant BS.
They work in a vacuum, on either false or artificial premises with nothing to back their claims except long strings of syllogism.
This is not Science, no measurements, no experiments, no validation, zero value apart from maybe intellectual stimulation and socialisation for nerds with too much free time…
Sorry, I haven't followed what is it that these guys call Rationalism?
https://en.wikipedia.org/wiki/Rationalist_community
Fair warning: when you turn over some of the rocks here you find squirming, slithering things that should not be given access to the light.
Fair warning: when you turn over some of the rocks here you find squirming, slithering things that should not be given access to the light.
> The closest to right-wing politics that I witnessed at LessOnline was a session, with Kelsey Piper and current and former congressional staffers, about the prospects for moderate Democrats to articulate a moderate, pro-abundance agenda that would resonate with the public and finally defeat MAGA.
I can't say I'm surprised.
I can't say I'm surprised.
I somehow was expecting talk about Descartes and Spinoza.
> A third reason I didn’t identify with the Rationalists was, frankly, that they gave off some (not all) of the vibes of a cult, with Eliezer as guru.
Apart from a charismatic leader, a cult (in the colloquial meaning) needs a business model, and very often, a sense of separation from, and lack of accountability to those who are outside the cult, which provides conveniently simpler environment under which the cults ideas operate. A sort of "complexity filter" at the entry gate.
I'm not sure how the Rationalists compare to those criteria, but I'd be curious to find out.
Apart from a charismatic leader, a cult (in the colloquial meaning) needs a business model, and very often, a sense of separation from, and lack of accountability to those who are outside the cult, which provides conveniently simpler environment under which the cults ideas operate. A sort of "complexity filter" at the entry gate.
I'm not sure how the Rationalists compare to those criteria, but I'd be curious to find out.
[deleted]
[deleted]
I stopped reading once I read the word "zionist"
[deleted]
> “You’re [X]?! The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right, but who sustains an unreasonable amount of psychic damage in the process?”
> “Yes,” I replied, not bothering to correct the “physicist” part.
Didn't read much beyond that part. He'll fit right in with the rationalist crowd...
> “Yes,” I replied, not bothering to correct the “physicist” part.
Didn't read much beyond that part. He'll fit right in with the rationalist crowd...
No actual person talks like that —- and if they really did, they’ve taken on the role of a fictional character. Which says a lot about the clientele either way.
I skimmed a bit here and there after that but this comes off as plain grandiosity. Even the title is a line you can imagine a hollywood character speaking out loud as they look into the camera, before giving a smug smirk.
I skimmed a bit here and there after that but this comes off as plain grandiosity. Even the title is a line you can imagine a hollywood character speaking out loud as they look into the camera, before giving a smug smirk.
I assumed that the stuff in quotes was a summary of the general gist of the conversations he had, not a word for word quote.
I don't think GP objects to the literalness, as much as to the "I am known for always being right and I acknowledge it", which comes off as.. not humble.
He is known for that, right or wrong.
I mean, Scott's been wrong on plenty of issues, but of course he is not wont to admit that on his own blog.
> —-
0.o
> No actual person talks like that
I think it is plausible that there are people (readers) that find other people (bloggers) basically always right, and that would be the first think they would say to them if they met them. n=1, but there are some bloggers that I think are basically always right, and I am socially bad, so there is no telling what would I blurt out if I met them.
0.o
> No actual person talks like that
I think it is plausible that there are people (readers) that find other people (bloggers) basically always right, and that would be the first think they would say to them if they met them. n=1, but there are some bloggers that I think are basically always right, and I am socially bad, so there is no telling what would I blurt out if I met them.
To be honest, if I encountered Scott Aaronson in the wild I would probably react the same way. The guy is super smart and thoughtful, and can write more coherently about quantum computing than anyone else I'm aware of.
if only he stayed silent on politics...
Why would you comment on the post if you stopped reading near its beginning? How could your comments on it conceivably be of any value? It sounds like you're engaging in precisely the kind of shallow dismissal the site guidelines prohibit.
Aren't you doing the same thing?
No, I read the comment in full, analyzed its reasoning quality, elaborated on the self-undermining epistemological implications of its content, and then related that to the epistemic and discourse norms we aspire to here. My dismissal of it is anything but shallow, though I am of course open to hearing counterarguments, which you have fallen short of offering.
You are clearly dismissing his experience without much though. If a park ranger stopped you in the woods and said there was a mountain lion up ahead would you argue that he doesn't have enough information to be sure from such a quick glance?
Someone spending a lot of time to build one or multiple skills doesn't make them an expert on everything, but when they start talking like they are an expert on everything because of the perceived difficulties of one or more skills then red flags start to pop up and most reasonable people will notice them and swiftly call them out.
For example Elon Musk saying "At this point I think I know more about manufacturing than anyone currently alive on earth" even if you rationalize that as an out of context deadpan joke it's still completely correct to call that out as nonsense at the very least.
The more a person rationalizes statements like ("AI WILL KILL US ALL") these made by a person or cult the more likely it is that they are a cult member and they lack independent critical thinking, as they outsourced their thinking to group. Maybe their thinking is "the best thoughts", in-fact it probably is, but it's dependent on the group so their individual thinking muscle is weaken, which increases their morbidity (Airstricking a data center will get you killed or arrested by the US Gov. So it's better for the individual to question such statements rather than try to rationalize them using unprovable nonsense like god or AGI).
Someone spending a lot of time to build one or multiple skills doesn't make them an expert on everything, but when they start talking like they are an expert on everything because of the perceived difficulties of one or more skills then red flags start to pop up and most reasonable people will notice them and swiftly call them out.
For example Elon Musk saying "At this point I think I know more about manufacturing than anyone currently alive on earth" even if you rationalize that as an out of context deadpan joke it's still completely correct to call that out as nonsense at the very least.
The more a person rationalizes statements like ("AI WILL KILL US ALL") these made by a person or cult the more likely it is that they are a cult member and they lack independent critical thinking, as they outsourced their thinking to group. Maybe their thinking is "the best thoughts", in-fact it probably is, but it's dependent on the group so their individual thinking muscle is weaken, which increases their morbidity (Airstricking a data center will get you killed or arrested by the US Gov. So it's better for the individual to question such statements rather than try to rationalize them using unprovable nonsense like god or AGI).
If a park ranger said "I looked over the first 2% of the park and concluded there's no mountain lions" - that is, made an assessment on the whole from inspection of a narrow segment - I don't think I would take his word on the matter. If OP had more experience to support his statement, he should have included it, rather than writing a shallow, one-sentence dismissal.
I think the recently popular way of saying "I looked over 2% and assumed it generalizes" these days is to call the thing ergodic.
Which of course the blog article is not, but then at least the complaint wouldn't sound so obviously shallow.
Which of course the blog article is not, but then at least the complaint wouldn't sound so obviously shallow.
That's great, I'm definitely going to roll that into my vocabulary.
Also...
> they gave off some (not all) of the vibes of a cult
...after describing his visit with an atmosphere that sounds extremely cult-like.
> they gave off some (not all) of the vibes of a cult
...after describing his visit with an atmosphere that sounds extremely cult-like.
At least one cult originates from the Rationalist movement, the Zizians [1]. A cult that straight up murdered at least four people. And while the Zizian belief system is certainly more extreme then mainstream Rationalist beliefs, it's not that much more extreme.
For more info, the Behind the Bastards podcast [2] did a pretty good series on how the Zizians sprung up out of the Bay area Rationalist scene. I'd highly recommend giving it a listen if you want a non-rationalist perspective on the Rationalist movement.
[1]: https://en.wikipedia.org/wiki/Zizians [2]: https://www.iheart.com/podcast/105-behind-the-bastards-29236...
For more info, the Behind the Bastards podcast [2] did a pretty good series on how the Zizians sprung up out of the Bay area Rationalist scene. I'd highly recommend giving it a listen if you want a non-rationalist perspective on the Rationalist movement.
[1]: https://en.wikipedia.org/wiki/Zizians [2]: https://www.iheart.com/podcast/105-behind-the-bastards-29236...
The podcast Behind the Bastards described Rationalism not as a cult but as the fertile soil which is perfect for growing cults, leading to the development of cults like the Zizians (who both the Rationalists and Zizians are at pains to emphasize their mutual hostility to one another, but if you're not part of either movement, it's pretty clear how Rationalism can lead to something like the Zizians).
I don't think that podcast has very in-depth observations. It's just another iteration of east coast culture media people who used to be on Twitter a lot, isn't it?
> the fertile soil which is perfect for growing cults
This is true but it's not rationalism, it's just that they're from Berkeley. As far as I can tell if you live in Berkeley you just end up joining a cult.
> the fertile soil which is perfect for growing cults
This is true but it's not rationalism, it's just that they're from Berkeley. As far as I can tell if you live in Berkeley you just end up joining a cult.
I lived in Berkeley for a decade and there weren't many people I would say were in a cult. It's actually quite the opposite. There's way more willingness to be weird and do your own thing there.
Most of the rationalists I met in the Bay Area moved there specifically to be closer to the community.
Most of the rationalists I met in the Bay Area moved there specifically to be closer to the community.
No, Guru Eliezer Yudkowsky wrote an essay about how people asking "This isn’t a cult, is it?" bugs him, so it's fine actually. https://www.readthesequences.com/Cultish-Countercultishness
Hank Hill: Are y'all with the cult?
Cult member: It's not a cult! It's an organization that promotes love and..
Hank Hill: This is it.
Cult member: It's not a cult! It's an organization that promotes love and..
Hank Hill: This is it.
Does this quote make more sense in context? I don't watch the show.
It makes perfect sense as a comment, seeing the clip from the show doesn't add anything.
I think i'm seeing it now. "This is it" implies Hank is actually looking for the cult in question, something that I guess seems odd to me without knowing the plot of the episode.
Thanks for clarifying though! Oh wait, you didn't.
Thanks for clarifying though! Oh wait, you didn't.
Extreme eagerness to disavow accusations of cultishness ... doth the lady protest too much perhaps? My hobby is occasionally compared to a cult. The typical reaction of an adherent to this accusation is generally "Heh, yeah, totally a cult."
Edit: Oh, but you call him "Guru" ... so on reflection you were probably (?) making the same point... (whoosh, sorry).
Edit: Oh, but you call him "Guru" ... so on reflection you were probably (?) making the same point... (whoosh, sorry).
> Extreme eagerness to disavow accusations of cultishness ... doth the lady protest too much perhaps?
You don't understand how anxious the rationalist community was around that time. We're not talking self-assured confident people here. These articles were written primarily to calm down people who were panickedly asking "we're not a cult, are we" approximately every five minutes.
You don't understand how anxious the rationalist community was around that time. We're not talking self-assured confident people here. These articles were written primarily to calm down people who were panickedly asking "we're not a cult, are we" approximately every five minutes.
I got to that part, thought it was a joke, and then... it wasn't.
Stopped reading thereafter. Nobody speaking like this will have anything I want to hear.
Stopped reading thereafter. Nobody speaking like this will have anything I want to hear.
Scott's done a lot of really excellent blogging in the past. Truthfully, I think you risk depriving yourself of great writing if you're willing to write off an author because you didn't like one sentence.
GRRM famously written some pretty awkward sentences but it'd be a shame if someone turned down his work for that alone.
GRRM famously written some pretty awkward sentences but it'd be a shame if someone turned down his work for that alone.
Since I had not engaged with the ASOIAF community for quite some time, I had forgotten that, apparently, the phrase "fat pink mast" is stored somewhere in my long term memory. I sometimes struggle to remember my own wedding anniversary, but "fat pink mast" pops right to the surface.
I'd like to thank my useless brain for deciding to write that one down.
I'd like to thank my useless brain for deciding to write that one down.
Is it not a joke? I’m pretty sure it was.
It doesn’t really read like a joke but maybe. Regardless, I guess I can at least be another voice saying it didn’t land. It reads like someone literally said that to him verbatim and he literally replied with a simple, “Yes.” (That said, while it seems charitable to assume it was a joke but that doesn’t mean it’s wrong to assume that.)
I'm certain it's a joke. Have you seen any Scott Aaronson lecture? He can't help himself from joking in every other sentence
I laughed, definitely read that way to me
I think the fact that we aren't sure says a lot!
It says a lot, but what does it say a lot about?
https://www.smbc-comics.com/comic/aaaah
https://www.smbc-comics.com/comic/aaaah
If that was a joke, all of it is.
*Guess I’m a rationalist now.
*Guess I’m a rationalist now.
[deleted]
[deleted]
As someone who likes both the Rationalist community and the Rust community, it's fascinating to see the parallels in how the Hacker News crowd treats both.
The contempt, the general lack of curiosity and the violence of the bold sweeping statements people will make here are mind-boggling.
The contempt, the general lack of curiosity and the violence of the bold sweeping statements people will make here are mind-boggling.
> the general lack of curiosity
Honestly, I find the Hacker News comments in recent years to be most enlightening because so many comments come from people who spent years immersed in rationalist communities.
For years one of my friend groups was deep into LessWrong and SSC. I've read countless blog posts and other content out of those groups.
Yet every time I write about it, I'm dismissed as an uninformed outsider. It's an interesting group of people who like to criticize and dissect other groups, but they don't take kindly to anyone questioning their own circles.
Honestly, I find the Hacker News comments in recent years to be most enlightening because so many comments come from people who spent years immersed in rationalist communities.
For years one of my friend groups was deep into LessWrong and SSC. I've read countless blog posts and other content out of those groups.
Yet every time I write about it, I'm dismissed as an uninformed outsider. It's an interesting group of people who like to criticize and dissect other groups, but they don't take kindly to anyone questioning their own circles.
> Yet every time I write about it, I'm dismissed as an uninformed outsider.
No; you're being dismissed as someone who is entirely too credulous about arguments that don't hold up to scrutiny.
Edit: and as someone who doesn't understand basics about what rationalists are trying to accomplish in certain contexts (like the concept of a calibration curve re the example you brought up of https://www.astralcodexten.com/p/grading-my-2021-predictions). You come across (charitably) as having missed the point, because you have.
No; you're being dismissed as someone who is entirely too credulous about arguments that don't hold up to scrutiny.
Edit: and as someone who doesn't understand basics about what rationalists are trying to accomplish in certain contexts (like the concept of a calibration curve re the example you brought up of https://www.astralcodexten.com/p/grading-my-2021-predictions). You come across (charitably) as having missed the point, because you have.
You keep saying things that are proven to be trivially false with 5 seconds of examination, so I think the dismissal is justified.
Both the Rationalist community and the Rust community are very active in pursuing their goals, and unfortunately, it's far easier to criticize others for doing things than it is to actually do things yourself. Worse yet, if you are not yourself actively doing things, you are far more likely to experience fear when other people are actively doing things as there is always some nonzero chance that they will do things counter to your own goals, forcing you to actively do something lest you fall behind. Alas, people often respond to fear with hatred, especially given the benefit of physical isolation and dissociation from humanity offered by the Internet, and I think that's what you're seeing here on Hacker News.
I thought HN liked Rust? I learned Rust because I heard about it on HN back in the day. I don't want to start a debate about programming languages but if I were going to look for Rust fans, this is where I would expect to find them.
I mostly see non-Rust languages' communities on the defensive on HackerNews, not the other way around.
If you open any thread related to Zig or Odin or C++ you can usually ctrl-F "Rust" and find someone having an argument about how Rust is better.
EDIT: Didn't have to look very far back for an example: https://news.ycombinator.com/item?id=44319008
If you open any thread related to Zig or Odin or C++ you can usually ctrl-F "Rust" and find someone having an argument about how Rust is better.
EDIT: Didn't have to look very far back for an example: https://news.ycombinator.com/item?id=44319008
There is little ideological overlap between rationalists and Rust folks (and thank god for that). IME unlike the rationalists, Rust has actually done something useful for the world, and HN’s opinion of Rust and its community has drastically improved over the last decade.
The rationalists have not had any such clearly positive effects, and rather their adherents (Thiel, Vance, etc) have had severely deleterious effects on society.
There is no comparison between the two communities.
The rationalists have not had any such clearly positive effects, and rather their adherents (Thiel, Vance, etc) have had severely deleterious effects on society.
There is no comparison between the two communities.
I don't know much about Thiel, but Vance does not seem like an "adherent" of rationalism in any way I recognize. I have trouble imagining Vance talking about "The Sequences" or "AI x-risk."
Mentioning Thiel and Vance together brings to mind a different thread of weird netizen philosophizing - one I don't really know much about but which I guess I'd sum up as the "Moldbug / Curtis Yarvin fandom." "Neoreactionaries" might be the right term?
I definitely recognize that the Venn diagram between those two "intellectual movements" (big scare quotes there) overlaps quite a bit, but it seems like a bit of stretch to lump what Vance, Thiel, and other right-wing tech bro types are up to under the rationalism banner.
Update: Having read through some of the other links in the thread, I have "updated" (as the rationalists say) my mental model of that Venn diagram to be slightly more overlapping. I still think they're distinct, but there's more cross-pollination between the Moldbugs and the ACX crowd than I initially realized.
Mentioning Thiel and Vance together brings to mind a different thread of weird netizen philosophizing - one I don't really know much about but which I guess I'd sum up as the "Moldbug / Curtis Yarvin fandom." "Neoreactionaries" might be the right term?
I definitely recognize that the Venn diagram between those two "intellectual movements" (big scare quotes there) overlaps quite a bit, but it seems like a bit of stretch to lump what Vance, Thiel, and other right-wing tech bro types are up to under the rationalism banner.
Update: Having read through some of the other links in the thread, I have "updated" (as the rationalists say) my mental model of that Venn diagram to be slightly more overlapping. I still think they're distinct, but there's more cross-pollination between the Moldbugs and the ACX crowd than I initially realized.
What kind of commentary here have you heard here around the Rust community? I haven’t heard much. Certainly nothing close to violent. I’ve heard lots of shit talking regarding Go because they refused to introduce generics - and to be fair, the reasoning behind the refusal was smug arrogance led by the pompous cock rpike.
I will say as someone who has been programming before we had standardized C++ that “programming communities” aren’t my cup of tea. I like the passion and enthusiasm but it would be good for some of those lads to have a drag, see a shrink and get some nookie.
I will say as someone who has been programming before we had standardized C++ that “programming communities” aren’t my cup of tea. I like the passion and enthusiasm but it would be good for some of those lads to have a drag, see a shrink and get some nookie.
[deleted]
[flagged]
I view this as a political constraint, cf. https://www.astralcodexten.com/p/lifeboat-games-and-backscra.... One's identity as Academic, Democrat, Zionist and so on demands certain sacrifices of you, sometimes of rationality. The worse the failure of empathy and rationality, the better a test of loyalty it is. For epistemic rationality, it would be best to https://paulgraham.com/identity.html, but for instrumental rationality it is not. Consequently, many people are reasonable only until certain topics come up, and it's generally worked around by steering the discussion to other topics.
I don’t really buy this at all: I am more emotionally invested in things that I know more about (and vice versa). If Rationalism breaks down at that point it is essentially never useful.
> I don’t really buy this at all
For what it’s worth, you seem to be agreeing with the person you replied to. Their main point is that this break down happens primarily because people identify as Rationalists (or whatever else). Taken from that angle, Rationalism as an identity does not appear to be useful.
For what it’s worth, you seem to be agreeing with the person you replied to. Their main point is that this break down happens primarily because people identify as Rationalists (or whatever else). Taken from that angle, Rationalism as an identity does not appear to be useful.
My reading of the comment was that there was only a small subset of contentious topics that rationalism is unsuited for. But I think you are correct
[flagged]
I'm curious how you assess, relatively speaking, the shittiness of defence of genocide versus false claims of genocide.
[flagged]
But that's not my question. My question was between defence of genocide and false accusations of genocide. (Of course actual genocide is "shittier" -- in fact that's a breathtaking understatement!)
Wouldn’t it be better to spend the time understanding the reality of the situation in Gaza from multiple angles rather than philosophizing on abstract concepts? I.e. there are different degrees of genocide, but that doesn’t matter in this context because what’s happening in Gaza is not abstract or theoretical.
In other words, your question ignores so much nuance that it’s a red herring IMO.
In other words, your question ignores so much nuance that it’s a red herring IMO.
[flagged]
If you suggest an answer to your own question, it could be disputed. Better to make a coy comment and expect others to take that risk.
I'm not the one making pronouncements about the "shittiness" of forms of human behaviour.
God forbid.
Sorry, I'm not sure what you mean. Could you explain what you mean by "god forbid" in this context?
Assuming you do believe that genocide is extremely shitty, wouldn't that imply that defense of (actual) genocide, or the principle of it, is in all likelihood shitter than a false accusation of genocide? Otherwise I think you'd have to claim that a false accusation is somehow worse than the actuality or possibility of mass murder, which seems preposterous if you have even a mote of empathy for your fellow human beings.
As others have pointed out, the fact that you would like to make light of cities being decimated and innocent civilians being murdered at scale in itself suggests a lot about your inability to concretize the reality of human existence beyond yourself (lack of empathy). It's this kind of outright callousness toward actual human beings that I think many of these so called "rationalists" share. I can't fault them too much. After all, when your approach to social problems is highly if not strictly quantitative you are already primed to nullify your own aptitude for empathy, since you view other human beings as nothing more than numerical quantities whenever you attempt to address their problems.
I have seen no defense for what's happening in gaza that anyone who actually values human life, for all humans, would find rational. Recall the root of the word ratio—in proportion. What is happening in this case is quite blatantly a matter of an inproportinate response.
As others have pointed out, the fact that you would like to make light of cities being decimated and innocent civilians being murdered at scale in itself suggests a lot about your inability to concretize the reality of human existence beyond yourself (lack of empathy). It's this kind of outright callousness toward actual human beings that I think many of these so called "rationalists" share. I can't fault them too much. After all, when your approach to social problems is highly if not strictly quantitative you are already primed to nullify your own aptitude for empathy, since you view other human beings as nothing more than numerical quantities whenever you attempt to address their problems.
I have seen no defense for what's happening in gaza that anyone who actually values human life, for all humans, would find rational. Recall the root of the word ratio—in proportion. What is happening in this case is quite blatantly a matter of an inproportinate response.
> Assuming you do believe that genocide is extremely shitty, wouldn't that imply that defense of (actual) genocide, or the principle of it, is in all likelihood shitter than a false accusation of genocide? Otherwise I think you'd have to claim that a false accusation is somehow worse than the actuality or possibility of mass murder
I'm struggling to follow, sorry.
I certainly agree with you that a false accusation is not worse than the actuality (I don't know why you brought up "possibility") of mass murder. Very far from it. But why does that imply that it's better than the defence of mass murder? After all, the "defence" here is not engaging in the practice, it's just saying something like "I condone that". Or did you think that by "defence" I actually mean committing the mass murder?
The reason that emotive false accusations are very, very harmful is that they can cause mobs to murder in (supposed) retaliation. Here's a story about someone in the UK who was killed by a riled-up mob, due to a false accusation:
https://www.dailymail.co.uk/news/article-3535839/Father-42-k...
One ought to be very, very cautious about making accusations that can rile up mobs.
Regarding your other comments directed at me personally, such as "you would like to make light of cities being decimated and innocent civilians being murdered at scale", "inability to concretize the reality of human existence beyond yourself", "outright callousness", "approach to social problems is highly if not strictly quantitative", "you view other human beings as nothing more than numerical quantities", they are completely unfounded speculation on your part. They are rude and completely inappropriate for a reasoned discussion.
Regarding proportion, do you believe the actions of the UK and USA against Nazi Germany were "proportionate"? Proportionate to what? What did Nazi Germany ever to do the USA?
I'm struggling to follow, sorry.
I certainly agree with you that a false accusation is not worse than the actuality (I don't know why you brought up "possibility") of mass murder. Very far from it. But why does that imply that it's better than the defence of mass murder? After all, the "defence" here is not engaging in the practice, it's just saying something like "I condone that". Or did you think that by "defence" I actually mean committing the mass murder?
The reason that emotive false accusations are very, very harmful is that they can cause mobs to murder in (supposed) retaliation. Here's a story about someone in the UK who was killed by a riled-up mob, due to a false accusation:
https://www.dailymail.co.uk/news/article-3535839/Father-42-k...
One ought to be very, very cautious about making accusations that can rile up mobs.
Regarding your other comments directed at me personally, such as "you would like to make light of cities being decimated and innocent civilians being murdered at scale", "inability to concretize the reality of human existence beyond yourself", "outright callousness", "approach to social problems is highly if not strictly quantitative", "you view other human beings as nothing more than numerical quantities", they are completely unfounded speculation on your part. They are rude and completely inappropriate for a reasoned discussion.
Regarding proportion, do you believe the actions of the UK and USA against Nazi Germany were "proportionate"? Proportionate to what? What did Nazi Germany ever to do the USA?
We have concrete examples of defence of genocide, such as by Scott Aaronson. Can you provide the examples of "false accusations of genocide", otherwise this is a hypothetical conversation.
I can certainly agree we have a concrete example of defence of purported genocide and a concrete example of an accusation of purported genocide. Beyond that I'd be happy to discuss further (although it's probably off topic).
[flagged]
Interesting presumption. Would you like it if I said "Did you find anything else helpful in your marriage, or was it just choosing to stop beating your wife?"?
And yes, I have some. One is that false claims of genocide are equally reprehensible to denying true genocide. But I'm not sure why my beliefs are particularly relevant. I'm not the one sitting publicly in judgement of a semi-public figure. That was voidhorse.
Did you want to discuss in more detail? I'm happy to, but currently I interpret your comment as an attempt at sniping me with snark. Please do correct me if I've misinterpreted.
And yes, I have some. One is that false claims of genocide are equally reprehensible to denying true genocide. But I'm not sure why my beliefs are particularly relevant. I'm not the one sitting publicly in judgement of a semi-public figure. That was voidhorse.
Did you want to discuss in more detail? I'm happy to, but currently I interpret your comment as an attempt at sniping me with snark. Please do correct me if I've misinterpreted.
[flagged]
Interesting conclusion, since I didn't make a claim either way.
Still, for the record, other independent observers have documented the practices and explained why they don't meet the definition of genocide, John Spencer and Natasha Hausdorff to name two examples. It seems by no means clear that it's valid to make a claim of genocide. I certainly wouldn't unless I was really, really certain of my claim, because to get such a claim wrong is equally egregious to denying a true genocide, in my opinion.
Still, for the record, other independent observers have documented the practices and explained why they don't meet the definition of genocide, John Spencer and Natasha Hausdorff to name two examples. It seems by no means clear that it's valid to make a claim of genocide. I certainly wouldn't unless I was really, really certain of my claim, because to get such a claim wrong is equally egregious to denying a true genocide, in my opinion.
voidhorse(1)
SantalBlush(1)
(Ive also been somewhat dogmatic and angry about this conflict, in the opposite direction. But I wouldnt call myself a rationalist)
Anything in particular you want to link to as unreasonable?
What's incredible to me is the political blindness. Surely at this point, "liberal zionists" would at least see the writing on the wall? Apply some Bayesian statistical analysis to popular reactions to unprompted military strikes against Iran or something, they should realize at this point that in 25 years the zeitgeist will have completely turned against this chapter in Israel's history, and properly label the genocide for what it is.
I thought these people were the ones that were all about most effective applications of altruism? Or is that a different crowd?
I thought these people were the ones that were all about most effective applications of altruism? Or is that a different crowd?
pbiggar(3)
I'm absolutely shocked that a rationalist would also be a Zionist /s
What the fuck am I reading lmao.
paganel(2)
Very Bay Area to assume you invented Bayesian thinking.
This is what rationalisme entails: https://plato.stanford.edu/entries/rationalism-empiricism/
That's a different definition of rationalism from what is used here.
It is. But the Rationalists, by taking that name as a label, are claiming that they are what the GP said. They want the prestige/respect/audience that the word gets, without actually being that.
(The rationalists never took that label, it is falsely applied to them. The project is called rationality, not rationalism. Unfortunately, this is now so pervasive that there's no fixing it.)
Hmm, interesting. Might I trouble you for your definitions of rationality and rationalism?
(Not a "gotcha". I really want to know.)
(Not a "gotcha". I really want to know.)
Sure! Rationality is what Eliezer called his project about teaching people to reason better (more empirically, more probabilistically) in the events I described over here: https://news.ycombinator.com/item?id=44320919 .
I don't know rationalism too well but I think it was a historical philosophical movement asserting you could derive knowledge by reasoning from axioms rather than observation.
The primary difference here is that rationality mostly teaches "use your reason to guide what to observe and how to react to observations" rather than doing away with observations altogether; it's basically an action loop alternating between observation and belief propagation.
A prototypical/mathematical example of a pure LessWrong-type "rational" reasoner is Hutter's AIXI (a definition of the "optimal" next step given an input tape and a goal), though it has certain known problems of self-referentiality. Though of course reasoning in this way does not work for humans; a large part of the Sequences is attempts to port mathematically correct reasoning to human cognition.
You can kind of read it as a continuation of early-2000s internet atheism: instead of defining correct reasoning by enumerating incorrect logic, ie. "fallacies", it attempts to construct it positively, by describing what to do rather than just what not to do.
I don't know rationalism too well but I think it was a historical philosophical movement asserting you could derive knowledge by reasoning from axioms rather than observation.
The primary difference here is that rationality mostly teaches "use your reason to guide what to observe and how to react to observations" rather than doing away with observations altogether; it's basically an action loop alternating between observation and belief propagation.
A prototypical/mathematical example of a pure LessWrong-type "rational" reasoner is Hutter's AIXI (a definition of the "optimal" next step given an input tape and a goal), though it has certain known problems of self-referentiality. Though of course reasoning in this way does not work for humans; a large part of the Sequences is attempts to port mathematically correct reasoning to human cognition.
You can kind of read it as a continuation of early-2000s internet atheism: instead of defining correct reasoning by enumerating incorrect logic, ie. "fallacies", it attempts to construct it positively, by describing what to do rather than just what not to do.
Classical Less Wrong text about "what is rationality": https://www.readthesequences.com/What-Do-I-Mean-By-Rationali...
Shortly: believing what is true, and choosing the actions that lead to the things you value.
If the sky is blue and you can verify that by looking at the sky, it is reasonable to believe that sky is blue, and it is unreasonable to believe that the sky is green just because some authority or your favorite political party said so. If you know that eating poison would kill you, and if you want to live a long life, it is reasonable to avoid the poison, and unreasonable to eat the poison.
These are separate skills, because many people know what to do, and yet don't do that, or are good at following their beliefs, but the beliefs happen to be wrong. Studying the rational beliefs is called "epistemology", studying the rational actions is called "decision theory".
.
The word "rationalism" is used by many people in different ways: https://en.wikipedia.org/wiki/Rationalism_(disambiguation)
In internet discussions inevitably someone mentions the 17th century definition of "rationalism" as opposed to "empiricism" as the only historically valid meaning of the word. But for example, the approach of Karl Popper is often called "critical rationalism", and although the Less Wrong philosophy is different from Popper's, it is closer to him than to the 17th century "rationalists".
(The difference between Popper and Less Wrong in a nutshell: Popper treats arguments in favor of a theory, and arguments against a theory, as two fundamentally different things that follow different rules. Less Wrong treats all arguments the same way, using the Bayes Theorem. In practice, the difference is smaller than it might seem, because Popper's main concern was to never treat a theory as a 100% truth, especially when there is evidence against it, and Less Wrong agrees that you should never treat any theory as 100% likely. The advantage of Less Wrong approach is that you can also apply it to probabilistic theories. For example, one person says that a coin is fair, another person says that actually the coin is 55% likely to come up heads, and 45% likely to come up tails. I have no idea how you would decide this problem from Popper's perspective, because any experimental result is kinda compatible with both theories, it's just the more coinflips you make, the less likely some of those theories become; but there is no clear line when you should call one of them "falsified".)
.
I think this is the kind of thing where you simply can't make everyone happy, no matter what you do. As an analogy, imagine a world where every time you say e.g. "American president", at least three people in the thread remind you that actually Donald Trump is not a president of the entire continent, only of a part of the North America. So the next time you pay extra attention and say carefully "the president of the United States", and then everyone is like: "you mean the American president, why don't you speak simply?".
There is a community around Less Wrong. Whatever we call them, we should choose the name so that it is obvious that we refer to them, and not to the 17th century philosophers who believed that evidence is not necessary. Things are real, words are just labels.
At the beginning, it was just a few dozen people from different parts of the planet, reading the same blog. They referred to themselves as "aspiring rationalists". (As a label that applies to an individual who aspires to be more rational.) I would be okay to use this label, but apparently many people are too lazy to say "aspiring" all the time, and when you only say "rationalists", (1) it sounds smug, as if you believe that you already are perfectly rational, rather than you are trying to become more so, and (2) inevitably, some outsider will mention the 17th century philosophers. I think people used "Less Wrong community", "rationality community", "LessWrong-style rationalists", "x-rationalists", and maybe a few other words. But there was always pushback against using "rationalist" without any adjective.
As a long-time member of the community myself, I don't have a problem with anyone using any of these labels. I know what you mean, and I am not going to play dumb. It's the non-members who need to coordinate on a standard label for us, so that they are not confused about who they are talking about. And I am even okay with them choosing a label that we don't use. (I am doing the same to other groups, e.g. I say "Mormons" instead of "The Church of Jesus Christ of Latter-day Saints".) If the world decides to call us "rationalists", so be it... but then don't blame us for using the same label as the 17th century philosophers, because we don't. I think that "rationality community" or "Less Wrong community" are both nice options. (Just please don't call us TESCREAL because that's a crazy conspiracy theory.)
Shortly: believing what is true, and choosing the actions that lead to the things you value.
If the sky is blue and you can verify that by looking at the sky, it is reasonable to believe that sky is blue, and it is unreasonable to believe that the sky is green just because some authority or your favorite political party said so. If you know that eating poison would kill you, and if you want to live a long life, it is reasonable to avoid the poison, and unreasonable to eat the poison.
These are separate skills, because many people know what to do, and yet don't do that, or are good at following their beliefs, but the beliefs happen to be wrong. Studying the rational beliefs is called "epistemology", studying the rational actions is called "decision theory".
.
The word "rationalism" is used by many people in different ways: https://en.wikipedia.org/wiki/Rationalism_(disambiguation)
In internet discussions inevitably someone mentions the 17th century definition of "rationalism" as opposed to "empiricism" as the only historically valid meaning of the word. But for example, the approach of Karl Popper is often called "critical rationalism", and although the Less Wrong philosophy is different from Popper's, it is closer to him than to the 17th century "rationalists".
(The difference between Popper and Less Wrong in a nutshell: Popper treats arguments in favor of a theory, and arguments against a theory, as two fundamentally different things that follow different rules. Less Wrong treats all arguments the same way, using the Bayes Theorem. In practice, the difference is smaller than it might seem, because Popper's main concern was to never treat a theory as a 100% truth, especially when there is evidence against it, and Less Wrong agrees that you should never treat any theory as 100% likely. The advantage of Less Wrong approach is that you can also apply it to probabilistic theories. For example, one person says that a coin is fair, another person says that actually the coin is 55% likely to come up heads, and 45% likely to come up tails. I have no idea how you would decide this problem from Popper's perspective, because any experimental result is kinda compatible with both theories, it's just the more coinflips you make, the less likely some of those theories become; but there is no clear line when you should call one of them "falsified".)
.
I think this is the kind of thing where you simply can't make everyone happy, no matter what you do. As an analogy, imagine a world where every time you say e.g. "American president", at least three people in the thread remind you that actually Donald Trump is not a president of the entire continent, only of a part of the North America. So the next time you pay extra attention and say carefully "the president of the United States", and then everyone is like: "you mean the American president, why don't you speak simply?".
There is a community around Less Wrong. Whatever we call them, we should choose the name so that it is obvious that we refer to them, and not to the 17th century philosophers who believed that evidence is not necessary. Things are real, words are just labels.
At the beginning, it was just a few dozen people from different parts of the planet, reading the same blog. They referred to themselves as "aspiring rationalists". (As a label that applies to an individual who aspires to be more rational.) I would be okay to use this label, but apparently many people are too lazy to say "aspiring" all the time, and when you only say "rationalists", (1) it sounds smug, as if you believe that you already are perfectly rational, rather than you are trying to become more so, and (2) inevitably, some outsider will mention the 17th century philosophers. I think people used "Less Wrong community", "rationality community", "LessWrong-style rationalists", "x-rationalists", and maybe a few other words. But there was always pushback against using "rationalist" without any adjective.
As a long-time member of the community myself, I don't have a problem with anyone using any of these labels. I know what you mean, and I am not going to play dumb. It's the non-members who need to coordinate on a standard label for us, so that they are not confused about who they are talking about. And I am even okay with them choosing a label that we don't use. (I am doing the same to other groups, e.g. I say "Mormons" instead of "The Church of Jesus Christ of Latter-day Saints".) If the world decides to call us "rationalists", so be it... but then don't blame us for using the same label as the 17th century philosophers, because we don't. I think that "rationality community" or "Less Wrong community" are both nice options. (Just please don't call us TESCREAL because that's a crazy conspiracy theory.)
[deleted]
For any speed-runners out there: https://en.wikipedia.org/wiki/Two_Dogmas_of_Empiricism
I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle.. And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know"
In the Pre-AI days this was sort of tolerable, but since then.. The frothing at the mouth convinced of the end of the world.. Just shows a real lack of humility and lack of acknowledgment that maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected