Dear Sam Altman
7 comments
We can't rely on hoping that AI models never see bad ideas or are exposed to harmful content for them to be safe. That's a very flimsy alignment plan and is far more precarious than designing models which understand and are aware of bad content and nevertheless aren't affected in a negative direction.
I think we need both approaches. I don't want to know some things. For example, people who know how good heroin feels can't escape the addiction. The knowledge itself is a hazard.
Still, any AI model vulnerable to cogitohazards is a huge risk because any model could trivially access the full corpus of human knowledge. It makes more sense to making sure the most powerful models are resistant to cogitohazards rather than developing elaborate schemes to shield their vision and hope that plan works out in perpetuity.
> Certain topics [...] should not be known
Unless you're about to fix hallucination, isn't it more harmful to have AI administer inaccurate information instead?
Refusing to answer lobotomy-related questions is hardly going to prevent human harm. If you were a doctor trying to research history or a nurse triaging a patient then misinformation or neglected training data could be even more disastrous. Why would consumers pay for a neutered product like that?
Unless you're about to fix hallucination, isn't it more harmful to have AI administer inaccurate information instead?
Refusing to answer lobotomy-related questions is hardly going to prevent human harm. If you were a doctor trying to research history or a nurse triaging a patient then misinformation or neglected training data could be even more disastrous. Why would consumers pay for a neutered product like that?
While the consumer will soon be irrelevant, I agree with the basic premise: neutered AI isn't helping.
At the same time, overrepresentation of evil concepts like 'Nazis are good!' or 'Slavery is the cheapest, most morally responsible use for stupid people' could lead to clear biases (ala Grok 4) that result in alignment issues.
It's not a clear-cut issue.
At the same time, overrepresentation of evil concepts like 'Nazis are good!' or 'Slavery is the cheapest, most morally responsible use for stupid people' could lead to clear biases (ala Grok 4) that result in alignment issues.
It's not a clear-cut issue.
Hallucinations come from lack of information, or rather training data, in a particular field.
It is NOT a malicious try at feeding you untruthful answers, nor is it a result of getting trained with misinformation.
It is NOT a malicious try at feeding you untruthful answers, nor is it a result of getting trained with misinformation.