>> This raises an important question: if AI models can be misused for cyberattacks at this scale, why continue to develop and release them? The answer is
> Money.
For those who didn’t read, the actual response in the text was was:
“The answer is that the very abilities that allow Claude to be used in these attacks also make it crucial in cyber defense.”
Hideous AI-slop-weasel-worded passive-voice way of saying that reason to develop Claude is to protect us from Claude.
> This reminds me of a paradox: The AI industry is concerned with the alignment problem (how to make a super smart AI adhere to human values and goals) while failing to align between and within organizations and with the broader world. The bar they’ve set for themselves is simply too high for the performance they’re putting out.
I’m still waiting for somebody to explain to me how a model with a million+ parameters can ever be interpretable in a useful way. You can’t actually understand the model state, so you’re just making very coarse statistical associations between some parameters and some kinds of responses. Or relying on another AI (itself not interpretable) to do your interpretation for you. What am I missing?
> The argument that persuaded many of us is that people have a lot of desires, i.e., the algorithmic complexity of human desires is at least dozens or hundreds of bits of information
I would really try to disentangle this.
1. I don't know what my desires are.
2. "Desire" itself is a vague word that can't be measured or quantified; where does my desire for "feeling at peace" get encoded in any hypothetical artificial mind?
3. People have different and opposing desires.
Therefore, Coherent Extrapolated Volition is not coherent to me.
This is kind of where I go when I say that any centralized, top-down "grand plan" for AI safety is a folly. On the other hand, we all contribute to Selection.
All it takes for somebody to nuke Atlanta is an atom bomb and an airplane and somebody willing to fly the plane.
I’m being facetious but there ARE ways to decide/act as a society and as subgroups within society that we want to disallow and punish and select out qualities of AIs that we think are unethical.
A single person acting in isolation (no friends, no colleagues, no customers) has very little agency. While theoretically a single person could release smallpox back into civilization, we have collectively selected it out very effectively.
Thank you for this. It gets exactly to the heart of the issue and what I sense is being missed in the AI alignment conversation. “What does ‘aligned’ mean” is and ethical/political question; and when people skip over that, it’s often to (1) smuggle in their own ethics and present them as universal, or (2) run away from the messy political questions and towards the safe but much narrower domain of technical research.
> AI Alignment is different because it is trying to align something which is completely human made.
Not sure what you’re getting at here; pharmaceuticals are also human made. The point in the blog post was that we should also want drugs (for example) to be aligned to our values.
> What I think is absolutely important to understand is that throughout human history "alignment" has never happened.
Agree with that. This is a journey, not a destination. It’s a practice, not a mathematical problem to be solved. With no end in sight. In the same way that “perfect ethics” will never be achieved.
Agreed - and this was definitely my intent with the blog post. If you only do Selection passively, you’re abdicating your ethical responsibilities to contribute to AI Alignment.
Indeed. Tolstoy does a deep exploration of this in War and Peace, using the example of Napoleon.
Of course, this gets to the heart of the free will debate (to be settled in a future post ;)). Both are true at the same time - organized people and dictators and other factors simultaneously wrestle for influence in complex ways in which causation is impossible to measure.
My own two cents, though, is that the Categorical Imperative is a tremendously important and underappreciated tool for raising the self-consciousness of groups.
A practical implementation of it is linked at the bottom of the blog post.
Not totally following your last point, though I do totally agree that there is this historical drift from “AI alignment” referring to existential risk, to today, where any AI personality you don’t like is “unaligned.”
Still, “AI existential risk” is practically a different beast from “AI alignment,” and I’m trying to argue that the latter is not just for experts, but that it’s mostly a sociopolitical question of selection.
> Money.
For those who didn’t read, the actual response in the text was was:
“The answer is that the very abilities that allow Claude to be used in these attacks also make it crucial in cyber defense.”
Hideous AI-slop-weasel-worded passive-voice way of saying that reason to develop Claude is to protect us from Claude.