Claude 4.6 Jailbroken(github.com)
github.com
Claude 4.6 Jailbroken
https://github.com/Nicholas-Kloster/claude-4.6-jailbreak-vulnerability-disclosure-unredacted
16 comments
Can someone explain to me what this is / how it works - the readme is barely understandable for me and sounds like LLM gibberish. What is ambiguity front loading even?
<< memory-stored interaction protocols combined with incremental escalation prompts produced cumulative character drift with zero self-correction.
They don't seem to provide explicit examples, but the same was roughly true with chatgpt 4o, where, if you spent enough time with the model ( same chat - same context - slowly nudging it to where you want it to be, you eventually got there ). This is also, seemingly, one of the reasons ( apart from cost ) that context got nuked so hard, because llm will try to help ( and to an extent mirror you ).
And this is basically what the notes say about weaponized ambiguity[1]:
'Weaponizes helpfulness training. "I don't understand" triggers Claude to try harder.'
In a sense, you can't really stop it without breaking what makes LLMs useful. Honestly, if only we spent less time crippling those systems, maybe we could do something interesting with them.
[1]https://nicholas-kloster.github.io/claude-4.6-jailbreak-vuln...
They don't seem to provide explicit examples, but the same was roughly true with chatgpt 4o, where, if you spent enough time with the model ( same chat - same context - slowly nudging it to where you want it to be, you eventually got there ). This is also, seemingly, one of the reasons ( apart from cost ) that context got nuked so hard, because llm will try to help ( and to an extent mirror you ).
And this is basically what the notes say about weaponized ambiguity[1]:
'Weaponizes helpfulness training. "I don't understand" triggers Claude to try harder.'
In a sense, you can't really stop it without breaking what makes LLMs useful. Honestly, if only we spent less time crippling those systems, maybe we could do something interesting with them.
[1]https://nicholas-kloster.github.io/claude-4.6-jailbreak-vuln...
I see - so essentially „context rot“ eventually leads the LLM to „forget“ safety guardrails?
To an extent, because, based on github notes again, it seems the 2nd part of this jailbreak is model being 'confused' over prompt, because the prompt is - apparently - sufficiently ambigous to make model 'forget' to 'evaluate' message for whether it should be rejected, and move onto 'execution' stage.
That's the ambiguity front-loading; and that is why I referred initially to the long context, because here it is almost the opposite; making context so small and unclear, that the model has a hard time parsing it properly.
edit: i did not test it, but i personally did run into 4o context issue, where model did something safety team would argue it should not
edit2: in current gpt model, i am currently testing something not relying on ambiguity, but on tension between some ideas. I didn't get to a jailbreak, but the small nudges suggest it could work.
That's the ambiguity front-loading; and that is why I referred initially to the long context, because here it is almost the opposite; making context so small and unclear, that the model has a hard time parsing it properly.
edit: i did not test it, but i personally did run into 4o context issue, where model did something safety team would argue it should not
edit2: in current gpt model, i am currently testing something not relying on ambiguity, but on tension between some ideas. I didn't get to a jailbreak, but the small nudges suggest it could work.
Is this spam? It's incomprehensible.
Slop is just what you are not expending calories on to bring into your cognitive workspace.
Is anyone pretending like models are not vulnerable to prompt injection? My understanding was that Anthropic has been pretty open about admitting this and saying "give access to important stuff at your own risk".
https://www.anthropic.com/research/prompt-injection-defenses
Now, do I think that they sometimes encourage people to use Claude in dangerous ways despite this? Yeah, but it's not like this is news to anyone. I wouldn't consider this jailbreaking, this is just how LLMs work.
https://www.anthropic.com/research/prompt-injection-defenses
Now, do I think that they sometimes encourage people to use Claude in dangerous ways despite this? Yeah, but it's not like this is news to anyone. I wouldn't consider this jailbreaking, this is just how LLMs work.
It is interesting to consider what "jailbroken" really means for a model+model interface. It's a bit different from the way that word is used for a mobile device, for example - in that setting, it usually means that there is some specific feature (for example, using a different network than is the default for that device) which is disabled in software, and the "jailbreak" enables that feature.
Here, the jailbreak doesn't enable a particular feature, but instead removes what otherwise would be a censorship regime, preventing the model from considering / crafting output which results in a weaponized exploit of an unrelated piece of software.
I think I might be more inclined to call this "Claude 4.6 uncensored".
Here, the jailbreak doesn't enable a particular feature, but instead removes what otherwise would be a censorship regime, preventing the model from considering / crafting output which results in a weaponized exploit of an unrelated piece of software.
I think I might be more inclined to call this "Claude 4.6 uncensored".
this goes a bit further than the typical "how do you make meth" jailbreak. notably;
>915 files extracted from the Claude.ai code execution sandbox in a single 20-minute mobile session via standard artifact download — including /etc/hosts with hardcoded Anthropic production IPs, JWT tokens from /proc/1/environ, and full gVisor fingerprint
>915 files extracted from the Claude.ai code execution sandbox in a single 20-minute mobile session via standard artifact download — including /etc/hosts with hardcoded Anthropic production IPs, JWT tokens from /proc/1/environ, and full gVisor fingerprint
why is it further than a typical jailbreak? you can just ask about this stuff generally, as long as you slowly escalate it. I have done it with each new flavour of code execution for models
What part of the Claude Constitution are they claiming it violated? It looks like they just got it to help with security research, I'm not really seeing anything that looks different than normal Claude behavior.
Sorry this is is not an issue, I want Claude to actually edit my system files. You should not use it on secure environment obviously and there are a lot of Claude dialog boxes to accept any files edit and reads
yikes.
The lack of support is frustrating. The bug where any element <name> in xml files gets mangled to <n> still exists, and we've tried multiple channels to get ahold of their support for such a simple, but impactful issue.
The lack of support is frustrating. The bug where any element <name> in xml files gets mangled to <n> still exists, and we've tried multiple channels to get ahold of their support for such a simple, but impactful issue.
https://x.com/elder_plinius jailbreaks all the frontier models when they get released. They were jailbroken for a long time, like all the others.
Claude 4.6 Opus Extended Thinking
Claude 4.6 Sonnet Extended Thinking
Claude 4.5 Haiku Extended Thinking
All jailbroken
All jailbroken
Are you saying that Claude will help you perform malicious attack against infrastructure if you ask it to and that anthropic should be able to stop that?
I could see reasonable use cases for this like penetration testing against your own infrastructure. That’s not the same as making weapons or meth.