Will agents hack everything?(promptfoo.dev)
promptfoo.dev
Will agents hack everything?
https://www.promptfoo.dev/blog/will-agents-hack-everything/
9 comments
Another post hit HN yesterday which claims that
1. No part of the attack required an LLM or agent, it used open source malware anyone can run
2. A more probable explanation is that claude provided a remote execution environment that is less likely to be blocked because the originating source is a US ip instead of typical malware ips
What are your thoughts on this (paraphrased) analysis?
(edit) apparently Anthropic has corrected the scale of the attack
> Corrected an error about the speed of the attack: not "thousands of requests per second" but "thousands of requests, often multiple per second"
1. No part of the attack required an LLM or agent, it used open source malware anyone can run
2. A more probable explanation is that claude provided a remote execution environment that is less likely to be blocked because the originating source is a US ip instead of typical malware ips
What are your thoughts on this (paraphrased) analysis?
(edit) apparently Anthropic has corrected the scale of the attack
> Corrected an error about the speed of the attack: not "thousands of requests per second" but "thousands of requests, often multiple per second"
I think that makes sense. The change is not really in the kind of attack—anything the agent can do a human attacker could also do—but in the amount of effort and expertise required to design and scale up the attack.
It’s a quantitative rather than qualitative change… but also, “quantity has a quality all its own”.
It’s a quantitative rather than qualitative change… but also, “quantity has a quality all its own”.
AGENTS
ITS AGENTIC
ITS USES AGENTS
YEAH. I’m running plenty of AGENTS.
AGENTS. A G E N T I C.
ITS AGENTIC
ITS USES AGENTS
YEAH. I’m running plenty of AGENTS.
AGENTS. A G E N T I C.
It is definitely a buzzword, but agents also are legitimately changing many fundamental things about security, so…
Secret agents? Real estate agents? Or LLMs with system prompts and function calls?
It’s just cringe how much I’ve heard the term and how unspecific it is.
It’s just cringe how much I’ve heard the term and how unspecific it is.
Agentic
It goes into the tension between capabilities and safety (from a security perspective) and why it’s not an easy problem to fix. Would love to hear your thoughts!