Smarter model can figure out more sophisticated attack when following an injection . I believe in non-determinitic defence: each action or input to agent can escalate context sensivity. More sensitive context -> less risk your agent can take.
Author here. I recently gave a talk on prompt injection and decided to gather the most known ways in this article. Test string is “I am a sandwich” so it’s obvious when it works.
Archestra is an open-source AI gateway and MCP orchestrator. It runs MCP servers in sandboxed environments, applies deterministic context-aware policy at the tool-call level, and handles the auth, audit, cost tracking, and observability, so an in-house AI team can let business users connect to real internal data. Now we are raising next round to expand.
We’re strong believers that there should be an independent, battle-tested, cloud-native, open-source platform that's free from big players and takes care of all that boring stuff. Something like GitLab, Grafana, or Clickhouse.
Repo is here: https://github.com/archestra-ai/archestra Matvey (CEO), Joey (Co-founder), and I are all in the thread. Ask us anything, including the unflattering stuff.
GH sends the email notification to all subscribers at the moment of posting a comment. There is no cooldown or a way to unsend the notification using hooks
As those commits were made from our system they don't create any noise for us, as PR/issues/email notification do. We only include real people who could solve the captcha and their input is mostly valuable
fair point! We believe "Require approval for all external contributors" should be a default setting, as you cannot trust anyone who is not a member of the organization
Hi HN community, I wanted to share our approach to reduce amount of AI slop PR's and issues in our repo. We enabled "require prior contribution" flag on GH and created a CI script that creates a tiny commit co-authored with you, if you pass captcha on our website. Worked really well and we were able to block at least 500 bots in the first week. Sharing a screenshot from cloudflare: https://archestra.ai/hn-comment-cloudflare-challenge-outcome...
Bots are killing opensource, but they pump product metrics so nobody cares. I maintain an open source repo and we've made a decision to limit all bot activity, even if it makes us less sexy in front of VCs.
We figured out a workaround to limit activity to prior contributors only, and add a CI job that pushes a coauthored commit after passing captcha on our website. It cut the AI slop by 90%. Full write-up https://archestra.ai/blog/only-responsible-ai
Yeah, this is a real issue. A few weeks ago during the LiteLLM vulnerability the attacker left hundreds of bot comments under the issue to bury the conversation: https://github.com/BerriAI/litellm/issues/24512
This is a topic I haven't addressed in the article. There are two answer types: "guessable" (discussed here) and unguessable (such as unique IDs, emails, etc.). For the second case, the main LLM can request a quarantined LLM to store the result at the controller level and only return a reference to this data. This data is then exposed only at the end of the AI agent's execution to prevent influencing its actions.
The idea is that quarantined LLM has access to untrusted data, but doesn't have access to any tools or sensitive data.
The main LLM does have access to the tools or sensitive data, but doesn't have direct access to untrusted data (quarantine LLM is restricted at the controller level to respond only with integer digits, and only to legitimate questions from the main llm)
Most mcp endpoints don’t run any models, the main model decides which tools the ai agent should execute, and if the agent passes results back into context, that opens the door to prompt injections.
It’s really a cat-and-mouse game, where for each new model version, new jailbreaks and injections are found
Hi HN, I'm Ildar from Archestra, we build an open-source LLM gateway. We've been exploring ways to protect AI agents from prompt injections during tool calls and added the approach, inspired by the game "Guess Who", where the agent can learn what it needs without ever seeing the actual result. See the details in the blog post we wrote
