HackerTrans
TopNewTrendsCommentsPastAskShowJobs

Acacian

no profile record

Submissions

Show HN: I tested 11 AI frameworks for basic security – none passed

acacian.github.io
1 points·by Acacian·3 mesi fa·1 comments

[untitled]

1 points·by Acacian·3 mesi fa·0 comments

Show HN: Aegis – Security framework for AI agents

acacian.github.io
3 points·by Acacian·4 mesi fa·0 comments

comments

Acacian
·3 mesi fa·discuss
LangChain, CrewAI, OpenAI Agents, Anthropic, LiteLLM, Pydantic AI, Google ADK — I went through 11 frameworks looking for basic runtime security: injection detection, PII masking, audit trails. None of them had it.

So I built a monkey-patching layer that intercepts LLM calls and runs them through guardrails:

import aegis; aegis.init()

Patches whatever frameworks you have installed. ~2.6ms overhead.

The nastiest find: streaming responses skip middleware entirely. Content leaks before any check runs. I wrote a streaming engine that auto-selects between windowed scanning and full buffering depending on what the guardrail needs — PII like "078-05-1120" can split across chunks, so regex won't catch it without the full buffer.

Context: https://github.com/langchain-ai/langchain/issues/35011 Source: https://github.com/Acacian/aegis
Acacian
·4 mesi fa·discuss
[dead]
Acacian
·4 mesi fa·discuss
[dead]
Acacian
·4 mesi fa·discuss
[dead]
Acacian
·4 mesi fa·discuss
[dead]
Acacian
·4 mesi fa·discuss
The base rate argument here is the right one. I maintain a solo project with 3,800+ tests and 92% coverage — zero stars for months because I never promoted it. Stars measure marketing, not quality.

What's more interesting to me is that Claude dramatically lowers the barrier to _testing_, not just writing code. I can mass-generate edge case tests that I'd never bother writing manually. The result is higher-quality solo repos that look "abandoned" by star count.

Is anyone tracking test coverage or CI pass rates for AI-assisted repos vs traditional ones? That seems like a much more useful signal than stars.
Acacian
·4 mesi fa·discuss
The verification pipeline is the most valuable part of your workflow. Most people who use AI for literature reviews skip exactly that step — they trust the output and move on.

What you're describing is closer to building a testing harness than "using AI to write." You're asserting claims, checking them against source PDFs, and reviewing manually. That's more rigorous than most manual lit reviews where people skim abstracts and cite papers they half-read.

Document the pipeline as methodology in your dissertation. That turns a potential misconduct question into a contribution.
Acacian
·4 mesi fa·discuss
[dead]
Acacian
·4 mesi fa·discuss
[dead]