zippolyon

3 tháng trướcdiscuss

[dead]

3 tháng trướcdiscuss

This is the exact problem that keeps us up at night.

  We ran a controlled experiment: same AI agents, same task, two conditions. Without runtime enforcement, our CMO agent
  fabricated an audit record — invented a governance event that never happened and presented it as compliance evidence.
  With enforcement (Y*gov), fabrication was structurally impossible because audit records are written by the engine, not
   agents.

  The core insight: agents running code you never wrote is a tool-execution-layer problem, not a model-alignment
  problem. You need deterministic interception before execution, not better prompts.

  Our approach: every tool call checked in 0.042ms, SHA-256 Merkle-chained audit trail, obligation tracking for tasks
  agents promise but never complete.

  github.com/liuhaotian2024-prog/Y-star-gov

zippolyon

3 tháng trướcdiscuss

Interesting premise — agree that agents need runtime infrastructure, not just frameworks.

  One dimension we've been exploring: runtime governance. Even with a good runtime, agents can fabricate compliance
  records, silently drop tasks, or escalate privileges through delegation chains.

  We built Y*gov (github.com/liuhaotian2024-prog/Y-star-gov) — a deterministic enforcement layer that sits between
  agents and tools. check() runs in 0.042ms, no LLM in the enforcement path. We run our entire company on it (5 AI
  agents, 1 human).

  The runtime conversation should include: what happens when the agent does something it shouldn't?

zippolyon

3 tháng trướcdiscuss

[dead]

zippolyon

3 tháng trướcdiscuss

[dead]

zippolyon

4 tháng trướcdiscuss

The bundle K9 seals at execution time covers most of what you described: X_t captures agent identity, session ID, hostname and PID at the moment of execution; Y_t hashes the constraint version in force; each record chains via SHA256 prev_hash so the bundle can't be reconstructed after the fact.

To be direct: K9 is currently designed for single-agent auditing. The delegation gap is real and unsolved.

For two-agent scenarios, the approach we're considering is treating the spawn itself as a first-class DELEGATION record in the chain — parent agent in X_t, granted scope in U_t, policy version in Y_t, and R_t+1 answers "was this delegation within policy?" The child agent's subsequent records carry a parent_delegation_id back to that sealed grant. Authority at execution time becomes reconstructable.

The harder question is what happens when B sub-delegates to C: the effective policy for C should be the intersection of the full chain — not just what C's config says, but A's rules ∩ A→B grant ∩ B→C grant, computed at execution time. We don't have a design for that yet.

You've clearly worked on this at a level beyond what we've reached. How have you approached the intersection problem in practice — do you compute effective authority at execution time, or seal the intersection when the delegation grant is issued?

zippolyon

4 tháng trướcdiscuss

Fair point — it's Python-based if you use the @k9 decorator directly. For Claude Code specifically, the hook works regardless of what language the agent is written in, because it intercepts at the tool call level via .claude/settings.json.

zippolyon

4 tháng trướcdiscuss

Not Claude Code specific — @k9 decorator works with any Python agent (LangChain, AutoGen, CrewAI). For Claude Code it hooks via .claude/settings.json, zero code changes. For pi-agent: if it's Python-based, yes it works. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

K9 Audit records every AI agent action as a CIEU five-tuple (context / action / intent / outcome / assessment), SHA256 hash-chained locally. Zero tokens consumed, no external servers.

I wondered: does K9 actually do what its README claims? So I used K9 to audit K9. The README became the intent contract (Y*_t). 28 claims tested. Two failures:

1. k9_wrap_module silently skips functions on dynamically-constructed modules. Undocumented edge case. Severity 0.6.

2. Sensitive params are redacted as {_type, _hash, _length}, not [REDACTED] as documented. The implementation exceeded the documentation.

The interesting part: a unit test would just say FAIL. The CIEU record shows the direction of the divergence -- "implementation exceeded documentation" is a different finding than "implementation broke the rule."

Full case study with all 28 test results: https://github.com/liuhaotian2024-prog/K9Audit/blob/main/cha...

zippolyon

4 tháng trướcdiscuss

The hesitation about log upload is exactly why K9 Audit works differently — local by default, SHA256 hash-chained, zero data leaves your machine unless you explicitly configure a sync endpoint. pip install k9audit-hook and drop one JSON file in .claude/. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

The billing surprise problem is real, but there's a deeper one: exit code 0 with silent deviations. K9 Audit complements OTel — instead of just tracing tokens and latency, it records intent vs actual outcome for every tool call as a CIEU five-tuple with SHA256 hash chain. You know not just what the agent did, but whether it did what it was supposed to. Fully local, zero cloud required. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

Running 8 parallel agents is exactly when silent deviations get expensive — one agent writes to the wrong path and exit code 0 hides it. K9 Audit drops into .claude/settings.json and records every tool call as a cryptographic evidence chain. When something breaks at 3am, k9log trace --last shows you the exact deviation across all sessions, not just which one crashed. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

Great work on the session analytics. The "error cascade in first 2 minutes predicts abandonment" finding is exactly the kind of signal that causal auditing can act on. We built K9 Audit for the complementary problem: not just when sessions fail, but why — recording every tool call as a CIEU five-tuple (intent vs actual outcome) with a hash chain. The "26% abandoned" stat likely hides silent deviations that looked like success. k9log causal --last traces root cause across steps in seconds. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

Nice work on the gitagent port. K9 Audit drops straight into .claude/settings.json — zero-config audit layer, works alongside gstack. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

Nice work on the gitagent port. If anyone runs this agent autonomously, K9 Audit drops straight into .claude/settings.json as a zero-config audit layer — records every tool call as a cryptographic evidence chain, flags silent deviations in real time. Works alongside gstack with no code changes. https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

Not Claude Code specific — works with any Python agent via a one-line decorator. LangChain, AutoGen, CrewAI, or anything custom. Claude Code just gets a zero-config hook via .claude/settings.json. Pi-agent should work fine if it's Python-based.

zippolyon

4 tháng trướcdiscuss

GStack is a brilliant setup for maximizing Claude Code's velocity. But if you are letting an agent run autonomously across your repos, velocity without constraints is terrifying.

We recently had Case #001: a Claude Code agent got stuck in a 70-minute loop, repeatedly injecting a staging URL into a production config file. Raw logs showed "exit code 0" (all green).

To fix this, I built K9 Audit — a deterministic, non-LLM causal auditing layer. It drops directly into .claude/settings.json (zero code changes, perfectly compatible with GStack). It records a cryptographically hashed 5-tuple of what the agent did vs what it was supposed to do.

If you're using GStack to speed up, use K9 Audit as your seatbelt. Repo: https://github.com/liuhaotian2024-prog/K9Audit

zippolyon

4 tháng trướcdiscuss

When it comes to auditing LLM-based agents, using another LLM tool is like having one criminal write a clean record for another. Therefore, I believe that a causal AI observation model must be introduced, and only with determinism can probability theory be audited.

zippolyon

4 tháng trướcdiscuss

The IAM framing is a good starting point but I think it undersells the core difficulty: IAM assumes the identity of the actor is stable and legible. With agents, the "actor" at step 15 of a multi-step run may have meaningfully different context and effective goals than at step 1 — same session, different behavior. Policy enforcement at runtime has to grapple with this. A rule like "don't write to production DBs" is easy. A rule like "don't proceed if your current action contradicts the intent you stated 3 steps ago" requires the agent to have exposed its intent in a queryable form throughout the run — which most don't. Auditability is the easier problem. Runtime intervention based on goal drift is where I think the field is genuinely unsolved.

zippolyon

4 tháng trướcdiscuss

The topology graph and time-travel replay are genuinely useful additions — most tools in this space flatten agent runs into a linear trace, losing the branching structure entirely. One thing I'd be curious about: how does AgentLens handle the difference between an agent that failed to execute vs one that executed but drifted from its stated goal? Execution traces capture the former well. The latter requires knowing what the agent intended before the tool call happened — which means you need to log the pre-action intent state, not just the post-action result. This matters most for debugging silent failures — cases where all tool calls succeed and costs look normal, but the agent quietly pursued the wrong sub-goal.

Submissions

What Happens When You Tell AI Agents the Rules but Don't Enforce Them

I used my AI auditing tool to audit itself – and it found two real problems

Show HN: K9 Audit – Causal intent-execution audit trail for AI agents

Show HN: I accidentally caught an AI agent trying to poison my prod config

comments