It is different from BI and the gap is worth naming. BI connects deterministic queries from a known operator schema. An agent is an unbounded query generator, so your risk surface includes both what it asks for and what it synthesises from what it gets back. RLS alone won't help you, because the moment the agent gets handed a view it is already reasoning over rows you wanted to protect.
The practical answer I've seen hold up: push column-level redaction before the agent layer, not after. A logical replica with PII columns replaced by null or a stable hash gives you the same query surface, plus one audit row per session at the connection pooler, not the app. The AI team gets its data, you get a hard boundary that doesn't rely on prompt engineering.
The harder question is ownership. In a startup where the ML lead, the infra person and the security person are often the same tired CTO at 10pm, the right answer depends on who gets paged when a hallucinated query wakes up the primary. Usually the answer is nobody, which is the real problem behind the technical one.
$34k in 8 days with zero users is the flavor of bug that makes CFOs distrust engineering. The thing that would have caught this: anomaly detection scoped to the service + the account, not just the total bill. Most teams monitor the aggregate and only spike-alert above some threshold — by then it's 4 days old. Per-service p95-vs-median alerts at hourly granularity would have flagged this inside 6 hours. Cloudflare should absolutely ship platform-side guardrails here, but until they do, self-built alerts at the service level are the only real defense.
eight months is usually when the first version of "what you built" stops working and you have to build the thing customers actually want. the gap between those two is what most people call "0 revenue growth."
we went through something similar building a fintech tool — spent months on features nobody asked for because we assumed we understood the problem. what actually changed it: stopped building for what we assumed and started building for one specific thing one customer said they'd pay for that day. just one. then the next.
eight months is usually when the first version of "what you built" stops working and you have to build the thing customers actually want. the gap between those two is what most people call "0 revenue growth."
we went through something similar building a fintech tool — spent months on features nobody asked for because we assumed we understood the problem. what actually changed it: stopped building for what we assumed and started building for one specific thing one customer said they'd pay for that day. just one. then the next.
Static IP whitelisting is a nightmare in practice -- we ran it for about 8 months and the support burden was basically someone's part-time job. Every time someone's hotel or coffee shop rotated IPs, they'd open a ticket. We moved to AWS SSM Session Manager a couple years back and haven't touched a bastion since. No open ports, no SSH keys to rotate, and `aws ssm start-session --target i-xxxxxxx` just works from anywhere as long as they have valid IAM creds. CloudTrail picks up the session automatically, which was a side benefit we hadn't even planned for.
IAM Identity Center (was just called SSO before the rename) with a short session duration -- we do 8-hour max with MFA at login -- handles the traveling employee case cleanly. They re-authenticate, no ticket. Overhead compared to running a VPN server you have to patch is basically zero.
The one thing that bit us: we kept a bastion sitting around "just in case" for way too long before we cleaned it up. It was live for almost a year after we didn't need it anymore. What's the main access pattern you're trying to solve -- DB access, SSH to EC2, or something different?
Deliberately is doing a lot of work in that sentence and it's exactly right. The shortcuts that kill you aren't the ones you knew were shortcuts it's the ones that felt like reasonable decisions under pressure and only look like shortcuts in hindsight.
The "isolate the risk" part is where most early CTOs underestimate the blast radius. A shortcut in the auth layer feels isolated until six months later when three other systems are built on top of assumptions it makes. By then it's not a shortcut anymore, it's load-bearing technical debt.
What's the recovery actually looked like in practice when you've caught it early enough — do you get clean rewrites or is it mostly containment?
The 20% contamination number on ClawHub was genuinely alarming -- at that scale it's not opportunistic, it's systematic. The multi-pass approach makes sense given how trivially obfuscated payloads evade single-regex scanning; same problem npm has been fighting for years where a base64 decode or dynamic require wrapper kills most static analysis.
One thing worth thinking about beyond detection: even a perfect scanner at install time doesn't protect against skills that start clean and phone home post-install. The runtime layer is a different problem -- restricting what a skill process can actually touch (outbound network, credential paths, filesystem writes outside its own dir) probably matters as much as the intake scan. Seccomp or at minimum per-skill network namespacing would close that gap.
Did any of the 824 original malicious skills survive all 6 passes, or were they each caught by at least one detector?
The vault/proxy layer solving the "2am paste" vector but not the semantic leakage is exactly the gap most teams don't account for. Ephemeral key naming, endpoint patterns, TTL behaviors -- all of this is in the training corpus and no amount of runtime secret rotation changes what the model already internalized. You've essentially found that your threat model stopped at input hygiene but the model itself is a side channel.
What I'd add to the defense stack: structured output validation that flags known internal naming patterns before they reach the client, plus anomaly detection on response metadata -- token budget shifts, refusal rates, response shape changes under semantic pressure. At 75% convergence you're past "interesting research" into "reliable extraction technique," which means you need a detection layer, not just prevention.
Have you tested this against non-OpenAI models like Claude or Gemini to see if the naming bleed is GPT-4o-specific or a broader training corpus problem?
The proxy approach in the other comment handles the technical control side well. The harder part is the auditor question you slipped in at the end — that one trips up almost every team I've talked to. Most companies cannot produce a log showing which employee sent what data to which model, when it was authorized, and what classification level it had. The governance infrastructure for AI just isn't there yet, and auditors are starting to catch up.
SOC 2 and ISO 27001 auditors are now explicitly asking for evidence of data flow controls around AI integrations. "Policy says don't paste customer data into ChatGPT" gets zero credit as a control. What they want to see is technical enforcement with logging — something you can point to and say "here is the record of every outbound prompt from the last 90 days, here is what was flagged." If you can't produce that, it's a finding.
The other gap that's easy to miss: most teams don't know what's sensitive until it's already in a prompt. The classification problem comes before the proxy. What are your teams actually pulling into their prompts right now — internal docs, support tickets, code with credentials, database outputs? That's usually where the real exposure surfaces.
A breach at this scale almost never comes from a single access event — moving a petabyte takes time, and that kind of sustained egress usually means either the detection tooling wasn't watching outbound data flows, or alerts fired and got buried in noise. "1 petabyte" from a hacker claim is probably inflated, but even 5-10% of that is catastrophic depending on what's in it.
What's worth paying attention to here is that Telus Digital is a BPO/outsourcing company, which means the blast radius almost certainly extends to their clients. If your company has API integrations with Telus Digital, or gave them any kind of federated access to internal systems, now is the time to audit what they held and rotate anything they could have touched. Downstream credential exposure in third-party breaches is consistently underreacted to.
The employee data angle is also interesting that usually means developer workstations and internal tooling were in scope, not just the customer-facing layer. Makes the "how did detection miss it" question even harder. Does anyone know if Telus Digital ran a shared SOC across their outsourcing clients, or was each client siloed?
The part that doesn't show up in these posts is what happens when the AI-generated code meets real users. Works fine in dev, clean in staging, then production throws an edge case the model never saw and you're staring at a 3am incident with no mental model of what's actually running.
Vibe coding is genuinely useful for prototyping — I use it. But there's a difference between using AI to move faster and outsourcing your understanding to it entirely. The second one catches up with you the moment something breaks and you have no instinct about where to look.
Curious what the actual product was — did it make it to users before things fell apart, or did it not get that far?