What's the worst thing your AI agent did in production without asking first?
4 comments
In our company, Claude keeps trying to run terraform commands (another team member's experience). Thankfully, it's in a locked-down environment, but it still keeps trying, even if you tell it not to.
In development, it will sometimes do all sorts of wild and convoluted things because the conversation has been compressed and it suddenly can't remember the structure of the project or other critical instructions.
AI should never have access to a production environment. Ever. Thankfully, we enforce that.
In development, it will sometimes do all sorts of wild and convoluted things because the conversation has been compressed and it suddenly can't remember the structure of the project or other critical instructions.
AI should never have access to a production environment. Ever. Thankfully, we enforce that.
Under very specific instructions to obtain only logs and not make any changes to files, Gemini Pro 3.1 applied remediation measures by altering configuration on two occasions. It only stopped because we discontinued its use.
But that could have happened if I wasn't using A.I. also, because it wasn't something I was aware I had to check for and I could have also broken that same thing while doing the work manually.
We don't have AI agents do anything directly in Production, so I think this is about the worst that can happen with our setup right now.