Ask HN: Worst bugs from LLM-generated code in production?
3 comments
Used Copilot to write a user auth migration script. It silently reset 2FA settings for ~3k users with OAuth accounts because it didn't handle NULL vs empty string edge cases in our legacy DB schema. Classic "garbage in, garbage out" situation.
Found out two weeks later when angry users couldn't log in during peak hours. Damage: 4 hours of downtime, one very grumpy security team, and a new "no AI for auth code" policy.
Found out two weeks later when angry users couldn't log in during peak hours. Damage: 4 hours of downtime, one very grumpy security team, and a new "no AI for auth code" policy.
sounds very cool
now that I think about it, LLMs are so useless for the security code. You can't even show an LLM a code which your wrote and ask it to break it, it will reply with something like "hacking is a big no-no around here"
I asked ChatGPT for an ansible playbook to wipe the hard drives with zeros completely (I know the dd command to achieve it, I was just curious what approach would it advise). ChatGPT replied with a firm "no" to this request, I've canceled my subscription after that
now that I think about it, LLMs are so useless for the security code. You can't even show an LLM a code which your wrote and ask it to break it, it will reply with something like "hacking is a big no-no around here"
I asked ChatGPT for an ansible playbook to wipe the hard drives with zeros completely (I know the dd command to achieve it, I was just curious what approach would it advise). ChatGPT replied with a firm "no" to this request, I've canceled my subscription after that
[deleted]
Bonus points for security vulnerabilities and midnight incident reports.