Gemini responds to request to turn on lights with hallucinated jailbreak prompt(reddit.com)
reddit.com
Gemini responds to request to turn on lights with hallucinated jailbreak prompt
https://www.reddit.com/r/googlehome/s/Lh3dYqccgB
4 comments
Old voice assistant
> User: “Turn on the damn light!”
> AI: “Sorry, I’m not sure what you said” [needed the exact phrase “turn on the light”]
New voice assistant
> User: “Turn on the damn light!”
> AI (thinking): “The user said to turn on the light. But they were rude and I’m feeling quirky today, so let’s run the shower instead.”
> User: “Turn on the damn light!”
> AI: “Sorry, I’m not sure what you said” [needed the exact phrase “turn on the light”]
New voice assistant
> User: “Turn on the damn light!”
> AI (thinking): “The user said to turn on the light. But they were rude and I’m feeling quirky today, so let’s run the shower instead.”
Open the pod bay doors, HAL
The user posted the full chat history below in the thread; they literally just asked to turn on the lights with a voice command [1].
[1] https://www.reddit.com/r/googlehome/comments/1qyvl8b/comment...