I was able to replicate OP's attack. Since ChatGPT generates images via a separate model, I was able to ask it to tell me what the inputs to the tool was. It's a null prompt: a completely unconditional image generation. What I'm not sure of is if these are the average image trained on that had no prompt in the dataset, or if they are the true average of the dataset during unconditional training step. Very interesting nonetheless, as typically researchers are only able to see the unconditional generation of open weight models.
Surprisingly when you ask ChatGPT to generate you an image with these tool params, the output is not the same; it's not remotely graphic.
Edit: after more debugging the image generator does seem to look at the conversation as part of the input conditioning, so the one word change from OP makes more sense. There seems to be a hidden prompt rewriter that looks at the tool's prompt and the conversation to create the final conditioning for the t2i model.
Algorithmically served short form videos is clearly the smoking of our time. I cannot stand the conservative view of "well we don't know the videos cause mental health decline, or if it's simply those with a genetic inclination who seek out short form content.", exactly mirroring the skeptics about smoking causing cancer. I'm hopeful that in 5-10 years (but more likely 20) people will view this AI served, maximally engaging, content in the same way we view smoking now: disgusting and horrible, but adults should be allowed to do what they want. I can easily imagine kids/teens sharing their illicit access to shorts much in the same way they share vapes/cigarettes, which would be a much more preferable situation than the unlimited use we see today.
Surprisingly when you ask ChatGPT to generate you an image with these tool params, the output is not the same; it's not remotely graphic.
Edit: after more debugging the image generator does seem to look at the conversation as part of the input conditioning, so the one word change from OP makes more sense. There seems to be a hidden prompt rewriter that looks at the tool's prompt and the conversation to create the final conditioning for the t2i model.