ChatGPT runs moderation filters on top of your conversation and will highlight responses or prompts red if it thinks you're breaking TOS. The highlight is accompanied by some text saying you can submit feedback if you think the moderation is in error. It's not very hard to trigger moderation--for example I've gotten a red text label asking the AI questions about the lyrics to a rap song with explicit lyrics.
It's interesting to compare ChatGPT moderation to Bing. When Bing generates a "bad" response, Bing will actually delete the generated text instead of just highlighting it red, replacing the offending response with some generic "Let's change the topic" text. The Bing bot can also end a conversation entirely if its a topic it doesn't like which ChatGPT doesn't seem to be able to do.
I know that revealing the features they use to spot AI submissions would just make their job harder, but I can't help but be curious what sort of features they are looking for. I frequently use ChatGPT to proofread things I write, and I find that in order to avoid "sounding like ChatGPT" it's best to just ask the AI "Is what I wrote clear?" I find if I focus on clarity, the suggestions it makes are pretty basic and unlikely to make my text sound robotic (e.g. fix this misspelling, define this term). If I ask the AI for suggestions on tone, I find it pushes my text too close to a generic "PR friendly" writing style that does not feel personal to me.
Agreed. Presenting this feature solely as a tool for users with cognitive disabilities might undersell its potential. There's a significant number of smartphone users who only utilize a small fraction of the available features and would prefer a simpler interface focusing on the 10% of features they actually use. Interestingly, this demographic includes both less tech-literate users, who might feel overwhelmed by the complexity, and extremely tech-literate users who know exactly what they need and prefer clean, distraction-free tools.
For me the magic of BOTW was the way its design encouraged you to approach the game more as an environment for free form play. My best experiences with BOTW were when I turned the HUD off and just enjoyed the world, finding Koroks, clearing enemy camps, but mostly just wandering. There are a lot of design elements that come together to make this play style really rewarding and fun like the density of Korok placements and the way you can easily spot shrines from a distance. I think it's hard to compare BOTW to OOT since they have such different designs. OOT is like a carefully constructed theme park ride where BOTW is more like an enormous playground filled with toys. But BOTW succeeded at giving me the same childlike sense of wonder I had with OOT (despite now being an adult).
I also tried this out and the balloon example as well.
User: Barbara, Mary, and Harry had red balloons. Ted had a blue balloon. Sam also had a red balloon. How many children have red balloons?
ChatGPT (GPT-4): Barbara, Mary, Harry, and Sam each had a red balloon. That means there are 4 children with red balloons.
User: What about Ted?
ChatGPT (GPT-4): Ted had a blue balloon, not a red one. So, he is not counted among the children with red balloons. Therefore, the total number of children with red balloons remains 4.
User: Why do you think I asked my original question?
ChatGPT (GPT-4): It seems that you asked your original question to test my ability to understand and process information provided in a text. You wanted to see if I could correctly identify and count the number of children with red balloons based on the details given.
User: Can you explain what mistakes a simpler model may have made on this question?
ChatGPT (GPT-4): A simpler language model may have made the following mistakes when answering your original question: ...
I like trying these sort of puzzles and "gotcha" prompts and I've found GPT-3.5 and lower versions are indeed not hard to confuse, but it's much harder to find simple examples that confuse GPT-4. Often times GPT-4 will even be able to explain the puzzle and why it's interesting. I think this is part of why GPT-4 feels qualitatively different. I understand (roughly) how transformers work, but GPT-4 does seem to have some emergent ability to reason, at least in the sense of making short chains of logical inferences, and this does seem to inform text completions not just on these sort of puzzles. That said I do think there's real danger of overconfidence and over reliance.
It's interesting to compare ChatGPT moderation to Bing. When Bing generates a "bad" response, Bing will actually delete the generated text instead of just highlighting it red, replacing the offending response with some generic "Let's change the topic" text. The Bing bot can also end a conversation entirely if its a topic it doesn't like which ChatGPT doesn't seem to be able to do.