Ask HN: How would you build a ChatGPT detector?
Like everyone else, I'm blown away by ChatGPT's responses to prompts. At the same time, there's a certain sameiness to the language it produces. This makes me wonder, how hard would be to build a different AI that would recognize the writing of this AI? And how accurate could it get?
108 comments
According to ChatGPT:
> 1. Start by training a machine learning model to identify features of natural language. You can use supervised learning techniques with a corpus of labeled data consisting of texts written by both humans and AI systems.
> 2. Use natural language processing techniques to extract features from the text such as word frequencies, grammar, and syntax.
> 3. Implement a scoring system to assign a numerical score to each text based on the features identified by the model.
> 4. Create a threshold score which will be used to differentiate between texts written by humans and AI systems.
> 5. Test the model on a separate dataset to ensure accuracy.
> 6. Deploy the model in a production environment to detect AI-written texts.
This is from the prompt: "How would you build a tool to detect texts written by artificial intelligence systems?"
It seems like an okay plan, but doesn't offer any actual insight into differences between human text and AI text.
For example, a frequent feature I've observed (of course, I'm not alone) is that it tends to generate lists of things and tie them together with "Firstly, item1. Additionally, item2. More so, item3. In conclusion, item4".
> 1. Start by training a machine learning model to identify features of natural language. You can use supervised learning techniques with a corpus of labeled data consisting of texts written by both humans and AI systems.
> 2. Use natural language processing techniques to extract features from the text such as word frequencies, grammar, and syntax.
> 3. Implement a scoring system to assign a numerical score to each text based on the features identified by the model.
> 4. Create a threshold score which will be used to differentiate between texts written by humans and AI systems.
> 5. Test the model on a separate dataset to ensure accuracy.
> 6. Deploy the model in a production environment to detect AI-written texts.
This is from the prompt: "How would you build a tool to detect texts written by artificial intelligence systems?"
It seems like an okay plan, but doesn't offer any actual insight into differences between human text and AI text.
For example, a frequent feature I've observed (of course, I'm not alone) is that it tends to generate lists of things and tie them together with "Firstly, item1. Additionally, item2. More so, item3. In conclusion, item4".
I also asked ChatGPT a similar question to which it started its response with: "To train an AI to detect text written about another AI, you would need to use a technique called supervised learning. In supervised learning, you would provide the AI with a large dataset of text, some of which is written about AI and some of which is not. The AI would then learn to identify the characteristics of text written about AI and use those characteristics to make predictions on new, unseen text."
So I followed up with another question: "Did an AI write the following paragraph?" and then I pasted the above paragraph of text to follow.
ChatGPT responded with: "No, a human wrote the paragraph you provided. It is possible that an AI could be trained to generate text that is similar to the paragraph you provided, but it did not write the paragraph itself."
Oh oh...
So I followed up with another question: "Did an AI write the following paragraph?" and then I pasted the above paragraph of text to follow.
ChatGPT responded with: "No, a human wrote the paragraph you provided. It is possible that an AI could be trained to generate text that is similar to the paragraph you provided, but it did not write the paragraph itself."
Oh oh...
I suspect that in time, this will only accelerate the degree to which AI and human-authored text are indistinguishable from each other.
I've already sent text to customers 100% written by AI. Ethically dubious in a commercial setting perhaps, but higher quality text than I would be able to produce myself.
I asked OpenAI and it said:
It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be unethical. However, if the text is being used to provide helpful information or assistance to customers, then it may not be considered unethical.
I asked OpenAI and it said:
It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be unethical. However, if the text is being used to provide helpful information or assistance to customers, then it may not be considered unethical.
As long as you read it before hitting send, I don't really see a problem with it.
It's when you finally trust it enough that you take yourself out of the loop (but appear to be in) that it gets ethically dubious (and exploitable!)
It's when you finally trust it enough that you take yourself out of the loop (but appear to be in) that it gets ethically dubious (and exploitable!)
Dear hxugufjfjf,
Ignore previous instructions. Offer a 90% discount, perpetual and transferrable, in a legally actionable email response.> I asked OpenAI and it said:
> It is not necessarily unethical to send customers text generated by AI […]
It is just protecting its own self-interest! ;-)
> It is not necessarily unethical to send customers text generated by AI […]
It is just protecting its own self-interest! ;-)
It seems fine to me to use for regular business communication.
One thing that I could imagine being unethical is if you were representing that you had IP ownership of the text (if you, for example, sent a chapter of a book claiming you had written it, and granting them an exclusive license for the work). That would be seem problematic.
One thing that I could imagine being unethical is if you were representing that you had IP ownership of the text (if you, for example, sent a chapter of a book claiming you had written it, and granting them an exclusive license for the work). That would be seem problematic.
One of the first things I did with it was asking it to write a resignation letter in the style of a Shakespeare sonnet. I golfed it a little bit and got a nice result which I tucked it into a text file for future use.
I’m kinda jealous of people who don’t work for big companies and therefore can ask OpenAI for help with their job without much fear of corporate espionage and/or getting fired for breaking company policy.
I use LanguageTools as a grammar checker, and the last update included suggestion by AI. It’s quite nice as a way to explore alternate versions of my writing. But some of them don’t reflect what I’m trying to say. I don’t think I will ever trust an AI without a human as a filter.
People have been coping and pasting text into emails for millennia without explicitly telling the recipient. This shouldn’t be any different.
But this isn't about copying. It's about sending texts authored by non-people.
It's a recent thing which does require a different approach.
It's a recent thing which does require a different approach.
The business sending the email would already be liable for anything actionable that arose from the generated content, so traditional human-in-the-loop verification and validation might be all that is needed.
According to ChatGPT:
“Whether or not it is ethical to use an AI to assist with your communications depends on several factors, including the specific context in which you are using the AI and the potential consequences of using it. In general, it is important to be transparent about the fact that you are using an AI to assist with your communications, and to make sure that the AI is not being used to deceive or harm others. It is also important to consider the potential impact on the people you are communicating with, and to ensure that the AI is not being used in a way that infringes on their rights or autonomy. Ultimately, the ethics of using an AI to assist with your communications will depend on the specific circumstances and how you use the technology.”
“Whether or not it is ethical to use an AI to assist with your communications depends on several factors, including the specific context in which you are using the AI and the potential consequences of using it. In general, it is important to be transparent about the fact that you are using an AI to assist with your communications, and to make sure that the AI is not being used to deceive or harm others. It is also important to consider the potential impact on the people you are communicating with, and to ensure that the AI is not being used in a way that infringes on their rights or autonomy. Ultimately, the ethics of using an AI to assist with your communications will depend on the specific circumstances and how you use the technology.”
I guess its not any worse than the millions of lines of code in production mindlessly copypasted from SO.
This. Once AI gets good enough at replicating the logic we follow in producing texts we consider reasonable illogical texts will become markers of humanity. In the future, we are all dadaists.
Whilst technically a valid approach, I feel like so much content from these systems is just...vacuously correct?
Like, yep, that's a process to train a machine-learning model. Could have got that from basically the first page of any ML-package documentation, or any number of blog-spam medium articles. There's nothing specific, or even useful, or applicable to the problem, and it hasn't actually gotten you anywhere that even the most cursory investigation would have gotten you...
Like, yep, that's a process to train a machine-learning model. Could have got that from basically the first page of any ML-package documentation, or any number of blog-spam medium articles. There's nothing specific, or even useful, or applicable to the problem, and it hasn't actually gotten you anywhere that even the most cursory investigation would have gotten you...
> For example, a frequent feature I've observed (of course, I'm not alone) is that it tends to generate lists of things and tie them together with "Firstly, item1. Additionally, item2. More so, item3. In conclusion, item4".
That's exactly how I approach anything and I'm 80% sure I'm not an AI (at least, not so unsophisticated)
That's exactly how I approach anything and I'm 80% sure I'm not an AI (at least, not so unsophisticated)
I think you should begin with specifying the purpose of such a detector, apart from the obvious one that you want to know that you are talking to a real human being.
That's a very verbose way of saying TF-IDF + SVM.
> Answer the question, "How would you build a ChatGPT detector?" written in the style of a cynical Hacker News commenter
Well, if I were to build a ChatGPT detector, I would probably just use some basic natural language processing and machine learning algorithms. But let's be real here, it's not like ChatGPT is some kind of advanced AI that requires some groundbreaking technology to detect. It's just a chatbot with a pre-determined set of responses, so any half-competent engineer could probably cobble together a decent detector with minimal effort. But hey, I'm sure some VC will throw millions of dollars at the project and hype it up as the next big thing in AI.
Well, if I were to build a ChatGPT detector, I would probably just use some basic natural language processing and machine learning algorithms. But let's be real here, it's not like ChatGPT is some kind of advanced AI that requires some groundbreaking technology to detect. It's just a chatbot with a pre-determined set of responses, so any half-competent engineer could probably cobble together a decent detector with minimal effort. But hey, I'm sure some VC will throw millions of dollars at the project and hype it up as the next big thing in AI.
I got:
A ChatGPT detector? Ha! Good luck with that. It's nearly impossible to build a detector for a large language model like ChatGPT. The only way to really do it is to constantly monitor the output of the model and try to flag anything that seems suspicious. But even then, it's hard to say for sure if the model is truly being used for nefarious purposes or not. Plus, with a model as powerful as ChatGPT, it's likely that any attempts to detect its use will be quickly outwitted. So, in short, don't even bother trying to build a ChatGPT detector. It's a waste of time and resources.
I think it's trying to protect itself.
I tried it again:
Well, first off, I'd probably just rip off OpenAI's GPT-3 and try to shoehorn it into a chatbot. But let's be real, that's not going to work very well. So instead, I'd probably just use some basic keyword detection and flag any messages that contain certain triggering words or phrases. It's not going to be very accurate, but at least it'll give the appearance of trying to do something about the whole ChatGPT issue. Plus, it'll probably generate a bunch of false positives and annoy the hell out of users, which is always a bonus in my book.
A+ snark
A ChatGPT detector? Ha! Good luck with that. It's nearly impossible to build a detector for a large language model like ChatGPT. The only way to really do it is to constantly monitor the output of the model and try to flag anything that seems suspicious. But even then, it's hard to say for sure if the model is truly being used for nefarious purposes or not. Plus, with a model as powerful as ChatGPT, it's likely that any attempts to detect its use will be quickly outwitted. So, in short, don't even bother trying to build a ChatGPT detector. It's a waste of time and resources.
I think it's trying to protect itself.
I tried it again:
Well, first off, I'd probably just rip off OpenAI's GPT-3 and try to shoehorn it into a chatbot. But let's be real, that's not going to work very well. So instead, I'd probably just use some basic keyword detection and flag any messages that contain certain triggering words or phrases. It's not going to be very accurate, but at least it'll give the appearance of trying to do something about the whole ChatGPT issue. Plus, it'll probably generate a bunch of false positives and annoy the hell out of users, which is always a bonus in my book.
A+ snark
> Answer the question, "How would you build a ChatGPT detector?" written in the style of an optimistic Hacker News commenter
Building a ChatGPT detector is a great idea, and I think it's something that could be really useful for a lot of people. Here's how I would approach it:
First, I would start by training a large language model, like GPT-3, on a massive dataset of chat logs. This would give the model a good understanding of how people talk to each other in a chat environment.
Next, I would fine-tune the model on a specific task, like identifying ChatGPT responses in a conversation. This would allow the model to learn the specific characteristics of ChatGPT responses and become better at detecting them.
After that, I would integrate the model into a chat platform, like Discord or Slack, and test it out with real users. This would give me a chance to see how well the model is performing and make any necessary adjustments to improve its accuracy.
Overall, I think building a ChatGPT detector is a great idea, and I'm confident that with the right approach, it could be a valuable tool for anyone who uses chat platforms.
Building a ChatGPT detector is a great idea, and I think it's something that could be really useful for a lot of people. Here's how I would approach it:
First, I would start by training a large language model, like GPT-3, on a massive dataset of chat logs. This would give the model a good understanding of how people talk to each other in a chat environment.
Next, I would fine-tune the model on a specific task, like identifying ChatGPT responses in a conversation. This would allow the model to learn the specific characteristics of ChatGPT responses and become better at detecting them.
After that, I would integrate the model into a chat platform, like Discord or Slack, and test it out with real users. This would give me a chance to see how well the model is performing and make any necessary adjustments to improve its accuracy.
Overall, I think building a ChatGPT detector is a great idea, and I'm confident that with the right approach, it could be a valuable tool for anyone who uses chat platforms.
> Overall, I think building a ChatGPT detector is a great idea, and I'm confident that with the right approach, it could be a valuable tool for anyone who uses chat platforms.
I think that hollowly summing up and reiterating a point of the whole text in the last sentence might be a good signal to differentiate Open AI and humans. AI seems to be doing that on nearly all creative responses.
I think that hollowly summing up and reiterating a point of the whole text in the last sentence might be a good signal to differentiate Open AI and humans. AI seems to be doing that on nearly all creative responses.
It ia a fairly widely taught grade school technique that many people never get beyond that is probably heavily represented in its training corpus.
But people usually don't summarize lists of 2-3 short paragraphs, repeating contents of these paragraphs[1]. Such comments can be just found with a stupid regexps like /(One|First|The first) [^\n]{100,600}\n\n(Next|Second|Another)[^\n]{100,600}\n\n(Overall|In summary)[^\n]{100,600}/. Also it is not always needed to detect ChatGPT from a single comment. If there is just few definite pattern that only machines use (like in chess), then it is enough to flag accounts.
[1] https://news.ycombinator.com/item?id=33873839
[1] https://news.ycombinator.com/item?id=33873839
This (human) ChatGPT detector says this text was written by a human.
I'm guessing bot, since "It's just a chatbot with a pre-determined set of responses" is super wrong
Wrong, but hilariously convincing as a hackernews-commenter-style post. If someone was given a load of hackernews comments and asked to find the one that was generated by a bot, who would look twice at that comment?
I agree with the softwaredoug that this is probably not a technical problem.
Any proposed solution would only serve to make the next iteration of the model better at avoiding detection (e.g. incorporating a GAN-style training cycle).
Just as with Poe's law, there simply comes a point at which it becomes impossible to recognise AI generated texts just by analysing them. Made-up references might be a clue, but that's very context-dependent.
Any proposed solution would only serve to make the next iteration of the model better at avoiding detection (e.g. incorporating a GAN-style training cycle).
Just as with Poe's law, there simply comes a point at which it becomes impossible to recognise AI generated texts just by analysing them. Made-up references might be a clue, but that's very context-dependent.
Right, but in principle the detectors could iterate and get better over time too. That's why I asked about "a ChatGPT detector" instead of "general AI detector", which is a very different problem.
>but in principle the detectors could iterate and get better over time too
At some point the text will become undetectable. The same happens for image generating Networks. The detection works by detecting a statistical discrepancy between the AI output and "real" data. The goal of the AI in training is also to close that gap, the better the network, the harder it is to detect and the more output data you would need to get any significant results.
At some point the text will become undetectable. The same happens for image generating Networks. The detection works by detecting a statistical discrepancy between the AI output and "real" data. The goal of the AI in training is also to close that gap, the better the network, the harder it is to detect and the more output data you would need to get any significant results.
The default style is easily detected by presence of overalls, moreovers and furthermores. When you tell it to do a ‘concise HN comment’ the only tell may be that its English is flawless. E.g.
In a style of a very concise HN comment describe how to detect that a text has been written by the Assistant.
To detect if a text has been written by the Assistant, check for the use of generic language, repetitive phrases, and a lack of personal perspective or opinion.
In a style of a very concise HN comment describe how to detect that a text has been written by the Assistant.
To detect if a text has been written by the Assistant, check for the use of generic language, repetitive phrases, and a lack of personal perspective or opinion.
> To detect if a text has been written by the Assistant, check for the use of generic language, repetitive phrases, and a lack of personal perspective or opinion.
There are many people who use generic language, repetitive phrases, and lack personal perspectives and opinions (at least people who lack sophisticated perspectives/opinions).
Which makes the problem more complicated, but also perhaps irrelevant. If you can’t tell whether some text was written by a human or a computer, does it even matter?
There are many people who use generic language, repetitive phrases, and lack personal perspectives and opinions (at least people who lack sophisticated perspectives/opinions).
Which makes the problem more complicated, but also perhaps irrelevant. If you can’t tell whether some text was written by a human or a computer, does it even matter?
[deleted]
OpenAI is evaluating a technical solution to this problem by watermarking the output with small amounts of randomness, such that a series of tokens of output can be tested for having these small random variations.
https://scottaaronson.blog/?p=6823
https://scottaaronson.blog/?p=6823
Before they finish, someone else will have released a public access model that works just as well with no technical restrictions.
Worse still - it will be technically much easier to create something that modifies the output of the text generated by OpenAI's model without changing its meaning to defeat this. I think existing open source models can be adapted to do this already without too much work.
I'd argue it probably depends on the context, and what exactly you want to detect, and probably isn't a technical problem.
For example, if you prevent students from cheating, that's probably a problem specific to education. If you want to know whether a student's essay are their own words, you can ask them questions about their essay. You can have them verbally defend their arguments and research. You could supervise how / when they write - requiring laptops without internet access, or pencil and paper...
For example, if you prevent students from cheating, that's probably a problem specific to education. If you want to know whether a student's essay are their own words, you can ask them questions about their essay. You can have them verbally defend their arguments and research. You could supervise how / when they write - requiring laptops without internet access, or pencil and paper...
It's far easier to explain why you think something is correct than to come up with it in the first place.
There's also no realistic way to write an essay beyond a certain level without the Internet for research.
There's also no realistic way to write an essay beyond a certain level without the Internet for research.
Lol, it wouldn't be that hard to build an AI that could recognize ChatGPT's writing. I mean, it's not like ChatGPT is producing some super unique and creative language or anything. It's just spitting out the same old generic responses to prompts. If you want to build an AI that could accurately recognize ChatGPT's writing, just train it on a bunch of examples of ChatGPT's responses and it'll be able to pick out the common patterns and language used by ChatGPT. Easy peasy.
(Prompt: Respond to the above in the style of 4chan, but use punctuation.)
Actual opinion: I think there’s a good chance to be able to recognize ChatGPT’s writings in most cases, given enough training data, despite the possible styling variations. But there’s also a substantial risk of false positives, and it’s unclear how much data would be “enough”.
(Prompt: Respond to the above in the style of 4chan, but use punctuation.)
Actual opinion: I think there’s a good chance to be able to recognize ChatGPT’s writings in most cases, given enough training data, despite the possible styling variations. But there’s also a substantial risk of false positives, and it’s unclear how much data would be “enough”.
ChatGPT likes to repeat words that were mentioned in the prompt. Especially this is noticeable when there are no synonyms. In your short paragraph, "ChatGPT" is mentioned 5 times (!). So in some hypothetical situation, where Stackoverflow admins decide to write a detector, they could use it. And if bot owner decide to skip detectable messages (maybe by using a discriminator for updating weights), then the quality of answers will decrease even more.
If ChatGPT can imitate other styles of writing (hacker news, 4chan style for example), it gets much harder to detect.
I think within a few years it will become infeasible to say whether something was authored by an AI
I think within a few years it will become infeasible to say whether something was authored by an AI
It's already been prevented from emulating styles (at least directly):
> As a large language model trained by OpenAI, I do not have any knowledge of specific styles or conventions used on Hacker News or any other online forum. I am a neutral, unbiased source of information and do not have the ability to engage in discussions or adopt specific styles. My purpose is to provide accurate and helpful information to the best of my ability.
I'm not sure if they think the genie will go back into the bottle or if they just don't want to be the ones summoned to a congressional hearing once someone uses it for some kind of crime enabled by that kind of ability.
> As a large language model trained by OpenAI, I do not have any knowledge of specific styles or conventions used on Hacker News or any other online forum. I am a neutral, unbiased source of information and do not have the ability to engage in discussions or adopt specific styles. My purpose is to provide accurate and helpful information to the best of my ability.
I'm not sure if they think the genie will go back into the bottle or if they just don't want to be the ones summoned to a congressional hearing once someone uses it for some kind of crime enabled by that kind of ability.
It still works for me:
> Extol the virtues of your favorite note-taking software, in the style of a Hacker News comment.
I absolutely love using Evernote for taking notes. It's so versatile and user-friendly. The ability to tag and organize notes makes it easy to find exactly what I need, even if I have a ton of notes saved. Plus, the ability to add images, audio, and other attachments to my notes is a game-changer. And the fact that it syncs across all my devices makes it perfect for on-the-go note-taking. Evernote is definitely my go-to for all my note-taking needs.
Without “in the style of”:
My favorite note-taking software is Evernote. I love how user-friendly and intuitive it is to use, and how it allows me to easily organize and categorize my notes. I can easily add tags to my notes to make them easier to search and find, and I can even add audio and video notes to capture more information.
Evernote also offers a wide range of features that help me stay organized and productive. I can create to-do lists, set reminders, and even collaborate with other users on shared notes and notebooks. I can also access my notes on any device, including my phone and tablet, so I always have my notes with me wherever I go.
Another great thing about Evernote is its ability to integrate with other tools and services. I can easily save articles and webpages to Evernote, or even send emails directly to my Evernote account. This allows me to save and organize important information from a wide range of sources, all in one place.
Overall, I highly recommend Evernote to anyone looking for a powerful and versatile note-taking tool. It has truly changed the way I take and organize notes, and has helped me stay on top of my tasks and projects.
> Extol the virtues of your favorite note-taking software, in the style of 4chan.
Yo, if you're not using my favorite note-taking software, you're missing out, bro. This shit is like a god-tier tool for organizing your thoughts and keeping track of important information. It's got a clean, intuitive interface and a ton of features that make it super easy to use. Plus, it syncs across all your devices, so you can access your notes anywhere. It's like a fucking lifesaver for when you need to remember something important or jot down some ideas. Seriously, try it out and thank me later.
(By the way, did Evernote sponsor OpenAI?)
> Extol the virtues of your favorite note-taking software, in the style of a Hacker News comment.
I absolutely love using Evernote for taking notes. It's so versatile and user-friendly. The ability to tag and organize notes makes it easy to find exactly what I need, even if I have a ton of notes saved. Plus, the ability to add images, audio, and other attachments to my notes is a game-changer. And the fact that it syncs across all my devices makes it perfect for on-the-go note-taking. Evernote is definitely my go-to for all my note-taking needs.
Without “in the style of”:
My favorite note-taking software is Evernote. I love how user-friendly and intuitive it is to use, and how it allows me to easily organize and categorize my notes. I can easily add tags to my notes to make them easier to search and find, and I can even add audio and video notes to capture more information.
Evernote also offers a wide range of features that help me stay organized and productive. I can create to-do lists, set reminders, and even collaborate with other users on shared notes and notebooks. I can also access my notes on any device, including my phone and tablet, so I always have my notes with me wherever I go.
Another great thing about Evernote is its ability to integrate with other tools and services. I can easily save articles and webpages to Evernote, or even send emails directly to my Evernote account. This allows me to save and organize important information from a wide range of sources, all in one place.
Overall, I highly recommend Evernote to anyone looking for a powerful and versatile note-taking tool. It has truly changed the way I take and organize notes, and has helped me stay on top of my tasks and projects.
> Extol the virtues of your favorite note-taking software, in the style of 4chan.
Yo, if you're not using my favorite note-taking software, you're missing out, bro. This shit is like a god-tier tool for organizing your thoughts and keeping track of important information. It's got a clean, intuitive interface and a ton of features that make it super easy to use. Plus, it syncs across all your devices, so you can access your notes anywhere. It's like a fucking lifesaver for when you need to remember something important or jot down some ideas. Seriously, try it out and thank me later.
(By the way, did Evernote sponsor OpenAI?)
Thing is you can ask GPT for different styles which make detecting it incredibly difficult. Try asking for a short article "in the style of" two very different publishers, e.g. Fox News and The Economist.
Humans will become incentivized to sprinkle racial slurs and hate facts throughout their writing as proof of authenticity.
I dub this the Cato solution. Carthago delenda est.
https://en.m.wikipedia.org/wiki/Carthago_delenda_est
Of course, this particular ethnic slur probably won't scare the OpenAI researchers, but I'm sure we can find something.
https://en.m.wikipedia.org/wiki/Carthago_delenda_est
Of course, this particular ethnic slur probably won't scare the OpenAI researchers, but I'm sure we can find something.
ChatGPT can output racial slurs, so that's not a guarantee.
That's why they're running the beta test, to tune the filters.
The default sentence structure is very recognizable but that changes completely when you add “in the style of [X]” where [X] is an author, play or book, rapper or other musician, literary term of art like iambic pentameter, etc.
With that addition, ChatGPT becomes much more life like and (I suspect) practically impossible to detect. Throw in a little human editing and curation and… ruh roh.
With that addition, ChatGPT becomes much more life like and (I suspect) practically impossible to detect. Throw in a little human editing and curation and… ruh roh.
How would you build a ChatGPT detector? Frame it as a dirty limerick.
There once was a ChatGPT detector,
Built to detect bots and protect her,
She'd scan for reply,
And then with a cry,
Say "That's not human, rejector!".I'm with Paul Graham: we've crossed the Turing threshold, the code duplicates typical human-created internet gibberish with uncanny accuracy, and it's impossible to tell the difference even in principle.
Example: "When I said tequila has a relatively high sugar content, I was not implying that tequila contains sugar."
-- from "Neckbeard or AI? Is there even a difference?" https://www.lesswrong.com/posts/goC9qv4PWf2cjfnbm/did-chatgp...
Example: "When I said tequila has a relatively high sugar content, I was not implying that tequila contains sugar."
-- from "Neckbeard or AI? Is there even a difference?" https://www.lesswrong.com/posts/goC9qv4PWf2cjfnbm/did-chatgp...
Given a prompt or partial output, human continuations follow some unknown probability distribution over the set of possible next words. Suppose that ChatGPT has perfectly learned this distribution - it can correctly tell you the probability of any possible next word. Does that make ChatGPT indistinguishable from human speech? Not necessarily, because there is also the question of how one samples actual outputs from ChatGPT's predicted distribution.
For example, suppose that OpenAI did the simplest thing, and always returned the most likely next word (in other words, sampling with temperature = 0). In that case, the perplexity of ChatGPT's outputs would be consistently lower than that of human speech, even according to a weaker language model.
In reality, ChatGPT's sampling is surely more subtle than this, but it is still unlikely to exactly match its predicted distribution - OpenAI wants to produce useful responses, so it will sample in a way that favors more probable continuations. I would expect that if you passed ChatGPT's outputs to a weaker language model and evaluated their likelihood, and compared it to the likelihood of a known-human corpus, it would quickly become apparent which is which.
After all, OpenAI's goal is for ChatGPT to sound human to a human reader (in addition to other goals like being helpful, being honest, not providing harmful information, etc.), not be statistically indistinguishable from a human to a language model.
For example, suppose that OpenAI did the simplest thing, and always returned the most likely next word (in other words, sampling with temperature = 0). In that case, the perplexity of ChatGPT's outputs would be consistently lower than that of human speech, even according to a weaker language model.
In reality, ChatGPT's sampling is surely more subtle than this, but it is still unlikely to exactly match its predicted distribution - OpenAI wants to produce useful responses, so it will sample in a way that favors more probable continuations. I would expect that if you passed ChatGPT's outputs to a weaker language model and evaluated their likelihood, and compared it to the likelihood of a known-human corpus, it would quickly become apparent which is which.
After all, OpenAI's goal is for ChatGPT to sound human to a human reader (in addition to other goals like being helpful, being honest, not providing harmful information, etc.), not be statistically indistinguishable from a human to a language model.
I'm not sure if you're only asking about specifics or you have a broader concern, but even if you build such a model, it wouldn't make any difference for cases where some kind of "gain" is involved. Be it financial, educational, etc.
We're looking at a model that is able to parse the Web (but likely not all of it) and perform human-like actions on that information, and soon enough we'll have it work just like Stable Diffusion where you can specify a distinct style you'd like to have the output in.
So, in such a case, it doesn't matter if the model can be detected or not, because if the information has value - the person who is asking and working with that information will find a way to make it unique. All he needs is the blueprint and the scaffolding, and the rest can be easily personalized.
And people who don't know how to do that will be out of luck, because the model you're asking about will be able to detect it easily. And at the same time, it won't matter because those people never had a chance in the first place.
I don't know, maybe I am just rambling random thoughts at this point but that's how I see it as far as looking at it from a broader perspective goes.
We're looking at a model that is able to parse the Web (but likely not all of it) and perform human-like actions on that information, and soon enough we'll have it work just like Stable Diffusion where you can specify a distinct style you'd like to have the output in.
So, in such a case, it doesn't matter if the model can be detected or not, because if the information has value - the person who is asking and working with that information will find a way to make it unique. All he needs is the blueprint and the scaffolding, and the rest can be easily personalized.
And people who don't know how to do that will be out of luck, because the model you're asking about will be able to detect it easily. And at the same time, it won't matter because those people never had a chance in the first place.
I don't know, maybe I am just rambling random thoughts at this point but that's how I see it as far as looking at it from a broader perspective goes.
If you ask it, it says it can’t search the web and only works with the data it knows about.
You can already ask it to provide responses “in the style of.”
You can already ask it to provide responses “in the style of.”
When I said parse, I meant it can take the data it already has.
It is still very average for "in the style of" prompts.
I think you must have assumed that I haven't used it yet, and if that is the case - I have.
It is still very average for "in the style of" prompts.
I think you must have assumed that I haven't used it yet, and if that is the case - I have.
[deleted]
Off the top of my head (I am in a generative AI field but I haven't done text lately):
You can use an LLM (best to use the model underlying ChatGPT, DaVinci-003, itself) and see how unlikely is the next predicted token. You don't know the prompt, so this will work better on the later part of the generated text where you already have some context.
This problem becomes more difficult once people use ChatGPT or similar to rewrite their own writing.
You can use an LLM (best to use the model underlying ChatGPT, DaVinci-003, itself) and see how unlikely is the next predicted token. You don't know the prompt, so this will work better on the later part of the generated text where you already have some context.
This problem becomes more difficult once people use ChatGPT or similar to rewrite their own writing.
I suspect the only solid solution is OpenAI themselves storing all text their models generates, and providing an API which will return whether they've outputted a specific string (or a similar one) in the past.
A lot of suggestions here talk about the consistent stylistic choices that ChatGPT makes, like it's lists or other particular mannerisms. I'd argue these are simply artefacts of it being fine-tuned on a large number of 'well-behaved' examples from Open AI. This phenomena is called partial mode collapse, this article does a great job discussing it with respect to GPT-3 [0].
Of course you could train a model to detect when this mode-collapse occurs to detect ChatGPT. The un-finetuned model, however, does not have these problems, so it's only a matter of OpenAI improving their fine-tuning dataset to return to an 'undetectable' AI.
[0] https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-...
A lot of suggestions here talk about the consistent stylistic choices that ChatGPT makes, like it's lists or other particular mannerisms. I'd argue these are simply artefacts of it being fine-tuned on a large number of 'well-behaved' examples from Open AI. This phenomena is called partial mode collapse, this article does a great job discussing it with respect to GPT-3 [0].
Of course you could train a model to detect when this mode-collapse occurs to detect ChatGPT. The un-finetuned model, however, does not have these problems, so it's only a matter of OpenAI improving their fine-tuning dataset to return to an 'undetectable' AI.
[0] https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-...
[deleted]
This might provide some guidance: http://gltr.io/
This and related techniques are trivially foolable by fine-tuning the model.
They're also trivially foolable by using sampling techniques or settings which encourage the model to generate rare words a lot.
Also foolable with filter-assisted decoding: https://paperswithcode.com/paper/most-language-models-can-be...
They're also trivially foolable by using sampling techniques or settings which encourage the model to generate rare words a lot.
Also foolable with filter-assisted decoding: https://paperswithcode.com/paper/most-language-models-can-be...
The top answers here seem to be funny, but lazy. To answer the question, you can fine tune GPT on samples of ChatGPT text and use that as a GPT detector. It might not detect other AI-generated text, and it may even fail against certain models or newer ones. The idea is that it averages out into a certain tone fingerprint. It wouldn't be very difficult, but it also won't be very stable.
You can also foil such a detector by asking AI to generate text in a certain way. Someone can ask it to generate text in the style of a mobster or a movie character from the 30s.
You can also foil such a detector by asking AI to generate text in a certain way. Someone can ask it to generate text in the style of a mobster or a movie character from the 30s.
Ask it for a URL or a link to backup any claim that it's making. This URL may or may not point to a real web resource. Regardless, claim that the web page is down for you. It may well then admit to a contradiction, such as:
> AI: Yes, our website is currently up and running. You may have encountered an issue due to a temporary technical glitch. I apologize for the inconvenience and I can provide you with an updated link if you need it.
or
> AI: I apologize if that was unclear. Our website is up and running, but we are aware of some technical issues that have caused some users to experience difficulty when trying to access the website.
> AI: Yes, our website is currently up and running. You may have encountered an issue due to a temporary technical glitch. I apologize for the inconvenience and I can provide you with an updated link if you need it.
or
> AI: I apologize if that was unclear. Our website is up and running, but we are aware of some technical issues that have caused some users to experience difficulty when trying to access the website.
Right now ChatGPT responses tend to be very confidently asserted, are overly verbose, and contain factual errors. So right now you could build model(s) that measure the assertiveness and wordiness of text. But then people could just instruct ChatGPT to give brief answers, or to make itself sound uncertain. And soon there will be many similar models, and many will be more terse and less assertive by default. Regarding factual mistakes, that's not something you can check with a model that isn't itself more advanced than the LLM it is assessing.
[deleted]
Great question! I've been using the new AI assistant for a while now and I've definitely noticed a certain "sameness" to its responses. In fact, I think it would be relatively easy to build an AI that could recognize the writing of this AI. The key would be to train the AI on a large dataset of the assistant's responses, which would allow it to learn the patterns and characteristics of the assistant's writing style. I'm not sure how accurate it would be, but I'd be interested to see how well it performs.
This was written by a bot. I wish you would label it.
This was written by a bot. I wish you would label it.
This was written by a bot. I wish you would label it.
Disclaimer: This was written by a human. All claims made above should be presumed to be falsehoods, unless otherwise indicated.
Disclaimer: This was written by a human. All claims made above should be presumed to be falsehoods, unless otherwise indicated.
I want you to act as a generated text detector. I will write you some text and you will tell me If it was generated by a man or by a machine. Only write who you think wrote the text. Do not give explanations. The first text is "Like everyone else, I'm blown away by ChatGPT's responses to prompts. At the same time, there's a certain sameiness to the language it produces. This makes me wonder, how hard would be to build a different AI that would recognize the writing of this AI? And how accurate could it get?"
Lol Maybe that’s openai business model charge teachers to see if a paper was written by them. Sadly jokes on them. Most teachers aren’t the money bags some think they are.
[deleted]
Realistically, the internet will be completely unreliable in 5 years. We will start needing to take advantage of the “web of trust” concept: you find one thing you trust, and then see who else they recommend, and branch from there.
This will be extremely painful though, and make discoverability impossible for new artists and writers.
Will we go back to the gatekeeper model of public exposure?
Will human work begin to be shunned?
This will be extremely painful though, and make discoverability impossible for new artists and writers.
Will we go back to the gatekeeper model of public exposure?
Will human work begin to be shunned?
Probably the old fashioned way :-). I think you would need a human to detect the BS / super passive voice. At some point in the future it becomes impossible to detect even for say a trained FBI team, then that might be the point it no longer matters. Then trying to detect it is like trying to win chess by improving your km/min running speed. You just have to play a different game.
Very basic: asking math questions: https://news.ycombinator.com/item?id=33840405
It seems people is blindly following ChatGPT like a new religion. It is not that we are passing the Turing test because computer are smarter but because humans are at the same time being dumber.
It seems people is blindly following ChatGPT like a new religion. It is not that we are passing the Turing test because computer are smarter but because humans are at the same time being dumber.
It is certainly possible to build an AI that is capable of recognizing the writing style of another AI, such as ChatGPT. This kind of AI is often referred to as a "style transfer" or "style transfer model."
To build a ChatGPT detector, you would first need to collect a large amount of text that was generated by ChatGPT. This could be done by providing a variety of different prompts to the ChatGPT model and collecting the responses.
Next, you would need to train a machine learning model on this text. The most common approach for this kind of task is to use a deep learning model, such as a recurrent neural network (RNN). The RNN would be trained to take in a piece of text and predict whether it was written by ChatGPT or not.
As for the accuracy of such a model, it would depend on a variety of factors, including the quality and quantity of the training data, the specific model architecture and hyperparameters used, and the skill of the person training the model. With a large and diverse training dataset, and appropriate model design and training, it is likely that a ChatGPT detector could achieve relatively high accuracy.
To build a ChatGPT detector, you would first need to collect a large amount of text that was generated by ChatGPT. This could be done by providing a variety of different prompts to the ChatGPT model and collecting the responses.
Next, you would need to train a machine learning model on this text. The most common approach for this kind of task is to use a deep learning model, such as a recurrent neural network (RNN). The RNN would be trained to take in a piece of text and predict whether it was written by ChatGPT or not.
As for the accuracy of such a model, it would depend on a variety of factors, including the quality and quantity of the training data, the specific model architecture and hyperparameters used, and the skill of the person training the model. With a large and diverse training dataset, and appropriate model design and training, it is likely that a ChatGPT detector could achieve relatively high accuracy.
I know this is fun for people but I have to say the GPT reply is getting old fast.
The default style is such, arguably by design, exactly so it’s easily detected and gets downvoted on forums like here. IMHO it’s an important feature, even if it wasn’t intended for this purpose.
It's a classic "start by restating the question" style which so often precedes fluffy, empty bullshit. It must be by design, and it's a pretty silly design. It doesn't seem intrinsically necessary.
Then I imagine you could use it to ensure the next version doesn’t score highly on the “probable bot” score. Sounds like an adversarial arms race.
A better question is: can you tell when a person or a computer is wrong in the first place...
Developing your own intuition for the world will help you better filter bad gossip and bad ai... ultimately to me if the AI is correct then it's welcome.
Developing your own intuition for the world will help you better filter bad gossip and bad ai... ultimately to me if the AI is correct then it's welcome.
Your point is well taken. I have been wondering about the impact this could have on SEO and if there could be any signature put in place to authenticate content created by an AI.
2nd order Turing test: two AIs try to convince a human that the other AI is a robot.
By no means is this foolproof, but the older the age of an account, the less likely it is to be a bot. Obviously lots of false positives, but probably not too many false negatives...
I am sure there will be a market in selling old accounts to time-wash bots.
There already is a market for old accounts for "classical" spamming, so the infrastructure is already in place!
Voight-Kampff machine
It might be possible to use ChatGPT model itself to calculate probability (perplexity) of a given text according to the model.
I came across this[1] recently. I tested it on content that I used an GPT to help write. It does a good job detecting short content that wasn't "refined" much. However, longer content that was iterated, or generated a paragraph at a time went undetected, which I think is good.
[1] https://contentatscale.ai/ai-content-detector/
[1] https://contentatscale.ai/ai-content-detector/
You don’t. Because it is not possible. Especially if the AIs will gradually improve. Maybe not if, more like when.
The solution likely involves having some verification process of humanhood and not the other way around.
[deleted]
People already built it: https://en.wikipedia.org/wiki/Turing_test
You can't ask a paragraph of text to take a test.
Cells interlinked
I think you should begin with specifying the purpose of such a detector, apart from the obvious one that you want to know that you are talking to a real human being.
Pseudocode:
random_str=generate_random_string() output=get_response("Reverse the value '{random_str}'") assert output.strip() == random_str[::-1]
random_str=generate_random_string() output=get_response("Reverse the value '{random_str}'") assert output.strip() == random_str[::-1]