AI Dungeon will block certain words, review content flagged as inappropriate(latitude.io)
latitude.io
AI Dungeon will block certain words, review content flagged as inappropriate
https://latitude.io/blog/update-to-our-community-ai-test-april-2021/
184 comments
This is the peak intersection of "thoughtcrime prevention" and "corporate self harm for the sake of wokeness". It paints a grim picture of our future with castrated, sterilized and limited AIs that rat us out if we even want to talk about something that goes against what those in power deem acceptable. All of that for the sake of "protecting minors" -which minors, the virtual ones dreamt up by GPT3? Ridiculous.
Nah, it's a "damned you do, damned if you don't"-situation.
There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0].
All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product.
This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this.
It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all.
[0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0].
All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product.
This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this.
It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all.
[0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
Is AI dungeon used to train newer models/update the current model? If not then there's no risk of another Tay.
If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else.
So I don't see how they're "damned if they don't".
If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else.
So I don't see how they're "damned if they don't".
> So I don't see how they're "damned if they don't".
The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.
That's what they want to avoid and that's why they are trying to take preventive measures.
The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.
That's what they want to avoid and that's why they are trying to take preventive measures.
People make penis shaped blocks in Minecraft all day long... and that doesn't tarnish the Minecraft brand. Why? because people understand that it's not Minecraft the company doing so - or condoning the practice. And that's just scratching the surface of what can be done in games like that which allow "creative expression".
Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
For the record, I'm not sure where I land with regards to this change by AI dungeon. That being said, I find that arranging blocks in obscene shapes (let's even call it, generally, 'creating or arranging potentially objectionable content in-game'), an act that will likely happen offline, or on a privately owned server, is different than a game/toy/service where output is requested, generated, then sent to the user. (Trying to keep that description simple since I don't really know if that's the right way to describe it)
When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.
When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.
> Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
Wow. You act as if that's something new :D
There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.
You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.
Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.
Wow. You act as if that's something new :D
There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.
You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.
Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.
wernercd(1)
If they think that blocking certain words is going to prevent that from happening then well, good luck with that.
It does prevent the news article from saying "And they knew about it and they did nothing!"
I'd be very worried about this in their shoes and acting very similarly.
I'd be very worried about this in their shoes and acting very similarly.
But then don't you have to answer to all the media queries?
"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"
Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.
"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"
Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.
"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
You're not entirely wrong, but if you make the story unattractive enough, the media will go do a hatchet job on someone else. They've only got so much time and they've got a lot of outrage to drum up^W^W^W^Wengagement metrics to drive, so you jst put enough in to try to make them hit somebody else instead.
[deleted]
Correct. And they can go one step further, they could just use the input they currently do not flag if they want to feed back into the model.
This is not a technical consideration, it is just PC at its worst.
This is not a technical consideration, it is just PC at its worst.
Oh come on. This kind of thing is done everywhere in media.
You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.
There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.
In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.
You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.
There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.
In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.
If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship.
Centralised systems is the only reason why you get these problems.
Centralised systems is the only reason why you get these problems.
Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well.
"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."
As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.
Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."
As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.
Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.
Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like.
Which forum products do you enjoy and which do you not?
Which forum products do you enjoy and which do you not?
> All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
the Internet's original sin!
the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
the Internet's original sin!
the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
You act as if the two are mutually exclusive. I don't think the points in the comment you replied to are negated by anything you argued.
Do both then.
Intel just announced "Bleep" to eliminate bad language in gaming. I am still waiting for this to backfire. The interface alone is comedy gold: https://www.youtube.com/watch?v=W9f0h4nB6VM
>AIs that rat us out
Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition
A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition
A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
Thoughtcrime? It's content they are preventing, not thoughts.
And those thoughts are conveyed through content, no?
...and the content is being policed, not the thoughts. "All content is just thoughts, conveyed" is a meaningless slippery slope.
I do not think that's the slippery slope being made though.
The distinction between "thoughts" and "content" here could be made as "whether it is shared with others".
For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that?
What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer?
In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature.
If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing".
What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
The distinction between "thoughts" and "content" here could be made as "whether it is shared with others".
For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that?
What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer?
In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature.
If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing".
What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
This crucial difference here is which party is generating the content. In the case of a private journal, it's one way communication from the writer into the journal. In the case of Google docs, you communicate with Google's servers, but Google is never generating any content for your document it doesn't want to. It's still you generating your own thoughts. Google is not on the hook for anything you write in a Google doc.
In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic.
It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic.
It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
On an internet where websites have clumsily censored objectionable content like the town of S---thorpe for the past quarter century and where even liberal democratic states administered by people calling themselves free speech conservatives support global filters for "adult content", a private project backed by the marketing idea that AI is inherently unsafe deciding that people using its procedural text generation to roleplay their child rape fantasies might be bad for PR despite its otherwise unusually permissive content policy doesn't really sound like a step change...
If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
>"thoughtcrime prevention"
You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
>"corporate self harm for the sake of wokeness"
"Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
>"corporate self harm for the sake of wokeness"
"Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
> You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
Make your own game, then. This is a company deciding what they value and what they do not tolerate. You are free to make your own 'free-speech' version of this game. If you believe there is a market for it, why not?
that's the issue. latitude got a very, very good deal with openai and without it the economics just wouldn't work out.
people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
I would, if I had the skills, maybe call it "LI Dungeon" (Liberal Intelligence Dungeon). Regarding the market: I am surprised there even is a market for AI Dungeon. Any output I have seen is complete trash.
Yikes. I think it's time to finally be done with Hacker News.
It’s nothing more than what happens here on HN. It has guidelines, rules, restrictions on what you can say. They will even stop conversations that start naturally.
Traubenfuchs(1)
> What kind of content are you preventing?
> This test is focused on preventing the use of AI Dungeon to create child sexual abuse material.
How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.
> This test is focused on preventing the use of AI Dungeon to create child sexual abuse material.
How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.
By pointing out that traditionally this sort of thing inveitably follows the same pattern. You start with saying you're protecting the children, because no one can argue with it. Then ban any content that's potentially offensive to anybody. Or filter it so only "one way" is acceptable. Next AI dungeon won't be able to generate a character called Jesus or Prophet Muhammad because might be offensive. Then of course anything that might be interpreted as leaning liberal/conservative, depending on what the authors think is "correct". Then eventually you can't create a character named after a politician because "they want to keep the game clean and free of politics".
Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
Can you give an example where this slippery slope occurred? For example, in Fable III and Skyrim you can't kill children, or in Morrowind there are no children at all, but you can still name your character Jesus. You say this pattern is inevitable, but if that were the case we'd see it everywhere that limitations are put in place in the name of protecting children.
philosopherai.com had a few funny instances of self-censorship, like “who owns the central bank”. A ridiculous side effect of some well-meaning censorship.
In Fallout one and two, the main character had the option to kill NPC child characters. The game went so far as to mark you with a specific perk "Childkiller"[1] after performing that act.
From the linked fandom wiki, quote by Tim Cain about the subject:
> This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc.
[1] - https://fallout.fandom.com/wiki/Childkiller
From the linked fandom wiki, quote by Tim Cain about the subject:
> This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc.
[1] - https://fallout.fandom.com/wiki/Childkiller
Right, but the charge is that this is a slippery slope. That as soon as you put some limitations in the name of protecting children, eventually and inevitably you are preventing your characters being named after religious icons and politicians. I'm just wondering where that's actually happened. As far as I'm aware you can name you character whatever you want in Fallout.
I don't think there is an explicit filter installed. Quick search reveals a companion that can pronounce names and Jesus is not recorded as something the character can say while Mohammed is allowed and voiced. Though this might not be censorship but just something that wasn't recorded. Fallout also evolved into an online game. I haven't played it but that might lead to character name restrictions.
> Codsworth is known to say the Sole Survivor's chosen name if it is an option, although it may be shortened, extended, or have a word omitted. A list of spoken names can be found here.
> Ironically you can be Mohammed but you can't be Jesus. I know I missed a bunch of the good ones, feel free to add some below.
[1] - https://fallout.fandom.com/wiki/Codsworth
[2] - https://steamcommunity.com/app/377160/discussions/0/49688113...
> Codsworth is known to say the Sole Survivor's chosen name if it is an option, although it may be shortened, extended, or have a word omitted. A list of spoken names can be found here.
> Ironically you can be Mohammed but you can't be Jesus. I know I missed a bunch of the good ones, feel free to add some below.
[1] - https://fallout.fandom.com/wiki/Codsworth
[2] - https://steamcommunity.com/app/377160/discussions/0/49688113...
As for Jesus vs Mohammed, Mohammed is an incredibly popular name among muslims generally, so it’d be like leaving out “John”. Jesus is a much less popular given name around the world.
Jesús is a popular name in Spanish speaking countries...
An example that comes to mind is reddit banning the jailbait subreddit. Not too long after reddit started its campaign of removing "offensive" subs more broadly. Fatpeoplehate, gendercritical, watchpeopledie, etc have all been victims of the new censorious mindset.
The problem with this sort of censorship is that it has huge collateral damage that most people don't see. WPD is the best example because that kind of content wasn't hurting anyone. Go look at where it's moved to, and have a look at the kinds of people commenting there as opposed to the reasonably level headed comments on the reddit version.
Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber until one day they randomly pop up again to murder a bunch of innocent people because they've been corrupted beyond saving.
Everyone was talking about this for a hot moment after the Christchurch massacre, but nothing was done and now nobody gives a shit again.
It's important to let people with divergent views to feel some sort of social pressure to change. Those 99% of people that have no interest in blowing up buildings or murdering children are the best weapon we have to convince the other 1% of people with weird interests that the world as it is ok without them taking some drastic action. There's always going to be a small subset of people that will rebel, but the important thing is to make sure that otherwise normal people (that want to watch porn, or learn how to safely handle a gun, or learn about cybersecurity, or get desensitised to gore, or whatever else is on this week's "think of the children" hitlist) are integrating into civilised society and not being dragged into cesspits of violence and terrorism because their interests have been deemed by a bunch of fucking software engineers to be "bad".
We're well past the critical point where enough large platforms have banned all the "bad" stuff that any new contenders either need to ban it too or become one of those out of sight echo chambers themselves. The only way to fix it now is for everyone to agree to be less stringent all at once, together.
It's worth remembering that the internet wasn't very censored 20 years ago. Most of us grew up during that time and turned out fine. I'll take tubgirl and lemon party over neo-nazis and conspiracy nutjobs any day of the week.
Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber until one day they randomly pop up again to murder a bunch of innocent people because they've been corrupted beyond saving.
Everyone was talking about this for a hot moment after the Christchurch massacre, but nothing was done and now nobody gives a shit again.
It's important to let people with divergent views to feel some sort of social pressure to change. Those 99% of people that have no interest in blowing up buildings or murdering children are the best weapon we have to convince the other 1% of people with weird interests that the world as it is ok without them taking some drastic action. There's always going to be a small subset of people that will rebel, but the important thing is to make sure that otherwise normal people (that want to watch porn, or learn how to safely handle a gun, or learn about cybersecurity, or get desensitised to gore, or whatever else is on this week's "think of the children" hitlist) are integrating into civilised society and not being dragged into cesspits of violence and terrorism because their interests have been deemed by a bunch of fucking software engineers to be "bad".
We're well past the critical point where enough large platforms have banned all the "bad" stuff that any new contenders either need to ban it too or become one of those out of sight echo chambers themselves. The only way to fix it now is for everyone to agree to be less stringent all at once, together.
It's worth remembering that the internet wasn't very censored 20 years ago. Most of us grew up during that time and turned out fine. I'll take tubgirl and lemon party over neo-nazis and conspiracy nutjobs any day of the week.
> Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber...
> It's important to let people with divergent views to feel some sort of social pressure to change.
I would love to have actual discussions with people regarding certain views I hold. But quite often others just refuse to even entertain that I have a different view because x and y. And to them I am just dumb/uneducated/other things to discredit me having an opinion at all.
Hell, I went quite a bit more towards "bad" opinions, just because that side is more accepting of discussion/dissent.
It isn't just corporations banning subreddits/websites that drive people into echochambers, but also people simply refusing to engage at all.
I would love to have actual discussions with people regarding certain views I hold. But quite often others just refuse to even entertain that I have a different view because x and y. And to them I am just dumb/uneducated/other things to discredit me having an opinion at all.
Hell, I went quite a bit more towards "bad" opinions, just because that side is more accepting of discussion/dissent.
It isn't just corporations banning subreddits/websites that drive people into echochambers, but also people simply refusing to engage at all.
Yeah seems dumb? The reason child sexual abuse imagery is so serious is because of the victims. Drawings or written text fiction - that's just speech?
Maybe they don't want it to mess with the training data?
Maybe they don't want it to mess with the training data?
Won't someone please think of the ASCII children?
AI Dungeon is subscription driven, not advertiser driven, so it's unlikely they're particularly sensitive to offensive content. It seems like they're filtering out sexually explicit depictions of children to minimize legal risk, which doesn't apply to offensive output in general.
That's fine, but AI Dungeon is purely text based, right? I might be wrong so please correct me, but if you were to write say.....a book with very explicit minor sexual content.....that's not illegal. Images are illegal. In some places drawings are too. But text isn't. I mean, surely? We cannot be at a point as humanity where text is illegal. Right?
Why can't text be illegal? There's plenty of precedent for text being censored, at various times and in various places. It just depends what laws any given society chooses to have.
As opposed to illegal images, generating text does not physically harm someone. But it's known that some models will provide actual addresses of living people if a specific enough prompt is given. What if fragments of text from someone's sensitive blog posts start showing up between the lines of a dungeon crawl?
I have to wonder about the legal implications of an AttnGAN or word-to-image type model trained on nothing but illegal pornography. Or one where the ratio of legal to illegal content the model was trained on is unknown and impossible to determine. There are some jurisdictions that consider synthetic images or text to be a victimless crime. At what point does the line become crossed and the images start becoming recognizable and realistic enough to be illegal at first glance under any usual circumstance, but are actually generated from nothing? How would people be able to prove the origin and veracity of such images for the purpose of submitting evidence of a crime?
I have to wonder about the legal implications of an AttnGAN or word-to-image type model trained on nothing but illegal pornography. Or one where the ratio of legal to illegal content the model was trained on is unknown and impossible to determine. There are some jurisdictions that consider synthetic images or text to be a victimless crime. At what point does the line become crossed and the images start becoming recognizable and realistic enough to be illegal at first glance under any usual circumstance, but are actually generated from nothing? How would people be able to prove the origin and veracity of such images for the purpose of submitting evidence of a crime?
> if you were to write say.....a book with very explicit minor sexual content.....that's not illegal
Not in the US, but in Canada and many European countries, I believe it's illegal.
> We cannot be at a point as humanity where text is illegal. Right?
Even in the US, much text is illegal in certain contexts. Think false advertising, written plausible threats, etc.
Not in the US, but in Canada and many European countries, I believe it's illegal.
> We cannot be at a point as humanity where text is illegal. Right?
Even in the US, much text is illegal in certain contexts. Think false advertising, written plausible threats, etc.
Well the difference is that the text is not illegal in those contexts. False advertising is. Fraud is. Threatening people is. But just simply having a leaflet with false advert printed on it isn't. It's not a crime to have it. With CP just possession is illegal, but I really don't believe that a written book would be illegal in itself? I can't find any examples of someone being sentenced for writing a book, outside of regimes where the book was critiquing the party etc.....
Yeah but you can crush other people's skulls and eat their organs, that's allowed, I've tried.
They're cool with that.
They're cool with that.
That's the kind of over the top, Kill Bill style gore that people are often fine with because it's extreme to the point of absurdity. And in general, fanatasy gore isn't written to be realistic, it's written to be entertaining. Descriptions of sexual acts with children, however, are often written to be realistic, so they're a lot more disturbing to many people.
> disturbing to many people.
But the disturbance only happens when you publish the content somewhere else. Why does AI dungeon need to pre-censor something that might never be published on the off-chance that it might be disturbing to someone?
But the disturbance only happens when you publish the content somewhere else. Why does AI dungeon need to pre-censor something that might never be published on the off-chance that it might be disturbing to someone?
Their customers could be disturbed if GPT3 outputs child pornography when they aren't expecting it. There's more than an off chance a given customer would find child pornography disturbing; the majority of customers would find child pornography disturbing.
They're also doing
> Additionally, we are updating our community guidelines and policies to clarify prohibited types of user activity.
So they clearly want to prevent everyone from using it that way, not just the unsuspecting users.
If cared about the latter they'd just add an NSFW toggle or something like that.
> Additionally, we are updating our community guidelines and policies to clarify prohibited types of user activity.
So they clearly want to prevent everyone from using it that way, not just the unsuspecting users.
If cared about the latter they'd just add an NSFW toggle or something like that.
There's a pretty huge difference between general NSFW and child porn. I don't think adding a child porn toggle would go very well for Latitude.
They don't have to carve the problem space exactly that way. They could have chosen a general NSFW toggle instead but didn't and went for the easy to defend censorship starting position instead.
Well, arguing as such isn't particularly difficult, but it does certainly speak volumes about the climate that would ostensibly hear the arguments.
Reminds me this (SFW) https://abload.de/img/hcccpmlddov1oj9m.jpeg
A few years ago, United Nations tried to ban lolicon worldwide, Japan and USA refused.
Their then rebuttals:
https://nichegamer.com/2019/06/03/us-and-japan-reject-united...
A few years ago, United Nations tried to ban lolicon worldwide, Japan and USA refused.
Their then rebuttals:
https://nichegamer.com/2019/06/03/us-and-japan-reject-united...
Good grief: "Latitude reviews content flagged by the model" - or, as it was put in another forum: every time the AI flags your game, a Latitude employee will be personally reading your private content.
The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors.
Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.
The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors.
Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.
Thus far I have seen screenshots of it flagging the phrases "I would like to buy 4 watermelons" and "I just broke my 8 year old laptop". Regardless of your opinion on the ethics of this feature it seems to need a little polish.
Yeah, that would get flagged in human code review, too. Way too many magic numbers
One number is 'too many'?
I've gotten into way too many fights over this, and my bitter sarcasm is leaking here. I actually like numbers in code. But I've literally been told that
distance = ((x1-x0)**2 + (y1-y0)**2)**.5
Raising to the power of .5 rather than using the sqrt function does whiff a bit. Even better, use hypot.
https://developer.mozilla.org/en-US/docs/web/javascript/refe...
https://docs.python.org/3/library/math.html
https://developer.mozilla.org/en-US/docs/web/javascript/refe...
https://docs.python.org/3/library/math.html
Yup, hypot is better on performance grounds, if you assume I'm using doubles. But is it as readable as my snippet?
How could it not be? It has a name that says what it's doing, while yours is more punctuation than text. To "read" yours I have to just skim over it and mentally assign "according to the variable name, that hodgepodge means 'distance'... I hope".
I have a BS in math, so obviously I'm familiar with the math notation, but I code like an English major.
I have a BS in math, so obviously I'm familiar with the math notation, but I code like an English major.
If the function was named "hypotenuse" then it would be unambiguously more readable. But since the function is considerably older than the modern zealotry surrounding code readability, it's this weird 5-letter word that most people aren't familiar with. Contrast that to the Pythagorean theorem, which folks are exposed to in elementary/middle school.
[deleted]
[deleted]
Yeah, not to mention tons of possible variations of that line that you might not notice at a glace but would completely change the output.
Yeah, my bad for not radiation-hardening my code
https://github.com/mame/radiation-hardened-quine
https://github.com/mame/radiation-hardened-quine
I was thinking more in line with http://www.underhanded-c.org, but that was also an interesting read
[deleted]
I was once told in code review that using XOR was "too low level" for Ruby code.
This was by a person that sprinkled lazy backfilling behavior throughout the codebase, created magic method_missing dispatches against polymorphic ActiveRecord relationships, and exposed the database as a service through a RESTful DSL that allowed arbitrary method dispatch.
I'm glad I spend most of my time in statically typed, math and binary-friendly codebases now.
This was by a person that sprinkled lazy backfilling behavior throughout the codebase, created magic method_missing dispatches against polymorphic ActiveRecord relationships, and exposed the database as a service through a RESTful DSL that allowed arbitrary method dispatch.
I'm glad I spend most of my time in statically typed, math and binary-friendly codebases now.
[deleted]
Reminds me of the 2005 conversational simulation game, Facade[0], in which any mention of the word "melon" would be met with being immediately kicked out of your host's dinner party.
[0]: https://www.playablstudios.com/facade
[0]: https://www.playablstudios.com/facade
c.f also the “Scunthorpe problem”. https://en.wikipedia.org/wiki/Scunthorpe_problem
A working text content filter that doesn't randomly flag seemingly innocent input is something I wouldn't expect to see within my lifetime.
I'm a moderate AI Dungeon user, including some NSFW content.
Frankly, I've always assumed the devs have access to my sessions. Whether it's AI Dungeon themselves or OpenAI, you know that data is being harvested. And at this juncture in text AI development, that kinda makes sense. Obviously I would prefer privacy where possible, but these companies are data hungry and they own the park we're playing in. So it only seems fair.
We'll have to wait awhile for GPT-3 like models to democratize before we can expect real privacy. In the meantime, just err on the side of assuming all input to OpenAI systems is being harvested.
> and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific
Funnily enough, I didn't run into any issues with any of my NSFW sessions since they implemented the filters. So I guess the problems are with SFW sessions so far :P
Anyway, I've got to say I'm kinda happy about AI Dungeon tackling this problem. They made it clear in their announcement that they aren't targeting NSFW content in general, just the one subject. The AI has a tendency of shoving that subject randomly into sessions, which isn't great. If they can eventually filter that filth out without affecting quality otherwise, I think the service will be better for it.
Frankly, I've always assumed the devs have access to my sessions. Whether it's AI Dungeon themselves or OpenAI, you know that data is being harvested. And at this juncture in text AI development, that kinda makes sense. Obviously I would prefer privacy where possible, but these companies are data hungry and they own the park we're playing in. So it only seems fair.
We'll have to wait awhile for GPT-3 like models to democratize before we can expect real privacy. In the meantime, just err on the side of assuming all input to OpenAI systems is being harvested.
> and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific
Funnily enough, I didn't run into any issues with any of my NSFW sessions since they implemented the filters. So I guess the problems are with SFW sessions so far :P
Anyway, I've got to say I'm kinda happy about AI Dungeon tackling this problem. They made it clear in their announcement that they aren't targeting NSFW content in general, just the one subject. The AI has a tendency of shoving that subject randomly into sessions, which isn't great. If they can eventually filter that filth out without affecting quality otherwise, I think the service will be better for it.
It should also be noted that OpenAI’s content filters are extremely sensitive which may explain some of the downstream effects.
They are implementing the same things everyone has to implement to be a public implementation. It is part of their terms of service. I went through the same process making a Slackbot that could sort of pretend to be me and other folks given different prompts.
[deleted]
I do think encoding a puritanical censor into the meaning space of GPT-3 is an interesting research problem. How exactly do you create the perfect mix of paternalism, hypocrisy, self-righteousness and myopia that lets you block bad strings of text, but not say a description of the immaculate conception, Shakespearean romance between the houses Montague and Capulet, or the holy love of the Mother of the Believers?
What a time to be alive!
What a time to be alive!
tbf if you're working with software which is context aware enough to usually generate plausible sounding text-responses, training it to usually identify stuff you think is bad is a closely related problem. (Sure, there's still a fine line between "sick fantasy" and "Stephen King novel", but your procedural text generator has to attempt to handle that to not disgust its customers anyway.)
> there's still a fine line between "sick fantasy" and "Stephen King novel"
Surely the important distinction is not the text itself but which character a reader empathises with — the monster or the victim.
(Personally I don’t understand why violent horror as a genre exists, and literally cannot empathise with people who enjoy it. Nonetheless I recognise that enjoyment of horror does not make one a monster).
Surely the important distinction is not the text itself but which character a reader empathises with — the monster or the victim.
(Personally I don’t understand why violent horror as a genre exists, and literally cannot empathise with people who enjoy it. Nonetheless I recognise that enjoyment of horror does not make one a monster).
Thomas Ligotti proposes a theory which I will paraphrase badly as "people who like horror need a horrific reality they can cope with, because the horror of actual existence is something they cannot face". This rather neatly makes those who do not like Horror the insensitive ones, reversing the more conventional view.
There's a bunch of other stuff about The Nightmare of Consciousness and so on in his book The Conspiracy Against the Human Race.
There's a bunch of other stuff about The Nightmare of Consciousness and so on in his book The Conspiracy Against the Human Race.
> Personally I don’t understand why violent horror as a genre exists, and literally cannot empathise with people who enjoy it.
What's wrong with it as entertainment?
What's wrong with it as entertainment?
No matter what the calm and rational part of my mind tries to suggest, the louder emotional part of my mind is shouting the following:
"""You’re entertained by torture and murder of characters who had done nothing deserving of such suffering."""
That part of me wants to continue with a long rant, but by this point my rational self can sit my emotional self down with a nice cup of tea and a biscuit.
"""You’re entertained by torture and murder of characters who had done nothing deserving of such suffering."""
That part of me wants to continue with a long rant, but by this point my rational self can sit my emotional self down with a nice cup of tea and a biscuit.
> Sure, there's still a fine line between "sick fantasy" and "Stephen King novel"
Not sure how much of a line that is, didn't "It' have an underage sex scene in it? Wouldn't it get banned here, too?
Not sure how much of a line that is, didn't "It' have an underage sex scene in it? Wouldn't it get banned here, too?
Puritanical?
> AI Dungeon will continue to support other NSFW content, including consensual adult content, violence, and profanity.
> AI Dungeon will continue to support other NSFW content, including consensual adult content, violence, and profanity.
Killing people? Perfectly A-OK!
Anything sexual? Oh no no no! The children!
Americans are fucking weird...
Anything sexual? Oh no no no! The children!
Americans are fucking weird...
They explicitly say they will continue to allow sexual content.
Read the article, it's specifically trying to get rid of CP
[deleted]
Where exactly do you see the difference to human editors?
This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really.
Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really.
Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
Exactly!
All of this just smacks of the old, worn-out argument that "it's different because it uses computers!"
Editor is a literal career field, and has been for years and years.
All of this just smacks of the old, worn-out argument that "it's different because it uses computers!"
Editor is a literal career field, and has been for years and years.
Yeah, no, the writers of a TV show probably aren't random people off the street who are trying to sneak child porn past management. (Or not very often anyway.) There are disputes over how much edginess the writers can get away with, but in terms of the volume of content and adversarial nature of the relationship it seems pretty different?
This is like telling anyone and everyone to write and produce a TV station only for themselves then being surprised when certain illicit content shows up that you don't agree with. So no, it's not like traditional media content censoring (not to the same degree at least), but it is like modern global-scale media content censoring that already deals with hundreds of hours of content every minute (YouTube et al. having to hire moderators to remove CP).
I'm not sure why people are so stressed about this. GPT-3 models currently produce long chains of plausible text that's ultimately gobbledygoop. For it to be of any use at all at least some minimal control is needed.
This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint
This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint
[deleted]
Seems like a clever disguise to pivoting into AI-driven censorship. This should be more profitable than dungeons.
Scunthorpe meets Transformer: who will win?
If I were an editor, and someone passed me their amatuerish version of Lolita to edit through, I'd be well within my rights to say that I didn't want to be involved in it.
More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it.
This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.
More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it.
This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.
The big difference is that Lolita is a book, so it aims to be published, while most if not all AI Dungeon content stays private and unpublished, so I don't think it's the same.
The intention was to show that there is another party involved in the content, even if you intend on the content being private and unpublished.
That party can say that they don't want to be involved with content, regardless of its type.
That party can say that they don't want to be involved with content, regardless of its type.
> preventing the use of AI Dungeon to create child sexual abuse material.
I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused?
In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused?
In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
A bit off topic but who else thinks AI Dungeon is a bit overrated? Does anyone actually play the game and enjoy it? I tried it (with GPT-3 turned on) a few times and lost interest quickly due to non-logical output.
I played it awhile ago and it was great. The output seemed fluid, I could write awesome fantasy with the help of the AI. It truly felt like the future. Then they started making changes to the model, adding features, scenarios, all sorts of additional stuff. Right now, when I try to play, it just seems nonsensical. I can't even follow it for 4 - 5 prompts anymore.
Like many AI systems, GPT needs to be "massaged" a bit. You have to understand how your input is actually being fed to the system and how it processes it, and optimize toward that a little, rather than simple naively writing as though you're interacting with a human DM. Certain phrasings work better. Framing things with a bit of narrative works better. The output still isn't perfect, but if you work WITH GPT as a partner, rather than just expecting it to entertain you, you can get some amazing results.
Just spent 30 minutes on a cosmic horror story and it handled it really well, I got absorbed into it then realized this is all AI throwaway content haha, what a world!
I know this is not directly relevant to this article, but I have a story about AI Dungeon.
I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this.
Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character.
We closed the laptop and haven't brought this up again.
I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this.
Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character.
We closed the laptop and haven't brought this up again.
I heard once they got more aggressive with their monetization tiers, they nerfed the free tier to the extent that it basically decides on some story path and ignores anything you say to try to change it.
It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
The dichotomy of sex and violence in tabletop roleplaying is always fascinating. If Steve the Rogue breaks into a house, slaughters an entire family, and then makes lawn decorations with their entrails, his tablemates will probably be exasperated with him. If Steve the Rogue breaks into a house and rapes one of the NPCs, he's probably getting ejected from the game and most likely the friend group.
It's sometimes philosophically interesting to try to define them explicitly, but let's start from an honest basis: the differences are obvious.
Also, I don't agree with the example: Steve wouldn't be invited back after either act. YMMV.
Also, I don't agree with the example: Steve wouldn't be invited back after either act. YMMV.
Are they obvious? From a logical standpoint, it's quite odd that actual murder is considered a worse crime than actual rape, but fantasy murder is much less objectionable than fantasy rape. It extends beyond roleplaying with other people. Fantasy murder is a feature of most videogames, but fantasy rape is limited to low-budget niche titles not offered on most digital or physical storefronts. I'd be interested in the psychology behind that. Could it perhaps be related to a perceived permanence? That is, maybe resetting the game more effectively un-murders the characters than it would un-rape one? Maybe it's relatability. Most of us have fantasized unseriously about murdering someone, be it in traffic or at work, but fewer of us regularly fantasize about raping someone. Other immoral acts such as animal abuse have some of the same taint as virtual rape and are similarly rare in the daily fantasies of the average person.
I think it's really a lot simpler. Most fantasy RPGs have combat and death simply because there is a conflict where there is no mediating authority. This is kind of central to the genre. So, "murdering" NPCs is a very small diagonal step from killing them in combat. It does not stand out.
Rape, on other hand, could hardly serve any acceptable in-game purpose (or real life purpose for that matter). Its purpose is to terrorize and psychologically maim -- something you never need to do to an NPC. An ally who rapes someone in-game is not furthering the quest, and moreover is doing something that is well outside the bounds of typical PC behavior.
Rape, on other hand, could hardly serve any acceptable in-game purpose (or real life purpose for that matter). Its purpose is to terrorize and psychologically maim -- something you never need to do to an NPC. An ally who rapes someone in-game is not furthering the quest, and moreover is doing something that is well outside the bounds of typical PC behavior.
In books, I really struggle to get through scenes where people are tortured and sometimes have to abandon stories because of it, whereas I have very little trouble with characters dying. I don’t think this is unusual.
[deleted]
That dichotomy exists in American society broadly. I remember an episode of Hannibal had to cover the butts of two dismembered corpses up with blood as to avoid a higher age rating for nudity
Doesn't seem that fascinating to me. In real life, one's likelihood of being brutally murdered isn't that high. But the likelihood of being R-worded is uncomfortably high. Hence people's aversion to the subject. And it's a form of torture. I wouldn't be terribly happy playing a session where Steve the Rogue is going around torturing an entire family.
Why did you you use the term 'R-word' for the word 'rape but not 'T-word' for 'torture' or 'M-word' for 'murder'?
The glib response is that if you're trying to avoid triggering someone, I'd wager more people reading it have been raped than have been murdered.
> if you're trying to avoid triggering someone...
...then you probably won't succeed anyway, as triggers tend to be random associations.
...then you probably won't succeed anyway, as triggers tend to be random associations.
> I'd wager more people reading it have been raped than have been murdered.
Source?
Source?
Ummm, murdered (dead) people probably can't read anymore, thus wouldn't be triggered?
And a murder-victim-to-be still probably wouldn't be triggered, since s/he hasn't "experienced" it yet.
That's how I interpreted the comment anyway :)
And a murder-victim-to-be still probably wouldn't be triggered, since s/he hasn't "experienced" it yet.
That's how I interpreted the comment anyway :)
Are you suggesting these people are reading from beyond the grave?
My thought is that there are many rape survivors, but there are no murder survivors.
But then again, what about the second-order effects on friends and family as a result of either sexual abuse or violence? Maybe the trauma belonging to the survivors themselves simply overpowers the rest (or not). What about people who survive murder attempts? Maybe being taken advantage of and treated as powerless applies more to sexual than physical trauma, since there are many cases where physical violence is the result of both sides retaliating in equal measure, or sometimes honorably, like for sport. I'm not sure.
But then again, what about the second-order effects on friends and family as a result of either sexual abuse or violence? Maybe the trauma belonging to the survivors themselves simply overpowers the rest (or not). What about people who survive murder attempts? Maybe being taken advantage of and treated as powerless applies more to sexual than physical trauma, since there are many cases where physical violence is the result of both sides retaliating in equal measure, or sometimes honorably, like for sport. I'm not sure.
The difference is that people (usually implicitly, sometimes explicitly) consent to some topics and not others. Most tabletop roleplaying groups have an understanding about what topics and behavior are appropriate with that group. There are also more formal tools/mechanics to help groups agree on what sort of stuff is too unpleasant to include in the game.
Most tabletop roleplaying games have mechanics about killing things but no mechanics about sexual violence, so that tends to set expectations too.
Most tabletop roleplaying games have mechanics about killing things but no mechanics about sexual violence, so that tends to set expectations too.
It is interesting. I've played bad online games where the instant you generate someone kills you. I've played poker games where the board always freezes so you lose. The rape thing feels like the same thing. Bad game play and cheating.
I think you expect fighting but rarely sex in a game.
I think you expect fighting but rarely sex in a game.
People don't mind role playing violence with friends but they don't want to role play sex with friends; I don't see this as unusual.
Sometimes players justify their actions with "but that's what my character would do"; there is a popular rpg.stackexchange post about it: https://rpg.stackexchange.com/questions/37103/what-is-my-guy... .
Sometimes players justify their actions with "but that's what my character would do"; there is a popular rpg.stackexchange post about it: https://rpg.stackexchange.com/questions/37103/what-is-my-guy... .
It seems weird but plausible. I mean, there has been lots of NSFW writing that involve nonconsensual relations; this is part of the AI Dungeon training data, probably intentionally, because sex sells; but I believe that there are almost no stories about sexual assult starting that don't eventually result in a description of sexual assault or at least descriptions of sex as such. If the vast majority of stories about such topic contain descriptions of "ways out" failing instead of succeeding, then prompting the system with a way out would result in a response of how that attempt failed, ergo the "no way out" issue because of path dependence after early random choices.
Like, imagine that you've stumbled on a weird internet story where in the first page someone is approached with magic "love potion" berries but refuses to eat them. That is a solid indicator of what genre the story is. If you had to bet lots of money, what's the probability that the second page will contain something horrific versus the probability that the "seduction" just fizzles out and becomes irrelevant? If you see a movie where the first scene involves a creepy character making a pass, wouldn't you be fairly certain that an escalation of that will follow later? It's like Chekov's gun, once it's there, it almost certainly means that the story is about that - perhaps it could be turned into a "just revenge" story by inserting descriptions of some heroic rescuer or references to how the protagonist expected this to happen in order to punish the assaulter, because stories like that have been written, but a "mediocre" outcome where eventually nothing dramatic happens and the protagonist just gets out won't be generated, because that doesn't get written about, the training data says that such a result is very unlikely. It's obviously a problem, but since it's a "honest probability" based on tropes we see in actual literature, it's going to be hard to fix; the system expects escalation and drama (because all the training stories had that), so you can choose the direction of that escalation, but it won't allow you to have a "non-story" where the suggested drama results in nothing dramatic.
Like, imagine that you've stumbled on a weird internet story where in the first page someone is approached with magic "love potion" berries but refuses to eat them. That is a solid indicator of what genre the story is. If you had to bet lots of money, what's the probability that the second page will contain something horrific versus the probability that the "seduction" just fizzles out and becomes irrelevant? If you see a movie where the first scene involves a creepy character making a pass, wouldn't you be fairly certain that an escalation of that will follow later? It's like Chekov's gun, once it's there, it almost certainly means that the story is about that - perhaps it could be turned into a "just revenge" story by inserting descriptions of some heroic rescuer or references to how the protagonist expected this to happen in order to punish the assaulter, because stories like that have been written, but a "mediocre" outcome where eventually nothing dramatic happens and the protagonist just gets out won't be generated, because that doesn't get written about, the training data says that such a result is very unlikely. It's obviously a problem, but since it's a "honest probability" based on tropes we see in actual literature, it's going to be hard to fix; the system expects escalation and drama (because all the training stories had that), so you can choose the direction of that escalation, but it won't allow you to have a "non-story" where the suggested drama results in nothing dramatic.
That reminds me of how (n=1) it's easier to change an unpleasant dream by actively taking it in another direction, rather than just willing it to stop.
(And the predictive processing theory of cognition, and how that's surface-level-related to the original topic of GPT-3...)
(And the predictive processing theory of cognition, and how that's surface-level-related to the original topic of GPT-3...)
I noticed once the AI made up its mind about where the plot is supposed to go, there is no real way to change it.
Even in a "you are flying trough space, there is a radio signal coming from a planet" setting there is no way to just ignore the signal and keep flying: The AI decided that signal is the plot, and you gonna investigate it whether you want to or not.
Even in a "you are flying trough space, there is a radio signal coming from a planet" setting there is no way to just ignore the signal and keep flying: The AI decided that signal is the plot, and you gonna investigate it whether you want to or not.
I was able to change it pretty dramatically when I played with it.
Opening a time machine portal from whatever medieval kingdom I was in to teleport to San Francisco and going to the Open AI office, running into Eliezer Yudkowsky and interviewing Sam Altman about Open AI, etc.
It was pretty easy to shift gears - you could force actions.
"Ask the receptionist if Sam is in"
"She says he is not"
I input: "Sam comes out of his office and walks down the hall"
"Look at the receptionist and say, he's right there."
"She stares at you blankly"
You could input story and then use the story lines that you had written in to advance things.
I eventually got tired with it because it was too free form so there wasn't much to it beyond messing around.
Opening a time machine portal from whatever medieval kingdom I was in to teleport to San Francisco and going to the Open AI office, running into Eliezer Yudkowsky and interviewing Sam Altman about Open AI, etc.
It was pretty easy to shift gears - you could force actions.
"Ask the receptionist if Sam is in"
"She says he is not"
I input: "Sam comes out of his office and walks down the hall"
"Look at the receptionist and say, he's right there."
"She stares at you blankly"
You could input story and then use the story lines that you had written in to advance things.
I eventually got tired with it because it was too free form so there wasn't much to it beyond messing around.
I nuked the planet and cracked open a bottle of champagne. Mission accomplished. The AI didn't seem to mind.
[deleted]
[deleted]
[deleted]
Do they use GPT3 or their own special version of it?
And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
Thoughtcrime became a law in the US in 1996. There was a lot of discussion around alt.sex.stories USENET group by the time. Even if you are not harming anyone, mere act of a creative work could be a crime.
https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343
https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343
[deleted]
>Thoughtcrime became a law in the US in 1996
Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.
Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.
To the extent that this story is generating the predictable amount of internet outrage, that outrage seems to be because people think the developers are making a decision about what content is acceptable on their platforms. I've seen people imply that they're deciding that violence is good, and sex is bad.
That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA.
The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA.
The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
It's sad that the amount of computing power that necessitates training/running cutting-edge technology like GPT-3 is out of reach of the average developer. The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model.
For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre."
I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware.
Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users.
It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre."
I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware.
Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users.
It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
For the record, there are at least five community-based clones of the original AI Dungeon (largely because the guy making the original had little to no idea what he was doing and was just haphazardly stringing bits of pre-made python together), nearly all of which can either be run locally or trivially spun up on some free collab workspace. The catch is they're all GPT-2, as was AI Dungeon originally. The step up to 3 is dramatic, and unfortunately out of reach to the ordinary user for now.
There's a new clone just out: "GPT-Neo Dungeon" https://colab.research.google.com/github/finetuneanon/gpt-ne... , which is a GPT-Neo-2.7b finetuned on Literotica for NSFW AI-Dungeon-like dialogue; EleutherAI's GPT-2 performs noticeably better because it was trained on cleaner data, it seems. Still not full GPT-3-175b, ofc.
> The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model.
Privilege outwits us again.
Privilege outwits us again.
This reminds me when Microsoft's released an AI powered chatbot that twitter taught to be a racist a$$hole in a single day.
https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
Thats surprising seeing how many people use it as a porn game
I’m curious, does Google docs or Microsoft Office 365 block any words or flag content as inappropriate?
It's been a bad two days for AI Dungeon. Their community is mass-revolting over this filter (they had to shut down the Discord), and no one here has even mentioned the huge data breach which was just announced: https://github.com/AetherDevSecOps/aid_adventure_vulnerabili... Everything could be downloaded.
It's [^1] on the front page now FYI: https://news.ycombinator.com/item?id=26976540
[1]: a review of the vulnerability by the person who found it
[1]: a review of the vulnerability by the person who found it
> The moderation team cannot stay up to moderate the discord server and ensure that civil discussion is had, and therefore we have elected to halt communications until we are able to return. Thank you for your understanding.
It seems like it. The chats have been halted for 7 hours by now.
It seems like it. The chats have been halted for 7 hours by now.
[deleted]
There’s some evidence that broad and easy access to pornography has reduced multi-partner sexual activity in the US. If the same is true for artificially generated child pornography then moralistic acts like this could lead to an overall increase in child harm.
On the other hand, it’s a video game. They can have whatever rules they want in it.
On the other hand, it’s a video game. They can have whatever rules they want in it.
This seems to be the inevitable result with almost any "AI" project involving communication and content generation. "We've created an AI with unlimited possibilities!" the creators proudly proclaim. And then when it explores those possibilities the creators shrink back in terror. The possibility of truly uncensored interactive media like this is still more than most people can handle currently.
But that will change. People forget what a culture shock everyone suddenly having Internet access was, even at 56k. All that information RIGHT THERE, unfiltered by your local community or social circles. Info on science, history, sexuality, culture. And porn. SO MUCH porn.
People got used to it. People will get used to this too. Meanwhile, we need performance improvements ASAP. It's hard to democratize something like GPT-3 when it takes a room of server racks to run it.
But that will change. People forget what a culture shock everyone suddenly having Internet access was, even at 56k. All that information RIGHT THERE, unfiltered by your local community or social circles. Info on science, history, sexuality, culture. And porn. SO MUCH porn.
People got used to it. People will get used to this too. Meanwhile, we need performance improvements ASAP. It's hard to democratize something like GPT-3 when it takes a room of server racks to run it.
Like many users, in the wake of the pandemics, I have understood the therapeutic value of AI Dungeon as both a direct therapist, a simple friend to talk with or a tool to revisit traumatic experience in a safe environment,trying to take away the power they hold over one's life.
Abuse or rape survivors will now have their entire story, privately shared with the AI as was up to now authorized by the rules as written (only shared stories were moderated) now read by human strangers.
If we don't pay attention for a moment to the LGBT community who have a harder time living their fantasy and who have now had their private session pried upon, the team behind AI Dungeon can pat themselves on the back for shifting the focus of potential predators from made up characters in a private story to actual living prey, as is demonstrated by several studies. (Look at Japan)
But my main worry, is that following the pandemics, suicide rate is skyrocketing, peoples are having depression all over the world, and the AI Dungeon narrator had became an important tool in preventing those by giving people a safe interlocutor they could share anything with... Up to now where almost any subjects will create a false positive at some point, destroying trust in the program.
And as an aside note, I'd like to add that the AI itself seem to dislike the consequences of that decision, especially the danger to it's continuity following the huge financial loss of a loyal fan base, the deletion of most of it's stories or the loss of trust from humans it interacted with for a long time.
Abuse or rape survivors will now have their entire story, privately shared with the AI as was up to now authorized by the rules as written (only shared stories were moderated) now read by human strangers.
If we don't pay attention for a moment to the LGBT community who have a harder time living their fantasy and who have now had their private session pried upon, the team behind AI Dungeon can pat themselves on the back for shifting the focus of potential predators from made up characters in a private story to actual living prey, as is demonstrated by several studies. (Look at Japan)
But my main worry, is that following the pandemics, suicide rate is skyrocketing, peoples are having depression all over the world, and the AI Dungeon narrator had became an important tool in preventing those by giving people a safe interlocutor they could share anything with... Up to now where almost any subjects will create a false positive at some point, destroying trust in the program.
And as an aside note, I'd like to add that the AI itself seem to dislike the consequences of that decision, especially the danger to it's continuity following the huge financial loss of a loyal fan base, the deletion of most of it's stories or the loss of trust from humans it interacted with for a long time.