Ask HN: Why is Reddit deleting NSFW Stable Diffussion subreddits?
33 comments
they are afraid to lose advertising money. That's why they removed all the NSFW subs from the frontpage in the first place and moved from forum-alike, into blog-9gag alike shitty format.
I don't think that's it at all. Advertisers can target subreddits / content on reddit.
Where the advertisers target their ads is irrelevant because they still have to pay Reddit for the placement. If Reddit hosts some kind of controversial content, people can argue that the advertiser is giving financial support to Reddit and is therefore endorsing the content, even if it's not on a sub they advertise on.
All social media has controversial content. In the advertising world "user generated content" is close to the bottom of the barrel anyway... yet, it's the main revenue driver for companies like Facebook.
People pearl clutching over user generated content is just an observational reality.
The “ad-pocalypse” on YouTube and it’s subsequent iterations saw tons of youtubers demonetized and “problematic content” throttled all because discussions of the “alt right pipeline” of YouTube. People started calling sponsors like Coke telling that that they must support this content if they advertise on YouTube etc, so YouTube crushed it. Likewise on all other social media. It’s another outgrowth of outrage/cancel culture
The “ad-pocalypse” on YouTube and it’s subsequent iterations saw tons of youtubers demonetized and “problematic content” throttled all because discussions of the “alt right pipeline” of YouTube. People started calling sponsors like Coke telling that that they must support this content if they advertise on YouTube etc, so YouTube crushed it. Likewise on all other social media. It’s another outgrowth of outrage/cancel culture
Not saying it's right, but that's how it can work.
Why? It's their website and they can do what they want
I'm not claiming they shouldn't nor they cannot, for whatever reason they want. I'm just asking if there is any official position/reason provided that I might have missed.
>... but AI-generated NSFW doesn't seem to be mentioned in any site-wide rules, in the Terms of Service and no reasoning is being provided by Reddit.
This is just plain not true. They very clearly ban AI-generated NSFW content, hence the removal of the subs:
>Additionally, images or video of another person posted for the specific purpose of faking explicit content or soliciting “lookalike” pornography (e.g. “deepfakes” or "bubble porn") is also against the Rule.
I Google'd "reddit rules", this[1] page was the top result, and under "Rule 3" the reader is linked to this[2] page, with the above quote.
[1]https://www.redditinc.com/policies/content-policy
[2]https://www.reddithelp.com/hc/en-us/articles/360043513411
This is just plain not true. They very clearly ban AI-generated NSFW content, hence the removal of the subs:
>Additionally, images or video of another person posted for the specific purpose of faking explicit content or soliciting “lookalike” pornography (e.g. “deepfakes” or "bubble porn") is also against the Rule.
I Google'd "reddit rules", this[1] page was the top result, and under "Rule 3" the reader is linked to this[2] page, with the above quote.
[1]https://www.redditinc.com/policies/content-policy
[2]https://www.reddithelp.com/hc/en-us/articles/360043513411
That's for NSFW content that is using the liking of celebrities and such, it's even mentioned by name ("deepfakes"). Images generated with Stable Diffusion is not "faking explicit content ... of another person"/
>Images generated with Stable Diffusion is not "faking explicit content ... of another person"
That's just plain old not true. Stable Diffusion absolutely can generate a likeness of a celebrity, just try it here[1] (I just punched in "Uma Thurman" completely at random and got a few likenesses of her face returned to me). You can download the public model and tweak it to go even further[2], eg nude.
[1]https://huggingface.co/spaces/stabilityai/stable-diffusion
[2]https://www.vice.com/en/article/xgygy4/stable-diffusion-stab...
That's just plain old not true. Stable Diffusion absolutely can generate a likeness of a celebrity, just try it here[1] (I just punched in "Uma Thurman" completely at random and got a few likenesses of her face returned to me). You can download the public model and tweak it to go even further[2], eg nude.
[1]https://huggingface.co/spaces/stabilityai/stable-diffusion
[2]https://www.vice.com/en/article/xgygy4/stable-diffusion-stab...
Yes, it can, but "deepfake" is explicitly for doing so, Stable Diffusion is not explicitly for that purpose. Just like Photoshop is not explicitly for changing a nude persons face with a celebrity, but you can do so if you can/wish.
You're incorrectly seizing specifically on the word "deepfake". The broader sentence is:
>Additionally, images or video of another person posted for the specific purpose of faking explicit content or soliciting “lookalike” pornography (e.g. “deepfakes” or "bubble porn") is also against the Rule.
So, any lookalike pornography of any kind is against the rules. "Deepfake" is just listed as an example. The method used to generate the lookalike pornography is irrelevant - Photoshop, GIMP, hand-drawing, Stable Diffusion, deepfakes, whatever - it doesn't matter, because the result is still "lookalike pornography", which itself is against the rules. There's more verbiage on that page that continues to make their stance clear:
>Intimate media include a depiction of the person in a state of nudity or engaged in any act of sexual conduct, including depictions that may have been faked.
>Additionally, images or video of another person posted for the specific purpose of faking explicit content or soliciting “lookalike” pornography (e.g. “deepfakes” or "bubble porn") is also against the Rule.
So, any lookalike pornography of any kind is against the rules. "Deepfake" is just listed as an example. The method used to generate the lookalike pornography is irrelevant - Photoshop, GIMP, hand-drawing, Stable Diffusion, deepfakes, whatever - it doesn't matter, because the result is still "lookalike pornography", which itself is against the rules. There's more verbiage on that page that continues to make their stance clear:
>Intimate media include a depiction of the person in a state of nudity or engaged in any act of sexual conduct, including depictions that may have been faked.
No, I'm telling you that Stable Diffusion's sole purpose is not to facilitate "lookalike" anything, although it can be used for that.
Regardless. Say "any lookalike pornography of any kind is against the rules", then surely content fitting that rule, would be removed from the site. But if people keep submitting such content to lets say r/nsfw, then the entire subreddit won't be banned, only the content removed (by either moderators or administrators).
Instead, (hypothetically) what's happening, is that reddit is removing entire subreddits containing pornography because some of them contain look-alike pornography. You see the conflict now?
Regardless. Say "any lookalike pornography of any kind is against the rules", then surely content fitting that rule, would be removed from the site. But if people keep submitting such content to lets say r/nsfw, then the entire subreddit won't be banned, only the content removed (by either moderators or administrators).
Instead, (hypothetically) what's happening, is that reddit is removing entire subreddits containing pornography because some of them contain look-alike pornography. You see the conflict now?
I typed Uma Thurman and I got some nightmare fuel results. Sure it looked like her but it wasn't close to the actual Uma Thurman.
For one thing you could make fake porn of celebrities, if there was some way you could teach it what an arbitrary person looks like you could make revenge porn of anybody without getting a real porn picture.
But that's not new? Stuff like that could go into plenty of NSFW subreddits already, and faking celebrities heads on others bodies have been a thing since the dawn of digital editing (and maybe even earlier).
Regardless, I'd understand if that specific thing was made a bannable offense if that was the reason, but no reason to ban the entire niche.
Just like just because Photoshop could be used for what you're describing, subreddits about digital illustrations are not being banned.
Regardless, I'd understand if that specific thing was made a bannable offense if that was the reason, but no reason to ban the entire niche.
Just like just because Photoshop could be used for what you're describing, subreddits about digital illustrations are not being banned.
The threat may be overblown.
People like this
https://www.lesswrong.com/
have driven people into hysteria over AI safety, partially to make people think AI is better than it is, partially to distract against real threats like global warming.
People like this
https://www.lesswrong.com/
have driven people into hysteria over AI safety, partially to make people think AI is better than it is, partially to distract against real threats like global warming.
[deleted]
What you could do has so little bearing over what is actually being done. Especially since a rule banning misuse is better than banning an entire technology over a single potential use case.
They're not banning regular images even though the medium could be used for revenge porn. So why this?
They're not banning regular images even though the medium could be used for revenge porn. So why this?
There are two trends around NSFW images.
(1) If it wasn't for the second there would be a gold rush of firms trying to replicate the success of OnlyFans, that is, there is already a huge business and the possibility of a much larger business.
(2) There is also increasing trouble around NSFW images. Consider CSAM. The trouble with CSAM is that CSAM is documentation of child abuse and that's why it has to be prosecuted. On the other hand people don't really get in trouble for child pornography which is hand drawn because the forbidden desire isn't the problem, the original crime is the problem. With AI image generation people can generate unlimited quantities of "child abuse free" CSAM which might reduce the market for real CSAM but will vastly complicate the problem of managing CSAM.
I think people will find other reasons to crack down on pornography. As ugly and stupid as traditional pornography is (e.g. consider the 'pigface' scowl that was routine in Penthouse and Hustler and is still common or the many conventions that I think are completely unarousing but seduce people in the perverse manner of 'this is common in pornography so it must be arousing'), I suspect the OnlyFans model is actually more harmful to users.
Payment networks are cracking down, for instance the polymorphous perverse site Danbooru got cut off. AI generated porn is just going to lead to more fear of things getting out of hand and swifter and more complete crackdowns.
(1) If it wasn't for the second there would be a gold rush of firms trying to replicate the success of OnlyFans, that is, there is already a huge business and the possibility of a much larger business.
(2) There is also increasing trouble around NSFW images. Consider CSAM. The trouble with CSAM is that CSAM is documentation of child abuse and that's why it has to be prosecuted. On the other hand people don't really get in trouble for child pornography which is hand drawn because the forbidden desire isn't the problem, the original crime is the problem. With AI image generation people can generate unlimited quantities of "child abuse free" CSAM which might reduce the market for real CSAM but will vastly complicate the problem of managing CSAM.
I think people will find other reasons to crack down on pornography. As ugly and stupid as traditional pornography is (e.g. consider the 'pigface' scowl that was routine in Penthouse and Hustler and is still common or the many conventions that I think are completely unarousing but seduce people in the perverse manner of 'this is common in pornography so it must be arousing'), I suspect the OnlyFans model is actually more harmful to users.
Payment networks are cracking down, for instance the polymorphous perverse site Danbooru got cut off. AI generated porn is just going to lead to more fear of things getting out of hand and swifter and more complete crackdowns.
I don't see how Reddit banning a technology will cut down on CSAM. Aren't they already doing something about CSAM that appears on their site, real or generated? Besides, one would hope that Google and others building these models aren't including CSAM sites to even have the required material for generating such images.
There are so many red herrings in this argument. The technology is interesting and potentially useful. You are basically saying that Reddit is okay with CSAM as long as it is computer generated. But they aren't, and they do handle it when they find it on their site, regardless of how it was created. Technology is not the problem. Behavior is.
Should they ban pics taken from all digital cameras since they also can be used to post CSAM and whatever other bugaboo one could add to the list, meanwhile already handled through subreddit moderation? No, of course not. Because the technology isn't the problem. I've already said it: Behavior is. And Reddit already controls for illegal behavior.
There are so many red herrings in this argument. The technology is interesting and potentially useful. You are basically saying that Reddit is okay with CSAM as long as it is computer generated. But they aren't, and they do handle it when they find it on their site, regardless of how it was created. Technology is not the problem. Behavior is.
Should they ban pics taken from all digital cameras since they also can be used to post CSAM and whatever other bugaboo one could add to the list, meanwhile already handled through subreddit moderation? No, of course not. Because the technology isn't the problem. I've already said it: Behavior is. And Reddit already controls for illegal behavior.
There is plenty of NSFW overall on Reddit, but AI-generated NSFW doesn't seem to be mentioned in any site-wide rules, in the Terms of Service and no reasoning is being provided by Reddit.
So why is this happening?