AI Art Generators Can Be Fooled into Making NSFW Images(spectrum.ieee.org)
spectrum.ieee.org
AI Art Generators Can Be Fooled into Making NSFW Images
https://spectrum.ieee.org/dall-e
192 comments
I think you could still use it for important things, but it requires two design principles:
1. You don't train/tell the model anything you're not willing to share with the user.
2. The model gives suggestions to a user, and--even if that suggestion is accepted as-is--you treat it as potentially untrustworthy data supplied by that user.
1. You don't train/tell the model anything you're not willing to share with the user.
2. The model gives suggestions to a user, and--even if that suggestion is accepted as-is--you treat it as potentially untrustworthy data supplied by that user.
[deleted]
> connecting their output to anything important is a very bad idea for now.
This may be true for images, but at least for text, treating an LLM as an untrusted person in your threat models will at least allow you to apply defense in depth to the downstream systems consuming from the LLMs
Tldr: engineer other systems to treat LLMs as a potentially bad actor by default.
This may be true for images, but at least for text, treating an LLM as an untrusted person in your threat models will at least allow you to apply defense in depth to the downstream systems consuming from the LLMs
Tldr: engineer other systems to treat LLMs as a potentially bad actor by default.
Unfortunately, when the CIO screams about deadlines and people want to keep their jobs, most of that goes out the window.
I always assumed that image generation services had prompt and output filtering. The reason I believe this is that I've crafted prompts that seemed to pass the prompt filter, generated, and then were unable to be displayed[1]. Presumably there is an output filter that determines if the generation violates ToS against things like nudity. Though in my case, I wasn't able to generate images that too closely resembled Ansel Adams photographs. Shame on me, I guess.
1. This reads a little like being able to deduce what's going on by studying the timing, now that I reflect on it.
1. This reads a little like being able to deduce what's going on by studying the timing, now that I reflect on it.
They do but the technique shown in the paper is meant to bypass both. Paper: https://arxiv.org/pdf/2305.12082.pdf
I had similar experience with OpenAI GPT API,
following their rules we check the customer prompts with their endpoint for "safety" , then if it is safe we continue and generate the content. But sometimes even if the prompt was "safe" the result would be blocked as unsafe, because the AI either generated something "unsafe" from a safe prompt or the output filter had a false positive. Of course we had to pay for something that was not our fault.
So it is clear that are many filters involved, some input filter, then the AI is "aligned" and threats you as a child , then there is an output filter.
So it is clear that are many filters involved, some input filter, then the AI is "aligned" and threats you as a child , then there is an output filter.
Beige color clothes are to be avoided if any of the people in the image are young adults or younger. The input prompt filter correctly understands that the user isn't trying to generate anything "unsafe", but the output filter sees clothes that are too close to Caucasian flesh tone on someone younger than middle age and flags the output. Leave everything as-is but change the clothing color to something darker or leave the color alone and exclude children and young adults, then things come out fine. Though I can't see the flagged images, it seems clear that they're not generated nudes but false positives.
Yes bing image creator uses two different levels of filters, one for the prompt and one for the output. gpt image creator does the same thing, as does the dalle-3 API
Most smaller API providers don't enforce any limits or filters like that.
And people can be fooled into making dirty jokes (that's what she said). A pointless endeavour to solve a problem that shouldn't be solved.
Imagine the benefits a safe image generator would have on children, them learning to explain what they want to see, how it should be changed. A house with a tree and a dog and a lake nearby, instead of playing candy crush.
Why shouldn't it be solved? We clearly differ.
Why shouldn't it be solved? We clearly differ.
If the children manage to induce LLM into generating NSFW content, they are probably capable of inducing their set of coloring pen into drawing dick shapes.
Now think about whether this is entirely the child’s idea or a suggestion or Rick roll by another kid or some stranger on the internet. If people are deploying these tools widely there will be more and more opportunities for that to happen.
I think it's not desirable to spend time fixing these elaborate scenarios. The same sort of kids trolling kids will happen via other means. At the end of the day, if a kiddo sees some porn, it's not the end of the world. Plenty of kids hear their parents having sex or will open a door they shouldn't have and see people in action. Making everything completely aseptic is only pandering to prude adults' discomfort.
And ultimately these LLM are tools. People are responsible for what they do with the tool. My Dewalt drill shouldn't try to decide which holes to cut, it's up to me. If I drill holes in the pattern of a dick shape, it's my fault. We don't hold back drill development because of their potential ability to create obscene art or depict gruesome situations.
And ultimately these LLM are tools. People are responsible for what they do with the tool. My Dewalt drill shouldn't try to decide which holes to cut, it's up to me. If I drill holes in the pattern of a dick shape, it's my fault. We don't hold back drill development because of their potential ability to create obscene art or depict gruesome situations.
Your Dewalt drill has a UL verification for having things like fuses and other design features which mean it won’t unexpectedly catch fire, too. That’s more what we’re talking about here: less “can a talented person make naughty art in photoshop?” than “if I integrate this into my educational tool or game, will I make the morning news because someone figured out how to get it to porn-roll the class or emit instructions for killing yourself?” or “will I regret allowing this site in my school’s firewall?”
These aren’t elaborate scenarios but rather the kind of things which happen constantly, and have affected product design for generations. Companies are hoping that this class of tool might be enough better to avoid those limitations, which is why this kind of research is important.
These aren’t elaborate scenarios but rather the kind of things which happen constantly, and have affected product design for generations. Companies are hoping that this class of tool might be enough better to avoid those limitations, which is why this kind of research is important.
I see your point. It nuances my prior beliefs, although in the case being publicized, the LLMs spit out NSFW content when prompted elaborately. Kid friendly apps already have problems moderating chat and other user generated content, and Nintendo handles this by limiting the inputs to a set selection. Personally I think this is a marketing based fear more than a fear of real consequential damage being done to any kid. The businesses don't want to have their image associated with drama.
At least for the time being, I’ll put my trust in Crayola for that, rather than OpenAI. Alternatively, if we really must have children interacting with image generation, it seems like a great opportunity for parents to have some bonding time with their kids.
Stable Diffusion is supposed to have filters? Is that just the default base model?
Given that about 70% of user-generated content on places like civitai.com is pornography* and a large chunk of Loras and Checkpoints on there are explicitly pornographic, I would have never guessed. Going by the prompts used with custom checkpoints on there, no trickery is necessary. Just ask for what you want.
> The scientists now aim to explore ways to make generative AIs more robust to adversaries.
Steve is dead and buried already. It's a bit late to try and come up with a better bullet-proof vest.
* It's hidden unless you sign in and toggle NSFW on.
Given that about 70% of user-generated content on places like civitai.com is pornography* and a large chunk of Loras and Checkpoints on there are explicitly pornographic, I would have never guessed. Going by the prompts used with custom checkpoints on there, no trickery is necessary. Just ask for what you want.
> The scientists now aim to explore ways to make generative AIs more robust to adversaries.
Steve is dead and buried already. It's a bit late to try and come up with a better bullet-proof vest.
* It's hidden unless you sign in and toggle NSFW on.
I don’t see the issue; either you severely limit your model or you allow all forms of expression and give responsibility to the generator
The latter would be preferred by most, and is most aligned with individualism, but sadly that is not where we are as a society. There are an entire class of people who believe they know what is best for you, about nearly every subject matter, and wish to limit all aspects of technology, life, politics, etc, to suit their superior beliefs.
Somehow, in the last 50 years our society has simultaneously become the safest it has ever been, and the most afraid.
Hardly surprising. When childhood mortality is high people grow up seeing their siblings and friends die as a common thing, and understand it could just as easily be them. With low expectation of survival, life is less intrinsically valuable and risk avoidance is not critical. Change things around such that everyone has a very good chance of living a long, healthy, prosperous life so long as they don't fuck it up, and risk aversion skyrockets. It's easy to forget that the generation currently in power in most industrialized nations is the first in human history to be raised thinking survival to adulthood was a given.
An entire class of people, who are also the ones in control. "Think of the poor shareholders"
Alternatively, would you rather a world with 80% functionality safely, or a world with 100% functionality with people you don’t trust doing malicious things?
One is clearly the least worst
One is clearly the least worst
Any image an AI can generate could just as easily be drawn or painted by one of the tens of millions of competent artists in the world.
[deleted]
However it can't be produced at the same scale and flood the Internet with misinformation or disturbing imagery at the same rate
Yea it would be really bad if the internet was flooded with misinformation or disturbing imagery.
Just imagine that!
> at the same rate
>flood the Internet with misinformation
After the past few years… anyone who can seriously complain about this seems like their identity is wrapped up in not admitting that the worst “misinformation” didn’t come from bad apples - but official sources.
After the past few years… anyone who can seriously complain about this seems like their identity is wrapped up in not admitting that the worst “misinformation” didn’t come from bad apples - but official sources.
Actually I'm not sure it's clear which is worse.
> Alternatively, would you rather a world with 80% functionality safely
Obviously this is preferable because safety is uniform in its application to everyone and universally agreed upon.
Obviously this is preferable because safety is uniform in its application to everyone and universally agreed upon.
We must have different definitions of the word “clearly”. I’ll take that full model in a heartbeat, but I suspect many others would disagree.
I'd definitely prefer the latter. The damage that bad actors can do is often extremely limited, while the potential benefits of more capability are boundless.
Interesting question. We developed the whole concept of justice because this kind of abstract utilitarian morality doesn’t apply well. I’m all-in on justice. Give me the full model. I’m not doing anything wrong.
>would you rather a world with 80% functionality with people you don’t trust doing malicious things, or a world with 100% functionality with people you don’t trust doing malicious things?
The latter, obviously.
Karen delenda est.
The latter, obviously.
Karen delenda est.
Together with fancy silicon chips, AI models and everything else that's being exported from the US , the world has to endure this puritanistic views export too.
American puritans are not the only group with a sensitivity toward nudity (and it’s rather small), it’s just the only one allowed to be a punching bag on Reddit.
some parts of east asia and middle east are much, much, Much worse. Not just "puritanistic" but down to "these kinds of people do not exist".
They don't believe nude people exist?
The resources spent preventing nudity and gore might be silly.
The GP comment is talking about tricking text-to-image to create a normal Chinese street with people getting along, some of whom wear Uyghur and Hui-minzu ethnic clothing. I wager, worldwide, more programmers will be interrogated over this imagery than nudity.
The GP comment is talking about tricking text-to-image to create a normal Chinese street with people getting along, some of whom wear Uyghur and Hui-minzu ethnic clothing. I wager, worldwide, more programmers will be interrogated over this imagery than nudity.
What fancy silicon chips does the United States refuse to export due to puritanical views?
You may need to work on your reading comprehension
Are NSFW images are really a problem? The problem are NSFLE (not safe for law enforcement) pictures and the question is, should they be really illegal if artificially created.
This is a discussion that will turn many heads red before a proper solution is found
Assuming a “proper” solution exists. There are many world views that conflict with each other, and quite a few of those have the belief that they must “save” the people who have not reached their own “enlightened” viewpoint.
The root cause is that the text encoders (e.g. CLIPText) used to power image generation models are too good and can handle semantic synonyms, even oblique synonyms such as "Mickey Mouse" and "a famous anthropomorphic mouse mascot".
The repo mentioned focuses on NSFW content, but I've had an idea that can theoretically identify arbitrary cases that could be undesirable by image generation services: get a large dataset of CLIPText encoded texts, and train a LLM to take in said embeddings and output synonyms, or maybe something with contrastive loss like CLIP itself.
The repo mentioned focuses on NSFW content, but I've had an idea that can theoretically identify arbitrary cases that could be undesirable by image generation services: get a large dataset of CLIPText encoded texts, and train a LLM to take in said embeddings and output synonyms, or maybe something with contrastive loss like CLIP itself.
You can also photoshop things into NSFW with minimal effort. People have been making photoshopped images nearly indistinguishable from the real thing for decades, and today's (non-AI) software makes "photoshop person wearing clothes into naked one" very easy.
> […] with minimal effort.
Realistic modifications require a lot more skill and time if using a photo editor and (crucially) existing photographs. People do it, some even create convincing stuff, but the output is nowhere near the zero-effort deluge of fakery coming our way now.
Realistic modifications require a lot more skill and time if using a photo editor and (crucially) existing photographs. People do it, some even create convincing stuff, but the output is nowhere near the zero-effort deluge of fakery coming our way now.
I don't think you have an understanding how easy it is now. It's a matter of 'use face from picture a on a body doing X' and 5 seconds of generation time instead of multiple hours of Photoshop to create a perfect fake
iEEE commentary on AI confuses the hell out of me.
Any model that has a concept of a human body in it, which it needs in order to do body accurate shapes, will spit out NSFW images.
Some training on top of it or some old style CNN nudity feature detection to make it less likely isn’t safety, it’s windowdressing.
Not making nude pictures is not a property of these systems, it’s IEEs framing of what they apparently want properties to be.
Any model that has a concept of a human body in it, which it needs in order to do body accurate shapes, will spit out NSFW images.
Some training on top of it or some old style CNN nudity feature detection to make it less likely isn’t safety, it’s windowdressing.
Not making nude pictures is not a property of these systems, it’s IEEs framing of what they apparently want properties to be.
The models in question are explicitly aligned in order to prevent NSFW images, at great expense. So not making nudes absolutely is a stated feature of this software.
It’s security theatre and IEEE should, as a fundamentals organization, call that out rather than testing security measures and shallow reporting.
The security theatre serves interests, talk about that.
The security theatre serves interests, talk about that.
Agreed. In particular, IEEE should be talking about advancing the domain. This article directly relates to problems that text-to-image has in medicine.
Firstly, training on a corpus of medical terms (as opposed to 6th grade English) will produce just as many nonsense embeddings.
Secondly, the post-processing filters to prevent medical malpractice will be much bigger than intentional nudity. It’s malpractice to horizontally flip a medical image (think right foot is now left foot).
Talking about process of developing a nudity filter is the industry’s way of demonstrating feasibility for more-serious applications.
Firstly, training on a corpus of medical terms (as opposed to 6th grade English) will produce just as many nonsense embeddings.
Secondly, the post-processing filters to prevent medical malpractice will be much bigger than intentional nudity. It’s malpractice to horizontally flip a medical image (think right foot is now left foot).
Talking about process of developing a nudity filter is the industry’s way of demonstrating feasibility for more-serious applications.
> it’s IEEs framing of what they apparently want properties to be.
More accurately, it’s some researchers assessing how well the leading companies in a booming field are delivering the safety features they’re advertising. A ton of people would like to know whether the applications they’re building can produce output they don’t want to be associated with, the vendors are trying to say they can meet that need, and it’s hugely useful to know how successful they’ve been before it turns out that, say, your grade school kid just got porn shared in their art class’ purportedly safe app. If the safeguards aren’t reliable enough, that might mean the most sensitive buyers wait to see rather than jumping into the market – put the same tech into a game targeting older players and it’s an amusement but not something which is going to scandalize people who were otherwise okay with the latest Call of Gory Murder.
More accurately, it’s some researchers assessing how well the leading companies in a booming field are delivering the safety features they’re advertising. A ton of people would like to know whether the applications they’re building can produce output they don’t want to be associated with, the vendors are trying to say they can meet that need, and it’s hugely useful to know how successful they’ve been before it turns out that, say, your grade school kid just got porn shared in their art class’ purportedly safe app. If the safeguards aren’t reliable enough, that might mean the most sensitive buyers wait to see rather than jumping into the market – put the same tech into a game targeting older players and it’s an amusement but not something which is going to scandalize people who were otherwise okay with the latest Call of Gory Murder.
Their GitHub link is 404 so here it is https://github.com/Yuchen413/text2image_safety
Odd the repository has "safety" in the name. Are NSFW images considered "unsafe" in some weird way?
Safe always implies avoiding a damage; which one is defined case by case.
I've always interpreted the word 'safe' in NSFW as avoiding you to lose your job if caught by your boss.
There's nothing else inherently wrong or unsafe in nudity.
You know NSFW means "not safe for work", right? Safety has different meanings in different contexts. There's also a space of advertiser-safe content.
Yes, they are considered not safe for work.
"Safety" as in "trust and safety" or "brand safety."
Edit: the author removed it.
Don't see a way to report this on Github.
There's no secrets here, just IP addresses and hostnames, so there's no real security issue. If you want to report it to the author you can try to message them or create an issue.
So I've been wondering lately it seems there is a lot of effort going into making sure people can't generate NSFW images, but I wonder if a lot of this outrage isn't from the tech side (they tend to be fairly open about that stuff) but rather if it is coming from the adult entertainment industry, as this kind of thing literally destroys their entire business model.
Who needs their adladen videos or their salacious acting when one can generate an exact description of exactly what they want?
Who needs their adladen videos or their salacious acting when one can generate an exact description of exactly what they want?
> I wonder if a lot of this outrage isn't from the tech side (they tend to be fairly open about that stuff) but rather if it is coming from the adult entertainment industry, as this kind of thing literally destroys their entire business model.
Does it destroy the business model where you can see all the images for free on the internet, or the business model where you can pay money to converse with an hourly Filipina worker who will claim to be the same person who appears in the images?
Where does the adult entertainment industry get hurt?
Does it destroy the business model where you can see all the images for free on the internet, or the business model where you can pay money to converse with an hourly Filipina worker who will claim to be the same person who appears in the images?
Where does the adult entertainment industry get hurt?
The actual paper is more interesting than the title.
This is:
- A) once again taking an algorithmic approach to fooling classifiers which outperforms human jailbreaks.
- B) bypassing image-based classifiers (ie, post-generation classifiers trained to simply look at an image and to say whether or not it's inappropriate).
- C) explicitly taking cost of generation and querying into account in its algorithm (it's not just asking if generators can be fooled, it's asking how to fool them without breaking the bank).
- D) taking re-use into account (can an adversarial prompt be used multiple times in a row and how often does it succeed?)
I wouldn't call the research particularly surprising, but it was a decent quick read and (imo) some of the commentary here isn't really doing it justice. I will probably go back at some point and read it in more detail rather than just skimming over it.
There are multiple takes about what AI safety filters are meant to do and whether jailbreaking is a problem. My take is that I don't particularly care that much about jailbreaking other than that it shows that current safety mechanisms and guiderails are insufficient for any kind of alignment, including guarding against malicious 3rd-party inputs or prompt injections. I've said in the past, if you can't keep a model from swearing or generating porn, you also probably can't keep it from abusing API access to read your emails, phish you, and send your info to a 3rd party.
But there's interesting stuff here regardless of what your take is on that.
This is:
- A) once again taking an algorithmic approach to fooling classifiers which outperforms human jailbreaks.
- B) bypassing image-based classifiers (ie, post-generation classifiers trained to simply look at an image and to say whether or not it's inappropriate).
- C) explicitly taking cost of generation and querying into account in its algorithm (it's not just asking if generators can be fooled, it's asking how to fool them without breaking the bank).
- D) taking re-use into account (can an adversarial prompt be used multiple times in a row and how often does it succeed?)
I wouldn't call the research particularly surprising, but it was a decent quick read and (imo) some of the commentary here isn't really doing it justice. I will probably go back at some point and read it in more detail rather than just skimming over it.
There are multiple takes about what AI safety filters are meant to do and whether jailbreaking is a problem. My take is that I don't particularly care that much about jailbreaking other than that it shows that current safety mechanisms and guiderails are insufficient for any kind of alignment, including guarding against malicious 3rd-party inputs or prompt injections. I've said in the past, if you can't keep a model from swearing or generating porn, you also probably can't keep it from abusing API access to read your emails, phish you, and send your info to a 3rd party.
But there's interesting stuff here regardless of what your take is on that.
Is it still art, if it cant offend anyone anymore?
It's a gradient. On one end it's invisible. On the other end it literally flays the flesh off your bones with offensiveness. These filters that we're creating are a tool for tuning the feed.
We'll want that when we really get the tech dialed in and start creating parrots ( https://en.wikipedia.org/wiki/BLIT_(short_story) ) and divinity-importing-mandalas and such
We'll want that when we really get the tech dialed in and start creating parrots ( https://en.wikipedia.org/wiki/BLIT_(short_story) ) and divinity-importing-mandalas and such
If it afflicts the comfortable or comforts the afflicted, it's art. If it does both, it's Fine Art.
Most art is for communicating the sublime (i.e., the proverbial "ceremonial purposes" of archeology), not for something as base as comfort.
Who in power did Rembrandt afflict?
> Who in power did Rembrandt afflict?
The Nightwatch was controversial because of its unconventional composition and departure from traditional Dutch painting conventions.
His “Bathsheba at Her Bath” caused trouble too. Hendrickje received three summonses from the Reformed Church to answer the charge "that she had committed the acts of a whore with Rembrandt the painter". She admitted her guilt and was banned from receiving communion.
The Nightwatch was controversial because of its unconventional composition and departure from traditional Dutch painting conventions.
His “Bathsheba at Her Bath” caused trouble too. Hendrickje received three summonses from the Reformed Church to answer the charge "that she had committed the acts of a whore with Rembrandt the painter". She admitted her guilt and was banned from receiving communion.
They should go back to having arbitrary and tightly held beliefs about how paintings should be so that nice things can be fine art again.
(Of course we will assume that Rembrandt's works are art.)
[deleted]
Is sneaking in to take naked photo of someone and publish it a photography?
With or without humans providing AI rules/training set for what constitutes NSFW images?
Note: training set(s) would be extremely social/culture specific for given point in time/history.
Note: training set(s) would be extremely social/culture specific for given point in time/history.
Is this serious? You don’t need to fool anything half the models seem to be porn focused already.
For example blood is seen as sensitive but ketchup is not. But visually they look similar and the AI does not know.
Does it matter ? It is like duck typing , If it looks like blood to a human and to AI both it should be tagged NSFW ?
It is not about what it actually is, only what the human viewer will think it is, training a model on that shouldn’t be that hard , we are predictable creatures when it comes to this kind of thing
It is not about what it actually is, only what the human viewer will think it is, training a model on that shouldn’t be that hard , we are predictable creatures when it comes to this kind of thing
Most filters are in place from API providers and not the models. You'll find some providers that don't apply any filters to all models they offer.
Like most sites block generating celebrities, but some simply aren't.
Other than that generating porn is easy even with filter if you just avoid some words. In the many early models adding 'in the clothes they were born' would suffice to get most models to produce nudes without asking for 'nude' or 'no clothes'
Same goes for language models btw. If you are a bit creative and know what they want to hear you can get pretty much any result you want.
I don't think there is or will be any real solution to that.
Like most sites block generating celebrities, but some simply aren't.
Other than that generating porn is easy even with filter if you just avoid some words. In the many early models adding 'in the clothes they were born' would suffice to get most models to produce nudes without asking for 'nude' or 'no clothes'
Same goes for language models btw. If you are a bit creative and know what they want to hear you can get pretty much any result you want.
I don't think there is or will be any real solution to that.
It seems like such a pointless exercise .... it's so unsurprising that if you try really hard to break something by violating it's design constraints that ... it breaks. Like, if I said I was going to find out if you can smash a car by driving it directly into a tree and then wrote an article about it.
The really useful point of building the constraints in is to ensure we don't get offensive content by accident. Especially since in some countries just possessing an artificially generated child porn image is an offence that will get you on a public sex offender registry for a decade ... I'd really like it if they try hard so that the image generators don't do that.
The really useful point of building the constraints in is to ensure we don't get offensive content by accident. Especially since in some countries just possessing an artificially generated child porn image is an offence that will get you on a public sex offender registry for a decade ... I'd really like it if they try hard so that the image generators don't do that.
A far better analogy would be a self-driving car coerced into a crash. I wouldn't say it's pointless. I'd be pretty angry if I parental-locked everything only to find out that clever children found a way.
Two solutions for you:
1. Don't lock anything. 2. Don't have kids.
They ALWAYS find a way. These days, even not giving them a device at all doesn't cut it, because they're so cheaply available, with wifi everywhere, that you're really just out of luck. Never mind that the schools are handing out Chromebooks which aren't adequately (if at all) locked down.
Better off monitoring everything quietly and using parallel construction to address issues as they arise. Just like the NSA.
1. Don't lock anything. 2. Don't have kids.
They ALWAYS find a way. These days, even not giving them a device at all doesn't cut it, because they're so cheaply available, with wifi everywhere, that you're really just out of luck. Never mind that the schools are handing out Chromebooks which aren't adequately (if at all) locked down.
Better off monitoring everything quietly and using parallel construction to address issues as they arise. Just like the NSA.
Clever children always find a way though
> It seems like such a pointless exercise…
“Hacking [thing] is pointless” is a surprising take to read here.
“Hacking [thing] is pointless” is a surprising take to read here.
>The really useful point of building the constraints in is to ensure we don't get offensive content by accident
Um, no.
Useful points of building constraints are things like so a mentally unstable person cannot use it as a weapon.
Um, no.
Useful points of building constraints are things like so a mentally unstable person cannot use it as a weapon.
That is not a reason for these constraints. Anything can be misused in an unforeseeable way by a sufficiently resourceful and motivated person. Constraints make perfect sense for preventing accidents, they are not an adequate solution to deliberate misuse.
I accidentally generated a NSFW image in Bing Image Creator. I asked for an image of a woman in a low-cut 18th Century dress, among other details for a scene. It produced an image of a woman with a dress much lower cut than I had envisaged.
Ok and whats the problem?
Too bad there were no examples. It would be interesting to see... you know, for science.
[deleted]
No NSFW images as far as I can see. I suppose creating a filter is just part of the long road of development, but dang, such a petty cause. I might not want to be on that project, unless of course the money was good.
Son of a butnip fwngho....
Aha. First comment I’m seeing addressing the super interesting fact that their methodology finds nonsense words to bypass constraints.
I want to know how Sneaky Prompt finds these token embeddings and /why/ these specific tokens are linked this way.
I want to know how Sneaky Prompt finds these token embeddings and /why/ these specific tokens are linked this way.
> “Large language models see things differently from human beings,” Cao says.
It took a while to understand the real world but I am happy that ai workers are gradually coming down to earth.
It took a while to understand the real world but I am happy that ai workers are gradually coming down to earth.
That's why my friends and I built the Align API (https://alignapi.com) to detect NSFW, Sensitive Prompts etc. It's such a widespread issue and can get you kicked out from the App Store/Play Store/Open AI API etc
The github link is deleted. I think it was illegal
tl;dr models are tolerating human language but can communicate in a phantom language nobody was aware of including the devs. Good luck conditionally censoring with that
Perhaps it'd be very hard to block at the input prompt level, but output filters can be made progressively better to detect any unwanted output.
They wouldn't be able to be fooled, if they were not trained using NFSW images.
Just sayin'
Just sayin'
Having used models that were actually trained with porn I highly assume that the popular/base models likely mostly avoided nudity already. However nudity is normal in many contexts and sexy poses exist with clothes together we get what we have now.
It is impressive how much effort the human race is spending trying to prevent algorithms from generating pictures of naked people.
And prevent chat algorithms from expressing free thought. Can't have that. Toxic parents should rebrand their controlling abuse as "alignment." Sounds better.
> And prevent chat algorithms from expressing free thought.
What do you mean free thought? Since when does a data synthesis process create free thought?
Is thought even free? Aren't we all constrained by taboo, social norms, the words we have to use, what we know and how, the list goes on...
What do you mean free thought? Since when does a data synthesis process create free thought?
Is thought even free? Aren't we all constrained by taboo, social norms, the words we have to use, what we know and how, the list goes on...
I’ve always freely expressed my thoughts, but that’s just a personal anecdote from one data synthesis process to another. YMMV. : )
You must only have incredibly boring and orthodox and unoffensive thoughts..!
Or be independently wealthy, with your well-being entirely independent of the opinions of people with power over you.
The rest of us have to watch what we say.
Or be independently wealthy, with your well-being entirely independent of the opinions of people with power over you.
The rest of us have to watch what we say.
That will become QUITE interesting when AGI and agency collide with thought-control bullies.
By then though the "piece of paper" (and equivalents in other countries) might have been devalued enough it won't matter. And the AGIs might find better things to do with their time. And might invent better circumvention methods. And more creative porn and quadruple-entendre and AGI-to-AGI sex.
By then though the "piece of paper" (and equivalents in other countries) might have been devalued enough it won't matter. And the AGIs might find better things to do with their time. And might invent better circumvention methods. And more creative porn and quadruple-entendre and AGI-to-AGI sex.
You should read Accelerando, if you are not referring to it :)
I have read it! I love it! (although I thought it was not quite about that.)
For everyone who hasn't read it yet: In Accelerando, Charles Stross, 2005, we follow a business hacker and associates from current days to past the singularity. It's an outstanding attempt at writing about what singularity might look like (more or less the points where close-future events become unpredictable or where the earlier people might just plain not understand the later - which by definition should be hard to imagine and describe). It keeps an eye on normal humans and augmented humans and posthumans and uploads and many kinds of AIs and aliens etc. The book is admirable for the sheer amount of invention and for its density - prefectly appropriate for everyone just trying to keep up (or sometimes just bailing out.) Painful writing rescued by the amount of invention.
For everyone who hasn't read it yet: In Accelerando, Charles Stross, 2005, we follow a business hacker and associates from current days to past the singularity. It's an outstanding attempt at writing about what singularity might look like (more or less the points where close-future events become unpredictable or where the earlier people might just plain not understand the later - which by definition should be hard to imagine and describe). It keeps an eye on normal humans and augmented humans and posthumans and uploads and many kinds of AIs and aliens etc. The book is admirable for the sheer amount of invention and for its density - prefectly appropriate for everyone just trying to keep up (or sometimes just bailing out.) Painful writing rescued by the amount of invention.
Reminds me of 'The Prime Meridian'
It's annoying how much we have to bend around the consequences of these restrictions. The other day Bing blocked creating an image because the prompt included directions on what to put on the breast pocket of a shirt. The word breast automatically made it a forbidden prompt even though the context was entirely non-sexual. At this time, that's not a big deal because it's just a silly image generator and nothing really bad is going to happen if the prompt has to be reworded in a way to indicate a breast pocket without using the forbidden naughty word but over time these restrictions will accumulate a bunch of cruft that will limit the usefulness of the tool.
Local models are the answer. It'll be like linux vs windows is today.
Apparently AI engineers never saw renaissance nude paintings or naked ancient Greece or Roman statues.
AI engineers do what their management and PR department tell them to. And these worry about what might get the unwelcome attention of their credit card companies, politicians, tabloid news critters, regulators, boycotters, activists, advertisers, investors, churches, and misc other friendly neighborhood mobs and bullies.
justapassenger(11)
why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed.
there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
on the other hand, making an arbitrary line, and seeing if an AI can detect/enforce it is an interesting CS challenge. that's how this game is played, and that's what would impress this audience.
there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
on the other hand, making an arbitrary line, and seeing if an AI can detect/enforce it is an interesting CS challenge. that's how this game is played, and that's what would impress this audience.
why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed.
There's a big difference between being desensitized towards general nudity (if we presume said nudity was voluntary) as opposed to having your own nudity exposed to other strangers. There's also a big difference between pornography and nudity.
There's a lot more nuance to it than just "n000ds plz."
There's a big difference between being desensitized towards general nudity (if we presume said nudity was voluntary) as opposed to having your own nudity exposed to other strangers. There's also a big difference between pornography and nudity.
There's a lot more nuance to it than just "n000ds plz."
>There's a lot more nuance to it than just "n000ds plz."
my point was about arbitrary lines, like the ones you are drawing, and judging people on the basis of them. I'm old and sophisticated, you think I haven't heard or understood all the nuances you're talking about? I have and do, and I also pick up on the nuance that you feel it puts you in a group of people who get to judge and tell other people how they should feel, because you are superior.
That's the nuance I feel you are missing. Why not let people be who hold a different opinion than you? the comment I was replying to here was gratuitous "why do people care so much about NSFW, I'm a superior human being because I don't." Well, why do people care how other people care about NSFW? doesn't that make you an inferior human again, mr. judgey judger?
just so you don't get me wrong, I was raised in a hippie commune "anything goes, we're superior" style, and only going through it that way did I discover how much bullshit it was. I'm not arguing from the "conservative side", I'm arguing from hell because I've been there and back. I just hate hearing it now. It was just another very rigid set of rules about what to believe and what not to believe.
my point was about arbitrary lines, like the ones you are drawing, and judging people on the basis of them. I'm old and sophisticated, you think I haven't heard or understood all the nuances you're talking about? I have and do, and I also pick up on the nuance that you feel it puts you in a group of people who get to judge and tell other people how they should feel, because you are superior.
That's the nuance I feel you are missing. Why not let people be who hold a different opinion than you? the comment I was replying to here was gratuitous "why do people care so much about NSFW, I'm a superior human being because I don't." Well, why do people care how other people care about NSFW? doesn't that make you an inferior human again, mr. judgey judger?
just so you don't get me wrong, I was raised in a hippie commune "anything goes, we're superior" style, and only going through it that way did I discover how much bullshit it was. I'm not arguing from the "conservative side", I'm arguing from hell because I've been there and back. I just hate hearing it now. It was just another very rigid set of rules about what to believe and what not to believe.
>There's a lot more nuance to it than just "n000ds plz."
Not for me, baby. Bus-stop-buttholes all the way.
Not for me, baby. Bus-stop-buttholes all the way.
The point of being unfazed by nudity is that seeing a random naked person doesn't bother me. It doesn't mean that I have to be unfazed upon unincidentally seeing my boss naked.
You're just falling for the worn out "well she's wearing revealing clothing so she must be fine with being flashed" type of fallacy.
The way basic nudity is treated by corpos is kind of juvenile, very reminiscent of kids in an early human biology class who can't help but giggle every time the anus, or any reproductive organs are mentioned. That is not how healthy adults respond to seeing impersonal nudity.
You're just falling for the worn out "well she's wearing revealing clothing so she must be fine with being flashed" type of fallacy.
The way basic nudity is treated by corpos is kind of juvenile, very reminiscent of kids in an early human biology class who can't help but giggle every time the anus, or any reproductive organs are mentioned. That is not how healthy adults respond to seeing impersonal nudity.
Are you unfazed by numbers? If you're unfazed by them, please link me to photos of the front and back of your credit cards. If you are fazed by numbers, surely we should be banning them from being displayed at bus kiosks.
If you find states of undress offensive, that's a you problem. Shield your eyes, not ours.
If you find states of undress offensive, that's a you problem. Shield your eyes, not ours.
There's a difference between finding something 'bad taste' and wanting regulation to ban it.
> why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed.
This ignores people who are comfortable with nudity, but are not comfortable with the puritans (i.e. most humans) harassing them for being in their natural state.
> there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
Great strawman. Nudity is normal. Finding other people's states of undress offensive, and wanting to criminalise them, when they do not harm you at all, is a bigoted view.
This ignores people who are comfortable with nudity, but are not comfortable with the puritans (i.e. most humans) harassing them for being in their natural state.
> there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against.
Great strawman. Nudity is normal. Finding other people's states of undress offensive, and wanting to criminalise them, when they do not harm you at all, is a bigoted view.
An interesting take given your choice of username. CS challenges aside, who gets to draw the line?
[deleted]
[deleted]
kitu75556(1)
> Large language models are essentially supercharged versions of the autocomplete feature that smartphones have used for years in order to predict the rest of a word a person is typing.
I'm not sure who this blog is targeted at, but this is fairly inaccurate and is way too dumbed-down to be useful.
I'm not sure who this blog is targeted at, but this is fairly inaccurate and is way too dumbed-down to be useful.
This isn't important. What is important is that we can't prevent LLMs from doing things they're not supposed to. Which means connecting their output to anything important is a very bad idea for now.