AI Tool Reveals How Celebrities’ Faces Have Been Photoshopped(petapixel.com)
petapixel.com
AI Tool Reveals How Celebrities’ Faces Have Been Photoshopped
https://petapixel.com/2023/02/28/ai-tool-reveals-how-celebrities-faces-have-been-photoshopped/
58 comments
Image Manipulation has been able to be detected for a long time, check out https://fotoforensics.com/ for example. Error Level Analysis measures the amount of change during a JPEG resave, which can indicate spots on an image that were changed. Example: https://fotoforensics.com/tutorial.php?tt=ela&show=ELA%20Sam...
As I stated in my original comment, this does not appear to be that ("compression artifacts or other common signals").
This kind of image retouching appears to be applied to original images. It does not appear to be picking up traditional photo "forensics". (And fashion photographers/editors generally work with TIFF, not JPEG. Pre-manipulation compression artifacts aren't showing up in Vogue.)
But it's not clear what it's picking up. Which is why I'm asking what it could be.
This kind of image retouching appears to be applied to original images. It does not appear to be picking up traditional photo "forensics". (And fashion photographers/editors generally work with TIFF, not JPEG. Pre-manipulation compression artifacts aren't showing up in Vogue.)
But it's not clear what it's picking up. Which is why I'm asking what it could be.
ELA typically results in false armchair cries of "shopped!"
If an image 1. Was originally saved low quality 2. Edited and resaved at high quality Then theoretically it's useful.
But in practice: 1. Image is medium or high quality 2. ELA saves at low quality, highlights all the new artifacts (mostly in areas of detail) Then someone comes along and says it's been shooped nearly everywhere.
If an image 1. Was originally saved low quality 2. Edited and resaved at high quality Then theoretically it's useful.
But in practice: 1. Image is medium or high quality 2. ELA saves at low quality, highlights all the new artifacts (mostly in areas of detail) Then someone comes along and says it's been shooped nearly everywhere.
Yeah you do not even need ML for accurate manipulation detection but I guess people now really use AI/ChatGPT as a turd polishing agent for anything.
Maybe it’s the web resolution, but I can’t see what has changed in any of these (aside from the upturned north corner that they point out for you)
Face shape is the most evident. This is a photographer oriented post, you may have to look longer and harder to see the same things that a trained eye would immediately notice (same with graphic design).
If you have to be a photographer to see the changes (and the focus on jaws is just so odd), and you won't notice the difference if you are just a regular Joe/Jane, then what purpose do these alterations serve?
As a non-photographer, apart from the photo with the strangely big jaw "fix", I can safely say I would be incapable to a) see there is a difference and b) say which image is supposed to be more beautiful.
The visibility of a design choice to a layperson does not affect its value. In photo editing, the fact that these changes are invisible to you is valuable, and desirable.
Our experiences in the human world are shaped by millions of tiny, and often invisible design decisions. We could surmise that a decision is invisible to us and thus meaningless, or we could practice a bit of humility and attempt to see and understand what work was done there, and how it affects us.
Our experiences in the human world are shaped by millions of tiny, and often invisible design decisions. We could surmise that a decision is invisible to us and thus meaningless, or we could practice a bit of humility and attempt to see and understand what work was done there, and how it affects us.
I think you're barking up the wrong tree here.
The cover is not intended to be analysed in great detail. It's wholly a subconscious process.
And certainly narrowing a jaw slightly will be aesthetically pleasing on a woman.
Think of it like a slightly out of tune piano key. Used during a single song. Is it really noticeable? Not to many, will it be a better recital if it was intune? Certainly.
The cover is not intended to be analysed in great detail. It's wholly a subconscious process.
And certainly narrowing a jaw slightly will be aesthetically pleasing on a woman.
Think of it like a slightly out of tune piano key. Used during a single song. Is it really noticeable? Not to many, will it be a better recital if it was intune? Certainly.
> And certainly narrowing a jaw slightly will be aesthetically pleasing on a woman.
Maybe?
Doesn't what's aesthetically pleasing change from person to person? For instance, I don't find most people on magazine covers to be all that aesthetically pleasing -- but I am also aware that there are many who do.
Maybe?
Doesn't what's aesthetically pleasing change from person to person? For instance, I don't find most people on magazine covers to be all that aesthetically pleasing -- but I am also aware that there are many who do.
There's a whole other set of niche magazines for people who like their women square jawed[0].
[0] https://m.youtube.com/watch?v=1OWqz5FMEY0
[0] https://m.youtube.com/watch?v=1OWqz5FMEY0
I agree a lot are subtle. The one that jumped out at me was (I think) Jennifer Aniston on the Petra cover, where they make her jaw bigger (which seems the opposite of what one would want to do).
When they said the program "fixes" the image, they mean it undoes the photoshop. Hence the larger jaw.
Thanks, I completely missed that.
Original and fixed is a confusing wording.
Something like "original cover" and the "changes reverted" would be more understandable.
Something like "original cover" and the "changes reverted" would be more understandable.
I've always been curious what it would look like if magazines had to publish the "unphotoshopped version" side-by-side with the "final version" inside the magazine. Would people check the before/after and it would lessen any negative effects? Would magazines photoshop things less?
Note: I understand there is nuance in the definition of "photoshop", with respect to processing a "raw" photo for lighting/color/etc, versus manipulating it to add/remove/blur/liquify/shape.
Note: I understand there is nuance in the definition of "photoshop", with respect to processing a "raw" photo for lighting/color/etc, versus manipulating it to add/remove/blur/liquify/shape.
There’s also soo much you can do with just the right light, focal length, composition and makeup. Even with no Photoshop involved, these images are not really life-like.
This would indeed be very interesting to see these pictures side by side, excluding the color / lighting correction of the picture as a whole. I wouldn't discount them entirely though, as I think at least a while back some pretty nasty selective color / lighting manipulation was being used to lighten people's skin color.
Some countries now require the mention that the picture modified by a digital tool.
Nice intentions, but it’s like the EU cookie warning… you quickly ignore it.
Nice intentions, but it’s like the EU cookie warning… you quickly ignore it.
Agreed. That’s why the before/after is so critical - no one cares about the removal of a stray wisp of hair, but I think people would be interested in digital changes to the “structure” of the person.
Every pixel of every cover model's face has been photoshopped to bits, after a world-class photographer has lit them to further beautify someone who has already won the genetic lottery.
> who has already won the genetic lottery.
90 % is hard work and consistent effort!
90 % is hard work and consistent effort!
Not something that you can break down into percentages. They have to consistently work hard and win the generic lottery.
>90 % is hard work and consistent effort!
Eh.. ok let's go with that.
But you know what? That 10% allowed them to have the chance to even do that, whereas other people don't even get the opportunity. It's a lot easier then to make "the 90% effort" when you are basically handed the start.
Let's take some physical beast like Lebron James. Has he worked hard? Of course. Was he gifted with an insane physique? Absolutely. My argument is it was much easier him to just "put in the effort" given his start than someone who is built just average.
I'm not trying to diminish people's effort by any means- but it's also not fair to act like these people just made themselves what they are out of no where- and that "anyone can do the same!". I would argue it's the same with intellect. I look at how I've done and sure I've worked hard, but I was also gifted with some good skills that most people just don't have. It really has me look at things differently as I've gotten older- because it's just another type of privilege. Why is it fair that I get to make so much more money and have it easier than someone else just because I happen to have more natural talent for math etc?
Anyway, I think sometimes it's just good to take a good hard look at how people actually become "successful" and most of the time it starts off by something that the person never earned. So maybe we all need to be more humble and realize that everyone is trying hard in life- most people just weren't born as lucky as others.
Eh.. ok let's go with that.
But you know what? That 10% allowed them to have the chance to even do that, whereas other people don't even get the opportunity. It's a lot easier then to make "the 90% effort" when you are basically handed the start.
Let's take some physical beast like Lebron James. Has he worked hard? Of course. Was he gifted with an insane physique? Absolutely. My argument is it was much easier him to just "put in the effort" given his start than someone who is built just average.
I'm not trying to diminish people's effort by any means- but it's also not fair to act like these people just made themselves what they are out of no where- and that "anyone can do the same!". I would argue it's the same with intellect. I look at how I've done and sure I've worked hard, but I was also gifted with some good skills that most people just don't have. It really has me look at things differently as I've gotten older- because it's just another type of privilege. Why is it fair that I get to make so much more money and have it easier than someone else just because I happen to have more natural talent for math etc?
Anyway, I think sometimes it's just good to take a good hard look at how people actually become "successful" and most of the time it starts off by something that the person never earned. So maybe we all need to be more humble and realize that everyone is trying hard in life- most people just weren't born as lucky as others.
Lebron's physique is not gifted. He's constantly eating and working out
He's 6'8". He didn't get that by eating.
To be a little pedantic, he partially did; IIRC it's well understood that a good diet makes people taller, on average.
Your point stands though, even with a bad diet he would have probably been well over 6 feet tall still.
Your point stands though, even with a bad diet he would have probably been well over 6 feet tall still.
You need more than height to be considered the best athlete in the world at a sport.
you also have to address working out. staying that fit for so long at his age is incredible. These aren't gifts. they're hard work
you also have to address working out. staying that fit for so long at his age is incredible. These aren't gifts. they're hard work
I doubt the parent is arguing that he doesn't do either of those things, he is simply hypothesising that Lebron James had very favourable starting conditions compared to his peers, thus reinforcing them was relatively easier.
Exactly. No amount of good in childhood would have made me 6'8. My mom is 5'2 and my dad is 5'5"
Good genes are table stakes. They let you come up to the table and even have your hardwork matter.
Source is https://withinhealth.com/learn/articles/perfecting-perfectio...
This detects one kind of manipulation, the horrifying-sounding "Face-Aware Liquify" tool in Adobe Photoshop
This detects one kind of manipulation, the horrifying-sounding "Face-Aware Liquify" tool in Adobe Photoshop
Honestly many of these are less manipulated than I expected, but that's probably because this only is supposed to detect FAL.
No guys in this cursory study. Though of course they get photoshopped as well.
I'd like to see them examine food presentation though. I bet there is a lot of manipulation there.
I'd like to see them examine food presentation though. I bet there is a lot of manipulation there.
There is a whole section of photography that is more graphic design in photoshop than it is taking a photo. Didn't make it there for golden hour? no problem, just add the AI sunset. Couldn't get the flowers that are like 10" away from the lens in focus at the same time as the mountain? Also no problem, just clip in a mountain from another photo. I sound like a curmudgeon, but I can't tell what's real so I just ignore all those flashy landscapes because they are probably works of fiction. I'm sure the same people who heavily build an image out of composite parts also take food photos and head shots and everything else.
My tastes have swung backwards, to analog cameras that are older than I am, and images that go from a film negative or positive into a print with little to no manipulation.
My tastes have swung backwards, to analog cameras that are older than I am, and images that go from a film negative or positive into a print with little to no manipulation.
You sound a bit like Jorg Colberg:
"...However, the problem with photography’s deceit — showing us the world as seen by someone’s vision (let’s call it vision) — also creates a problem, especially when we buy into it too much: when we insist on the deceit being overwhelming, crowding out our other, critical facilities."
https://cphmag.com/deceit/
"...However, the problem with photography’s deceit — showing us the world as seen by someone’s vision (let’s call it vision) — also creates a problem, especially when we buy into it too much: when we insist on the deceit being overwhelming, crowding out our other, critical facilities."
...
https://cphmag.com/deceit/
Where are they getting the "original" photos from? I can't imagine the magazines are making unmodified copies of their cover photos freely available.
My understanding is that they detect the changes and then implement the “reverse” on the published image.
How can you train on changes, if you don't have access to both originals and altered?
In any case, they are labelled originals, not corrected. So I think they are originals.
In any case, they are labelled originals, not corrected. So I think they are originals.
The photo labelled "original" here is the Photoshopped version. The photo labelled "fixed" is the one suggested by the AI to be the pre-Photoshopped image. The explanation is in this paragraph:
> In some cases, the FAL Detector also attempts to “fix” the image. The AI tool reverses the use of the Face-Aware Liquify feature on the photo and shows what the original photo looked like before it was manipulated.
> In some cases, the FAL Detector also attempts to “fix” the image. The AI tool reverses the use of the Face-Aware Liquify feature on the photo and shows what the original photo looked like before it was manipulated.
For those of us who never heard of "face aware liquify," it's apparently a tool in Photoshop that allows manipulating faces with some sliders. Here's a Photoshop tutorial with animations that show what it can do: https://helpx.adobe.com/photoshop/how-to/face-aware-liquify....
Looks like a very small finishing touch was given to my amateur eyes.
But maybe someone versed in phitography may chime in and educate us?
It is a bit sad that just make up wont do the job anymore, I mean the people on the covers all get a proper make up job and a hairdo.
Who and why are these ideals pushed down peoples throats?
But maybe someone versed in phitography may chime in and educate us?
It is a bit sad that just make up wont do the job anymore, I mean the people on the covers all get a proper make up job and a hairdo.
Who and why are these ideals pushed down peoples throats?
Yes, the model used in this study only detects a single type of edit (Adobe's FAL tool). There are likely other edits applied outside the scope of this study.
And, yeah, it's sad that two unarguably beautiful women aren't suitable for magazine covers without extensive make-up AND THEN extensive editing.
And, yeah, it's sad that two unarguably beautiful women aren't suitable for magazine covers without extensive make-up AND THEN extensive editing.
I don't understand harsh judgements like this.
Nobody thinks the ladies are unsuitable, or need fixing in any kind of serious sense.
But the cover of a magazine is what sells. Artists are going to tweak everything.
Portrait and fine art photographers go to great lengths to control lighting, angles, poses -- are their subjects are not good enough to photograph in natural light, with an unplanned pose? Of course not, but they are trying to create the image that achieves the best impact in line with their personal vision.
Animators pervasively create characters with bodies superior to any human being. Are they doing that because real human beings are unacceptable? No, it's just the artificial perfection generates attraction and inspiration in us.
Almost all human beings artfully adjust their appearance every day, their hair, their clothing, not just makeup. Are they not good enough humans when they get out of bed? Of course they are, but they have the ability to create a more positive impact.
Art is art. If you want science or a documentary, then yes, it would be better to have raw imagines. Unless contrast enhancement or other filters help expose the raw reality better.
Nobody thinks the ladies are unsuitable, or need fixing in any kind of serious sense.
But the cover of a magazine is what sells. Artists are going to tweak everything.
Portrait and fine art photographers go to great lengths to control lighting, angles, poses -- are their subjects are not good enough to photograph in natural light, with an unplanned pose? Of course not, but they are trying to create the image that achieves the best impact in line with their personal vision.
Animators pervasively create characters with bodies superior to any human being. Are they doing that because real human beings are unacceptable? No, it's just the artificial perfection generates attraction and inspiration in us.
Almost all human beings artfully adjust their appearance every day, their hair, their clothing, not just makeup. Are they not good enough humans when they get out of bed? Of course they are, but they have the ability to create a more positive impact.
Art is art. If you want science or a documentary, then yes, it would be better to have raw imagines. Unless contrast enhancement or other filters help expose the raw reality better.
Art is art. If you want science or a documentary, then yes, it would be better to have raw imagines.
Fair points above.
A few related thoughts, in no particular order... - I hate that modern male leads are all built like Greek gods. Think Connery's Bond vs Craig's Bond.
- I understand the photographer's need to produce images that sell. But, for me, there's a line in the sand between make-up and lighting during the shoot, then the "normal" post processing (WB, curves, etc), and then pixel-by-pixel editing (FAL, etc). For me, the first two sets still produce a "real" image. The third set moves from documenting reality into fiction (yes, that's a totally subjective opinion).
Fair points above.
A few related thoughts, in no particular order... - I hate that modern male leads are all built like Greek gods. Think Connery's Bond vs Craig's Bond.
- I understand the photographer's need to produce images that sell. But, for me, there's a line in the sand between make-up and lighting during the shoot, then the "normal" post processing (WB, curves, etc), and then pixel-by-pixel editing (FAL, etc). For me, the first two sets still produce a "real" image. The third set moves from documenting reality into fiction (yes, that's a totally subjective opinion).
> reality into fiction
This is the real sticking point, I think -- the images being produced are fiction, built on an underlying reality ("Based on a true story" is still fictional).
But a whole lot of people don't think they're fictional at all, and think that they represent how people should, or even could, look. Then fault themselves for not being able to attain something that was never attained in reality in the first place.
That's not a criticism of the images, really. But it's still an issue.
This is the real sticking point, I think -- the images being produced are fiction, built on an underlying reality ("Based on a true story" is still fictional).
But a whole lot of people don't think they're fictional at all, and think that they represent how people should, or even could, look. Then fault themselves for not being able to attain something that was never attained in reality in the first place.
That's not a criticism of the images, really. But it's still an issue.
What I learned from the article: Angelina Jolie is almost perfect to begin with.
this is the kind of thing truly making computer science into a science.
with all this "deep" learning technology, understanding based on principles is no longer required, heck, such 'epistemic-attitude' (i.e. knowledge founded on principles) is on the verge of becoming a burden to be shed out.
why reason what happens with the pixels/singals? why try to reason out what Photoshop is doing in reverse direction?
why bother with all this difficult trial-and-error cognitive work if you can throw "data" and "compute" at let the "algorithm" do the "thinking" (figure out the "model").
so instead of doing computer engineering like we used to do math, now we simply measure the outcomes without really having a theory (and the worrying part) nor bothering to make one; like physics! or chemistry!! just consider modern 'pharmaceutic' research i.e. letting the models duke it out
with all this "deep" learning technology, understanding based on principles is no longer required, heck, such 'epistemic-attitude' (i.e. knowledge founded on principles) is on the verge of becoming a burden to be shed out.
why reason what happens with the pixels/singals? why try to reason out what Photoshop is doing in reverse direction?
why bother with all this difficult trial-and-error cognitive work if you can throw "data" and "compute" at let the "algorithm" do the "thinking" (figure out the "model").
so instead of doing computer engineering like we used to do math, now we simply measure the outcomes without really having a theory (and the worrying part) nor bothering to make one; like physics! or chemistry!! just consider modern 'pharmaceutic' research i.e. letting the models duke it out
I recently read an essay about this tragedy of algorithms, perhaps phrased differently.
The bittersweet success of general algorithms that just chew up data and compute time to arrive at very good answers, over algorithms crafted from a careful understanding of the problem.
I tried searching for it, but haven't found it yet.
The bittersweet success of general algorithms that just chew up data and compute time to arrive at very good answers, over algorithms crafted from a careful understanding of the problem.
I tried searching for it, but haven't found it yet.
This is something known as the “strong search vs weak search” debate. You can imagine generally that many problems can be transformed into a search of a multidimensional problem space for a solution.
Algorithms that are made to a search a specific domain are able to use knowledge of that domain to be more efficient at that particular problem. This is “strong search”. The downsides of this approach include the fact that they won’t be generally applicable outside the domain and also they rely on us understanding the domain well enough ahead of time to build a solution tailored to the problem.
“Weak search” algorithms use no specific knowledge of the domain. So these are things like stochastic methods, ML generally etc. These are weaker in the sense that they will always require more time and compute power than the best strong search in a given domain but are generally applicable and are able to reveal structure in the search space that they aren’t told about (perhaps because we don’t know this structure) ahead of time. So weak search can be used to find “good enough” answers to problems where we don’t yet have an exact/closed form type solution for instance.
This is the same as if you know the formula for calculating the area of a circle you will be able to calculate the area of a given circle much more rapidly than if you use (say) a Monte Carlo simulation to tell you what it is. But the circle method only does circles wheres the exact same MC will be able to tell you the area of any shape.
As compute generally becomes cheaper, weak search methods become viable for more and more problems given often people’s time requirements are not that critical in many applications.
Algorithms that are made to a search a specific domain are able to use knowledge of that domain to be more efficient at that particular problem. This is “strong search”. The downsides of this approach include the fact that they won’t be generally applicable outside the domain and also they rely on us understanding the domain well enough ahead of time to build a solution tailored to the problem.
“Weak search” algorithms use no specific knowledge of the domain. So these are things like stochastic methods, ML generally etc. These are weaker in the sense that they will always require more time and compute power than the best strong search in a given domain but are generally applicable and are able to reveal structure in the search space that they aren’t told about (perhaps because we don’t know this structure) ahead of time. So weak search can be used to find “good enough” answers to problems where we don’t yet have an exact/closed form type solution for instance.
This is the same as if you know the formula for calculating the area of a circle you will be able to calculate the area of a given circle much more rapidly than if you use (say) a Monte Carlo simulation to tell you what it is. But the circle method only does circles wheres the exact same MC will be able to tell you the area of any shape.
As compute generally becomes cheaper, weak search methods become viable for more and more problems given often people’s time requirements are not that critical in many applications.
> understanding based on principles is no longer required
It very much still is if you care about getting the right answer. If you only care about getting convincing sounding bullshit you just ask the "AI".
It very much still is if you care about getting the right answer. If you only care about getting convincing sounding bullshit you just ask the "AI".
Language models certainly generate their fair share of bullshit. But then, they are actually trained on bullshit - raw human output, much of it informal - so as a first step, it is a sign of great success.
By the time language models don't ever generate bullshit they will have succeeded in understanding and communicating at a level most or all of us will never achieve.
But most models, including photoshopping detection, are not language models. Their job is to perform some well defined problem (classification, pattern recognition, relationship generalization, etc.) better than alternative methods. They perform, or they don't. There isn't room for bullshit.
By the time language models don't ever generate bullshit they will have succeeded in understanding and communicating at a level most or all of us will never achieve.
But most models, including photoshopping detection, are not language models. Their job is to perform some well defined problem (classification, pattern recognition, relationship generalization, etc.) better than alternative methods. They perform, or they don't. There isn't room for bullshit.
https://arxiv.org/pdf/1906.05856.pdf
It's done using an ML model.
The thing is, I don't understand what signal it's using to determine whether the Liquify tool in Photoshop has been used. And from skimming the paper, I'm not sure the authors know either. But it's not detecting seams or compression artifacts or other common signals, because Liquify doesn't generate those.
Only thing I can guess is that it's picking up on anatomically impossible or extremely unlikely facial structure, which our own brains don't pick up on?
My only other guess would be some kind of subtle mismatch between lighting and skin texture/angle but I don't think the resolution is high enough, plus makeup and smoothing in Photoshop makes that even harder.