Ctrl Shift Face: A growing ‘deepfakes’ YouTube channel(theregister.co.uk)
theregister.co.uk
Ctrl Shift Face: A growing ‘deepfakes’ YouTube channel
https://www.theregister.co.uk/2019/05/28/youtube_deepfakes_channel/
127 comments
I would imagine that in the future video encoders will start to automatically include a cryptographic signature every X frames, which then players will become aware of and show an indicator for "original / modified" content.
It'd be too easy to fake, just replace the sensor in an approved camera with your own data feed.
And that's assuming it wouldn't be somewhat trivial to dump keys from one or more cameras on the market to create a an easy tool or resignvideo.com... which is a massive assumption.
As the movie industry has already seen, cryptography doesn't help you when you push out the keys to everyone. It'd be about as effective as DRM is at stopping video piracy.
And that's assuming it wouldn't be somewhat trivial to dump keys from one or more cameras on the market to create a an easy tool or resignvideo.com... which is a massive assumption.
As the movie industry has already seen, cryptography doesn't help you when you push out the keys to everyone. It'd be about as effective as DRM is at stopping video piracy.
You're not thinking this through.
The problem isn't that creators can make fake videos (which would be analogous to DRM). Let them. There is no harm in it. The problem is that users have no way of verifying true videos. This can be solved with cryptography, because both parties involved (creators and viewers) would want this to work.
In other words, DRM is about forcing everyone to do a thing. Verification is about allowing willing participants to check a thing. Different scenarios.
Companies could make tamper-proof sensor modules that do the signing right where it matters. The important thing is to make sure they aren't astronomically priced and don't compromise privacy.
The problem isn't that creators can make fake videos (which would be analogous to DRM). Let them. There is no harm in it. The problem is that users have no way of verifying true videos. This can be solved with cryptography, because both parties involved (creators and viewers) would want this to work.
In other words, DRM is about forcing everyone to do a thing. Verification is about allowing willing participants to check a thing. Different scenarios.
Companies could make tamper-proof sensor modules that do the signing right where it matters. The important thing is to make sure they aren't astronomically priced and don't compromise privacy.
> You're not thinking this through.
Would you please edit swipes like that (and "you must live in a parallel universe, to put it politely") out of your comments here? They break the guidelines, lower the signal/noise ratio, and invite more of the same from others. Your comments would be just fine without those bits.
https://news.ycombinator.com/newsguidelines.html
Would you please edit swipes like that (and "you must live in a parallel universe, to put it politely") out of your comments here? They break the guidelines, lower the signal/noise ratio, and invite more of the same from others. Your comments would be just fine without those bits.
https://news.ycombinator.com/newsguidelines.html
[deleted]
I'm sorry, but you're not thinking this through. Being able to prove that a video file is the unaltered output of a particular camera is in no way a useful protection against fake videos. Proving that file A is genuine tells us nothing about whether file B is fake.
Let's say I want to create and circulate a fake sex tape featuring a senior politician. In a world where signed sensors are common but not mandatory, the absence of a signature says nothing at all about the integrity of my video - there are still millions of cameras in the world that produce unsigned files. In a world where they are mandatory, I can simply fake the video in software and then re-record it using a high-resolution display and a signed sensor. A digital video expert might be able to prove that such a re-recording process took place, but I highly doubt that they'd be able to say so with any degree of confidence if there's a bunch of sensor noise and a couple of rounds of low-bitrate transcoding.
Signed sensors won't do anything to undermine the credibility of fake videos, but they may have the opposite effect if they're hacked - I can "prove" that my deepfake is genuine, because I've extracted a signing key or injected a bitstream into a camera module.
https://www.forbes.com/sites/kalevleetaru/2018/09/09/why-dig...
Let's say I want to create and circulate a fake sex tape featuring a senior politician. In a world where signed sensors are common but not mandatory, the absence of a signature says nothing at all about the integrity of my video - there are still millions of cameras in the world that produce unsigned files. In a world where they are mandatory, I can simply fake the video in software and then re-record it using a high-resolution display and a signed sensor. A digital video expert might be able to prove that such a re-recording process took place, but I highly doubt that they'd be able to say so with any degree of confidence if there's a bunch of sensor noise and a couple of rounds of low-bitrate transcoding.
Signed sensors won't do anything to undermine the credibility of fake videos, but they may have the opposite effect if they're hacked - I can "prove" that my deepfake is genuine, because I've extracted a signing key or injected a bitstream into a camera module.
https://www.forbes.com/sites/kalevleetaru/2018/09/09/why-dig...
> I'm sorry, but you're not thinking this through.
Please don't respond to swipes with more swipes. Your comment would be just fine without that bit.
https://news.ycombinator.com/newsguidelines.html
Please don't respond to swipes with more swipes. Your comment would be just fine without that bit.
https://news.ycombinator.com/newsguidelines.html
You don't need to be able to prove the video is raw footage, you just need to be able to verify that a publisher you trust says they recorded it. An HMAC allows you to verify that a message (like a video file) was not modified after the author signed it.
Hypothetical: An anonymous source releases CCTV footage that appears to show a public figure urinating on a homeless person. Do you really believe that we'll all ignore that footage because the signature doesn't belong to a major news organisation?
Yes. We will.
Because the future us will be conditioned to do so by our cryptographically integrated devices. There'll be so much fake video out there that we'll get accustomed to seeing it on our devices with the big huge "fake" sign flashing on it.
(Or, again, even worse, it will not have the "real" sign flashing on it that the video from validated publishers and cameras will have.)
We can be conditioned to do this extremely quickly. Again, ActiveX is what made all the plugin security warnings start popping up everywhere. But after a while, plugins like flash and unity even fell to the reality of us having been conditioned to click "Do Not Run Plugin" whenever we saw that warning. Are we better off without flash and unity? Probably, but that's beside the point. The point is that masses can be conditioned to implement their own control when you set up a system where only data meeting a certain standard is accepted. And it's a hardware standard. Consider very carefully the position over us that such a system would put hardware makers and those with power over hardware makers in.
Now again, I'm not arguing that fakes are not a problem. They are indeed a problem. Just saying we should go about solving the problem with extreme care and caution.
Because the future us will be conditioned to do so by our cryptographically integrated devices. There'll be so much fake video out there that we'll get accustomed to seeing it on our devices with the big huge "fake" sign flashing on it.
(Or, again, even worse, it will not have the "real" sign flashing on it that the video from validated publishers and cameras will have.)
We can be conditioned to do this extremely quickly. Again, ActiveX is what made all the plugin security warnings start popping up everywhere. But after a while, plugins like flash and unity even fell to the reality of us having been conditioned to click "Do Not Run Plugin" whenever we saw that warning. Are we better off without flash and unity? Probably, but that's beside the point. The point is that masses can be conditioned to implement their own control when you set up a system where only data meeting a certain standard is accepted. And it's a hardware standard. Consider very carefully the position over us that such a system would put hardware makers and those with power over hardware makers in.
Now again, I'm not arguing that fakes are not a problem. They are indeed a problem. Just saying we should go about solving the problem with extreme care and caution.
So you're arguing that at some point in the future, nobody will give even the slightest credence to amateur footage captured on camera phones, small business CCTV systems and a million other sorts of non-corporately-owned camera? Because I think that future is worse than the one that's full of fake video.
>Because I think that future is worse than the one that's full of fake video...
That's my point.
Going down the hardware cryptography path certainly does solve the problem. I'm not here to try and claim that it doesn't. The issue is, 10 or 20 years down the road, how have humans been conditioned because of the fact that only certain cameras can be validated to be real?
If cryptography takes over, 20 years from now, no one is going to convict any guy of rape because they see him on a non-signed camera video, or a non-signed CCTV feed. (And don't get me wrong, they shouldn't convict the guy if that's the so called evidence the prosecutor has.)
What I'm asking is that everyone take a deep breath, slow down for a minute, and just think about the implications of that.
And I was being generous in the scenario I laid out here. In reality, I don't believe it would take 20 years to condition us to dismiss non-signed video out of hand.
That's my point.
Going down the hardware cryptography path certainly does solve the problem. I'm not here to try and claim that it doesn't. The issue is, 10 or 20 years down the road, how have humans been conditioned because of the fact that only certain cameras can be validated to be real?
If cryptography takes over, 20 years from now, no one is going to convict any guy of rape because they see him on a non-signed camera video, or a non-signed CCTV feed. (And don't get me wrong, they shouldn't convict the guy if that's the so called evidence the prosecutor has.)
What I'm asking is that everyone take a deep breath, slow down for a minute, and just think about the implications of that.
And I was being generous in the scenario I laid out here. In reality, I don't believe it would take 20 years to condition us to dismiss non-signed video out of hand.
> So you're arguing that at some point in the future, nobody will give even the slightest credence to amateur footage captured on camera phones
I think that's the end goal of all the "deep fake" hysteria we're being primed with. In the future if some damning video of a politician or other elite member of society pops up the defense will simply be "well you can't rule out that it's a fake, look, it's not signed and it's not in our database of trusted videos!"
I think that's the end goal of all the "deep fake" hysteria we're being primed with. In the future if some damning video of a politician or other elite member of society pops up the defense will simply be "well you can't rule out that it's a fake, look, it's not signed and it's not in our database of trusted videos!"
Or the end goal of the "fake news" scare in general (not to say that there aren't sites that specialize in making up news.) To combat "fake news," you start whitelisting "true news," and defining it by the influence of the provider, not the content. In the end, you just end up with state news sources.
related: It's telling that "factchecking" isn't a dialogue between various authorities, but instead just a bunch of weird private judicial systems with no appeals courts, selected and assigned to specific stories by internet aggregation intermediaries. Eventually, this can't help but shake out as stories being divided into two categories: factual and banned from distribution.
I sadly feel that we're about 10 years away from licensing journalists with a security clearance-type process that must be sponsored by an approved outlet. The PropOrNot at the Post shook me a bit.
related: It's telling that "factchecking" isn't a dialogue between various authorities, but instead just a bunch of weird private judicial systems with no appeals courts, selected and assigned to specific stories by internet aggregation intermediaries. Eventually, this can't help but shake out as stories being divided into two categories: factual and banned from distribution.
I sadly feel that we're about 10 years away from licensing journalists with a security clearance-type process that must be sponsored by an approved outlet. The PropOrNot at the Post shook me a bit.
1st amendment won't allow that, but then again people seem to want to throw out the 2nd without a constitutional convention so it's not impossible. I can see the arguments now, "it's the freedom of the press, are you the press? Then why do you have a own a fully automatic printing press?"
We probably won't ignore that.
So, the "signature solution" doesn't solve every problem, but I would still argue that it solves enough problems, that it would be worth spending resources on trying to implement it.
I was under the impression that the whole point of such a system was you could trust any video from any (non-cracked) camera.
If we're at the point of trusting videos only from select providers, well, that's not really much of an improvement over just finding the video on that provider's website over HTTPS.
If we're at the point of trusting videos only from select providers, well, that's not really much of an improvement over just finding the video on that provider's website over HTTPS.
I absolutely agree with this point but I think it's terrifying - rather than a reassurance. In modern society our "truth" is becoming partitioned into tribal groups with topics as extreme as the curvature of the earth and efficacy of vaccines being suddenly up for debate. If someone like Alex Jones were to publish a video where the Clintons discuss making the frogs gay then his audience - a decent portion of which trust him as a journalistic source - would be able to confirm that this highly edited video has been verified true by Alex Jones.
Audio and video were sort of aberrant in fact that they could still generally be accepted at face value (as compared to quotes and attributions) and we're very quickly losing that stability.
Audio and video were sort of aberrant in fact that they could still generally be accepted at face value (as compared to quotes and attributions) and we're very quickly losing that stability.
Publishing has always been based on trust. Trusting publishers with a track record of openness and journalists with a track record of integrity is the best we have. We just have more options for cross referencing information now.
I believe what HN User gambler was getting at is a world far more insidious. One where no one cares whether or not file B is fake. No one cares whether transcoded file C is fake. He's talking about a world where the power elites have effectively set themselves up as the only source of cryptographically secure and verified image feeds.
ie - A world where so much fake image data has flooded into the zeitgeist that people only believe the feeds that are announced to be "real" by their cryptographically secure devices. They can play the fake video, even play video that may not be fake, but was not cryptographically validated. Their devices will just be flashing a huge warning sign right next to each such image feed. (Or even worse, it will only flash a "real" sign next to the image feeds it and the central have concerted to cryptographically validate.) It'd be kind of like the huge warning dialogs that always popped up at a certain point when people went somewhere with an ActiveX plugin in the page. After a while, it wasn't just ActiveX, it was everything. People just got conditioned to believing that every plugin, no matter the source, was horribly bad for them. (Every plugin probably was, but that's a separate issue.)
Only this time, the powers that be will be the only ones capable of delivering data that is guaranteed to be pure. Everything else will be either, not validated by their cryptography standard. Transcoded, (and consequently the masses will dismiss it out of hand). Or out and out fake, (which, yes, the masses should dismiss out of hand. I get that).
But I'm just saying that people should think a little further ahead here when they are trying to solve this problem. If we want to use cryptography to solve this problem, that's perfectly reasonable. (And yes, it would work. It would work only too well, and that's the entire problem.) I'm only saying that we should do what we can to make sure that cryptographically validated image feeds are not uniquely identifiable, for instance. Because when you go down this road you need to realize that with hardware cryptography, the hardware makers and those with power over them become de facto arbiters. And they can potentially use distribution to control the masses. Because they have the only pure sources. They could potentially even use attribution to control the masses. (You sure you want to video tape that police beating and mail it to the media "anonymously" with one of these devices?)
I'm just saying. Stop and think it through a bit.
And having a plan based on, "Oh... We're smart enough to get around it!" Is a very foolish plan for two reasons. First, because we won't be able to get around it, and second because it's the same as having no plan at all.
ie - A world where so much fake image data has flooded into the zeitgeist that people only believe the feeds that are announced to be "real" by their cryptographically secure devices. They can play the fake video, even play video that may not be fake, but was not cryptographically validated. Their devices will just be flashing a huge warning sign right next to each such image feed. (Or even worse, it will only flash a "real" sign next to the image feeds it and the central have concerted to cryptographically validate.) It'd be kind of like the huge warning dialogs that always popped up at a certain point when people went somewhere with an ActiveX plugin in the page. After a while, it wasn't just ActiveX, it was everything. People just got conditioned to believing that every plugin, no matter the source, was horribly bad for them. (Every plugin probably was, but that's a separate issue.)
Only this time, the powers that be will be the only ones capable of delivering data that is guaranteed to be pure. Everything else will be either, not validated by their cryptography standard. Transcoded, (and consequently the masses will dismiss it out of hand). Or out and out fake, (which, yes, the masses should dismiss out of hand. I get that).
But I'm just saying that people should think a little further ahead here when they are trying to solve this problem. If we want to use cryptography to solve this problem, that's perfectly reasonable. (And yes, it would work. It would work only too well, and that's the entire problem.) I'm only saying that we should do what we can to make sure that cryptographically validated image feeds are not uniquely identifiable, for instance. Because when you go down this road you need to realize that with hardware cryptography, the hardware makers and those with power over them become de facto arbiters. And they can potentially use distribution to control the masses. Because they have the only pure sources. They could potentially even use attribution to control the masses. (You sure you want to video tape that police beating and mail it to the media "anonymously" with one of these devices?)
I'm just saying. Stop and think it through a bit.
And having a plan based on, "Oh... We're smart enough to get around it!" Is a very foolish plan for two reasons. First, because we won't be able to get around it, and second because it's the same as having no plan at all.
What in the current political climate makes you believe that, in the future, we'll trust big corporations more than we trust whichever in-group we identify with?
Apparently, we're talking about different problems. I am talking about the problem of content creators being able to demonstrate that what they post is real footage. You're talking about the problem of someone being fooled by some arbitrary video online. If you think that the latter is somehow a fundamentally new threat created by AI, you must live in a parallel universe, to put it politely.
In what world is content creators needing to prove the footage they took is "real" a problem? Considering most content creators also do post editing I don't think that has any relationship at all to what's being discussed. We're talking about the kind of video that triggers a scandal, be it a police shooting, politician caught committing a crime, sex tape, etc. This is not the kind of thing being captured by "content creators". What's the purpose at all of what you're proposing?
I will try explaining this for the last time.
We already live in the word where a video is not a 100% proof that something happened. Given enough time and money you can use video editing, actors and CGI do show almost anything. This does not, however, completely invalidate all videos as evidence, because the cost of faking something is fairly high.
Deep fakes can potentially decrease the cost of impersonation of someone in any video to almost nothing.
To mitigate that we can introduce additional technologies that will bring the cost of video impersonation back up, effectively restoring the current status quo.
Like most security, it's all about the costs involved in attacks and mitigations. I remember the time where most of HN visitors understood this very well.
We already live in the word where a video is not a 100% proof that something happened. Given enough time and money you can use video editing, actors and CGI do show almost anything. This does not, however, completely invalidate all videos as evidence, because the cost of faking something is fairly high.
Deep fakes can potentially decrease the cost of impersonation of someone in any video to almost nothing.
To mitigate that we can introduce additional technologies that will bring the cost of video impersonation back up, effectively restoring the current status quo.
Like most security, it's all about the costs involved in attacks and mitigations. I remember the time where most of HN visitors understood this very well.
That has nothing to do with the comment you responded to:
Can you explain what you mean as the distinction between "content creators" and "some arbitrary video online" if the same technology is used to "authenticate" both?
> Like most security, it's all about the costs involved in attacks and mitigations.
Ignoring the fact that this could likely be done fairly cheaply, unlike other security improvements this actually weakens the next layer down substantially, faked content that does beat the system introduces a new seal of approval on that content. I don't want to see people get convicted because footage that's later found out to be faked has a green checkmark when the jury watches it.
Can you explain what you mean as the distinction between "content creators" and "some arbitrary video online" if the same technology is used to "authenticate" both?
> Like most security, it's all about the costs involved in attacks and mitigations.
Ignoring the fact that this could likely be done fairly cheaply, unlike other security improvements this actually weakens the next layer down substantially, faked content that does beat the system introduces a new seal of approval on that content. I don't want to see people get convicted because footage that's later found out to be faked has a green checkmark when the jury watches it.
Play deepfake on a TV, point signed camera at it.
I think the point of cryptographic signing wouldn't be verifying that it was recorded with a camera, but that it was recorded with a specific camera.
I wonder if future studio cameras will need water cooling lines to dissipate the crypto heat
Have you ever tried this? The quality will be awful.
As far as I'm concerned, the need to use a real camera to capture real light is sufficient to thwart creation of truly convincing fakes.
As far as I'm concerned, the need to use a real camera to capture real light is sufficient to thwart creation of truly convincing fakes.
>Have you ever tried this?
Yes. It used to be the industry standard method for recording TV programmes prior to the invention of videotape recorders. If you know what you're doing and use the right equipment, the generational quality loss is very minor.
https://en.wikipedia.org/wiki/Kinescope
Yes. It used to be the industry standard method for recording TV programmes prior to the invention of videotape recorders. If you know what you're doing and use the right equipment, the generational quality loss is very minor.
https://en.wikipedia.org/wiki/Kinescope
>It used to be the industry standard method for recording TV programmes prior to the invention of videotape recorders.
Back when everything was analog and low-resolution, had noise and artifacts anyway, sure. Do you have a video that compares modern digital 1080p with its best "Kinescoped" version? You have to deal with parallax, mismatched frame frequencies, pixels and differences between natural light and whatever spectrum the source monitor produces. I have never seen a recording of a screen that looked anywhere close in quality to the original in 1080p.
Back when everything was analog and low-resolution, had noise and artifacts anyway, sure. Do you have a video that compares modern digital 1080p with its best "Kinescoped" version? You have to deal with parallax, mismatched frame frequencies, pixels and differences between natural light and whatever spectrum the source monitor produces. I have never seen a recording of a screen that looked anywhere close in quality to the original in 1080p.
We're talking about faked videos here though, often of things where the quality is already quite questionable. If it shows something that looks 480p good enough and gets the green checkmark it's game over for this concept of video verification. No one needs perfect 1080p or 4k video to believe politician or celebrity is sleeping around.
If it gets it to the point where it takes years of experience with this to figure out what is and isn't fake... I doubt that's any better than the situation without such a video verification system at all. In fact, I'd argue it's worse due to the fake seal of approval being given to the video.
If it gets it to the point where it takes years of experience with this to figure out what is and isn't fake... I doubt that's any better than the situation without such a video verification system at all. In fact, I'd argue it's worse due to the fake seal of approval being given to the video.
> Do you have a video that compares modern digital 1080p with its best "Kinescoped" version?
As an informal comparison, check out a downloaded "cam version" of any recent movie compared with a DVD rip. If you ignore the occasional noise or interference from the crowd, the quality can be quite good.
As an informal comparison, check out a downloaded "cam version" of any recent movie compared with a DVD rip. If you ignore the occasional noise or interference from the crowd, the quality can be quite good.
You can have the camera record metadata as well, like focus settings and super speed. That will make it harder.
And you can build a camera that lies about this.
True, but today you also can hire a lookalike to some target person and record footage of them doing something terrible, or edit footage carefully enough that the edit cannot be detected.
Deepfake makes it less expensive to create fake video of someone, special signing cameras could make it expensive again in the face of that.
Deepfake makes it less expensive to create fake video of someone, special signing cameras could make it expensive again in the face of that.
Not quite as you can't necessarily clone the keys from an existing camera without some difficulty, however you can say, replace those sensors with your own inputs fed using a cheap microcontroller you wrote the firmware for. There's a time and effort investment but it's not exactly high...
There's a video of a politician taking a bribe.
How do you verify it's real? You can't - because the camera of the guy that published it says it's not modified, but it could be just as well a camera recording a display with the fake playing.
How do you verify it's real? You can't - because the camera of the guy that published it says it's not modified, but it could be just as well a camera recording a display with the fake playing.
Cameras recording displays also leave artifacts in the resulting stream. There would at the very least likely be a moire pattern somewhere, at some level, and somehow the capturing camera would need to resolve that. That resolution will leave a fingerprint.
“I work at a hotel and $politician came in with his mistress. Here’s a video I taped off of the security monitor.”
That's actually an excellent point, even if such technology worked perfectly it wouldn't solve simple problems like that.
Seeing footage literally taped by a phone off a NVR display is pretty common...
Seeing footage literally taped by a phone off a NVR display is pretty common...
Yeah, but it also is usually pretty obvious about what it is. That's hard to cover up.
It's actually not too bad - you have to essentially line up the pixels on the camera's sensor with the pixels on the display, it's a challenge and might involve some careful choices of camera, display, focal length, distance, etc, but you only have to get it right once. This gets even easier with a projector where pixels can be blended together, that's why cam'd movies can be surprisingly watchable, despite being filmed in far less than optimal conditions, they don't show the sort of artifacts you're talking about. While poor contrast can certainly be an issue in such situations, think about how poor that setup is compared to what would be possible with a purpose built version.
[deleted]
> Companies could make tamper-proof sensor modules that do the signing right where it matters.
Just film a screen displaying the fake video. As soon as it's possible for a bad actor to create signed fake content it invalidates the entire protocol.
Oh, ok, I see what you mean now. The viewer already trusts the creators, so the threat model is not about an attacker creating fake videos. But if you already trust the creator why do you need further proof? Identity is already solved. The problem at stake here is videos released to the general public where the source is unknown, or not trusted by all stakeholders, i.e, during a trial.
Just film a screen displaying the fake video. As soon as it's possible for a bad actor to create signed fake content it invalidates the entire protocol.
Oh, ok, I see what you mean now. The viewer already trusts the creators, so the threat model is not about an attacker creating fake videos. But if you already trust the creator why do you need further proof? Identity is already solved. The problem at stake here is videos released to the general public where the source is unknown, or not trusted by all stakeholders, i.e, during a trial.
To play devil's advocate, video evidence is already not very strong. Can't it already be faked enough without this technology such that we should never trust it as irrefutable proof of anything
Invisible watermarking can be encoded into video frames.
https://inis.iaea.org/collection/NCLCollectionStore/_Public/...
I remember visiting the MIT media lab when I was in college, they described the audio version of "invisible watermarking" - manipulating the source audio with a generated acoustic effect, similar to a doppler shift. The effect is not detected by the human ear, but signal analysis can translate it into binary metadata, such as a DRM signature.
https://inis.iaea.org/collection/NCLCollectionStore/_Public/...
I remember visiting the MIT media lab when I was in college, they described the audio version of "invisible watermarking" - manipulating the source audio with a generated acoustic effect, similar to a doppler shift. The effect is not detected by the human ear, but signal analysis can translate it into binary metadata, such as a DRM signature.
What's the relevant application here? If you're suggesting this could be added by video editing software or display hardware, it seems if everyone knew about such watermarking it'd be trivial to strip out and turn into noise, so it would work publicly exactly once.
If you're saying it'd be added by legitimate camera sensors, well it wouldn't beat the possibilities I suggested already of simply feeding tampered data to the sensor.
If you're saying it'd be added by legitimate camera sensors, well it wouldn't beat the possibilities I suggested already of simply feeding tampered data to the sensor.
As it was described to me, the watermarking is encoded in such a way that it would be prohibitively expensive computationally to offset or overwrite it.
Okay, let's assume that's remotely possible. Injecting such a watermark at the camera doesn't help you - in fact, such a watermark you're describing is attempting to be resistant to things like re-transcoding so it might actually be carried through an editing process successfully. Not to mention the previous camera sensor bypasses. So let's ignore camera based options.
You have to get all video editing software or displays to inject such a watermark while keeping their keys private. Getting direct LVDS access to a panel and being able to write your own firmware pretty much bypasses most display options, and well... software based options are even worse I'd say, software is open to such a wide array of attacks and the mere existence of open source video editing tools, including FaceSwap itself makes this seem... questionable at best.
You have to get all video editing software or displays to inject such a watermark while keeping their keys private. Getting direct LVDS access to a panel and being able to write your own firmware pretty much bypasses most display options, and well... software based options are even worse I'd say, software is open to such a wide array of attacks and the mere existence of open source video editing tools, including FaceSwap itself makes this seem... questionable at best.
You could go on the level of Apple - hardware security chip with presigned bootloader + a chain of trust to boot the OS, and the private key stored in a secure enclave. Then generate a cert for every second of video or so and embed it into the media format. Don’t sign the video if you detect any foreign hardware.
You could try but it'd still be possible to just fake it at the sensor level, essentially so the IC would see your video as the analog signal coming in and encode it as if it were any other image. It's basically the "analog hole" of the video world.
If you wanted to do the cheap version of this you could probably set up a screen with proper lighting such that it'd line up well enough that screen and sensor pixels matched closely making it near if not completely impossible to tell a screen was involved.
Worse, such a signature on a fake video may give people false confidence in it being real, when really it still needs scrutiny especially if a possible state level or dedicated attacker is involved.
If you wanted to do the cheap version of this you could probably set up a screen with proper lighting such that it'd line up well enough that screen and sensor pixels matched closely making it near if not completely impossible to tell a screen was involved.
Worse, such a signature on a fake video may give people false confidence in it being real, when really it still needs scrutiny especially if a possible state level or dedicated attacker is involved.
Nikon’s been offering chip-based video crypto for years, it’s just not popular in consumer-grade hardware yet. If it was, we’d all be up in arms about our devices having a unique fingerprint in every photo (or video) that could be used to track us.
>If you wanted to do the cheap version of this you could probably set up a screen with proper lighting such that it'd line up well enough that screen and sensor pixels matched closely making it near if not completely impossible to tell a screen was involved.
You could make things more difficult by using multiple cameras (and perhaps some other sensors) to detect a 3D scene rather than just a 2D representation, but I agree that it's all ultimately fakeable.
You could make things more difficult by using multiple cameras (and perhaps some other sensors) to detect a 3D scene rather than just a 2D representation, but I agree that it's all ultimately fakeable.
You could have onboard acceleration and depth sensors on the camera to make the fake costlier to produce.
As a random user, this is what I would like:
When I watch a video-clip of, let's say a press-conference, it would be nice to have a way to verify that the video-clip is indeed the one recorded by whatever journalist was at the press-conference.
It's not about camera hardware, it's about making it easy to check if the video someone is presenting to me is what they claim it is.
When I watch a video-clip of, let's say a press-conference, it would be nice to have a way to verify that the video-clip is indeed the one recorded by whatever journalist was at the press-conference.
It's not about camera hardware, it's about making it easy to check if the video someone is presenting to me is what they claim it is.
Better would be for people/organizations to be in the habit of signing things they publish.
This is fundamentally just another MITM attack.
This is fundamentally just another MITM attack.
That works great if I'm looking for an official video from the Wall Street Journal, but not if I'm watching Fred's Blog Channel. Also, what if I want to edit something? What if I want to tweak the color profile? Add an effect? And as pointed out by another comment, you'd get a modified camera and feed it into the sensor, or modify the firmware. And some guy in China will do a clean-room RE and make little video signers.
The bottom line is it's a cool idea, but I don't think it will work, and I don't think it will stop fake anything. Also, most of us complain about code signing when it affects us; why would this be different? Fred the Blogger can't get an "official video signing certificate", which is probably what we would end up with. Or better yet, get the government involved and create a Bureau of Trustworthy Video Sources!
The bottom line is it's a cool idea, but I don't think it will work, and I don't think it will stop fake anything. Also, most of us complain about code signing when it affects us; why would this be different? Fred the Blogger can't get an "official video signing certificate", which is probably what we would end up with. Or better yet, get the government involved and create a Bureau of Trustworthy Video Sources!
That works great if I'm looking for an official video from the Wall Street Journal, but not if I'm watching Fred's Blog Channel. Also, what if I want to edit something? What if I want to tweak the color profile? Add an effect? Oh, so now adobe software can re-sign something, but not the open-source option?
And as pointed out by another comment, you'd get a modified camera and feed it into the sensor, or modify the firmware. And some guy in China will do a clean-room RE and make little video signers.
The bottom line is it's a cool idea, but I don't think it will work, and I don't think it will stop fake anything.
And as pointed out by another comment, you'd get a modified camera and feed it into the sensor, or modify the firmware. And some guy in China will do a clean-room RE and make little video signers.
The bottom line is it's a cool idea, but I don't think it will work, and I don't think it will stop fake anything.
I'm thinking that we'd need an Originality Authority (like a CA), to act as the anchor. I assume anyone could just strip the cryptographic signature and then re-sign it with their own signature, and we'd need an authority to prevent that.
Upload your content, request a key you can sign with, the OA checks against previously registered content in it's database for originality (I can only assume this would be as buggy as Youtube's algorithm is today).
Upload your content, request a key you can sign with, the OA checks against previously registered content in it's database for originality (I can only assume this would be as buggy as Youtube's algorithm is today).
If everyday people are allowed to possess hardware that can sign images, it would only require someone feeding the fake images into the sensor that would be recording. Then, they would have a verified yet still fake image, which if anything would lead to more people falling for it. You could require that information about the recording also be included (gps, timestamp, etc.) but not only does that harm privacy, it means that when someone fakes it then people will trust the fake even more.
I wonder if it'd be possible to "deepfake" the crypto sigs...?
It should be entirely unfeasible if its good crypto
In order to deepfake it, a neural net would have to understand the crypto algo in such a way it could modify its deepfake input to collide with the original input's hash. Basically it would need to reverse the crypto to generate all frames that evaluate to a certain hash, and then find one that is mostly similar to the intended input.
I don't think it will be realistic at all, if even theoretically possible.
In order to deepfake it, a neural net would have to understand the crypto algo in such a way it could modify its deepfake input to collide with the original input's hash. Basically it would need to reverse the crypto to generate all frames that evaluate to a certain hash, and then find one that is mostly similar to the intended input.
I don't think it will be realistic at all, if even theoretically possible.
Yep, this might make Intel-style rootkit+DRM legally mandatory... welcome to 1984 ! :(
We've been through this whole argument before when HDCP was first proposed. I'd imagine this following exactly the same path.
Well, the stakes are much higher now. See how Facebook got roped into having to do moderation !
See how it's much harder for anyone else to replace Facebook now, because they need more staff to start out with?
I don't follow ?
I believe it's a reference to a general effect of regulation, which is by making the business being regulated more expensive due to following regulations, it prevents smaller businesses from popping up to potentially displace the larger business.
For a generally positive example of this, see a good chunk of the history of Silicon Valley with smaller tech companies displacing larger companies like IBM, because there was no regulation stopping it. For an example generally less loved around here, consider Uber making a good pass at eating the entire Taxi industry by virtue of dodging regulations.
For a generally positive example of this, see a good chunk of the history of Silicon Valley with smaller tech companies displacing larger companies like IBM, because there was no regulation stopping it. For an example generally less loved around here, consider Uber making a good pass at eating the entire Taxi industry by virtue of dodging regulations.
And sooner or later, Uber will start proposing regulations for that same reason: to avoid competition.
[deleted]
So then.....how do we start hobbyist chip-fab@home?
That wouldn't stop from someone filming a fake video.
It wouldn't, but it is a pretty big hurdle to stage a whole thing for an actual camera.
What you describe has always been possible, but it seems rare.
What you describe has always been possible, but it seems rare.
A blockchain could be used to solve this problem.
Someone has to put a deepfake Zuckerberg video up on Facebook where he forecasts a terrible financial quarter.
Maybe that would have some influence about how seriously they take takedown requests.
Maybe that would have some influence about how seriously they take takedown requests.
To any one who reads this, don't actually do that. I wouldn't put it past Facebook to file a serious lawsuit against you. If it spread far enough, you could cause serious damage, and the SEC might even get involved.
If I were going to do this, and I thought it would cause such a storm, I'd take a raspi wardriving and find an open router somewhere, and then upload it via TOR to liveleak or imgur or something like that using an old netbook sourced from Craigslist. Then wipe the devices used and sell 'em back on Craigslist, or drop 'em off at Goodwill or some recycling company. Plausibly deniable and all that.
Some one's been watching too much Mr. Robot... for what it's worth, don't bother with imgur over tor; they block uploads from those ips. You can, however, use the lunapic upload to imgur feature. Public library also might be an option, though I might reccomend i2p. Whether it's really "more secure", I dont know, but it doesn't have the same mainstream knowledge (or use by journalists), so three-letter guys would be slower to break it.
If they did chase such a lawsuit, it would expose them as having serious double standards as per the recent Pelosi news item: https://www.nytimes.com/2019/05/29/technology/facebook-pelos...
"Speaker Nancy Pelosi on Wednesday strongly rebuked Facebook, saying the company’s refusal to take down altered videos of her demonstrated how the social network contributed to misinformation"
"Speaker Nancy Pelosi on Wednesday strongly rebuked Facebook, saying the company’s refusal to take down altered videos of her demonstrated how the social network contributed to misinformation"
Could we get the link changed to the actual YouTube channel? https://www.youtube.com/channel/UCKpH0CKltc73e4wh0_pgL3g
How hard is it for news sites to include links to other websites other than their own? Like thanks for at least calling out the name of the channel but maybe make the name a hyperlink?
Agreed that they should link to the channel explicitly, but if you click on one of those videos, they take you to the channel
Articles like these make me eager for advances made in the 'authenticity' market. I recently learned of a company called Truepic[0] that specializes in photo and video verification.
From the New Yorker article I read:
"Truepic, a startup in San Diego, aims at producing a new kind of photograph—a verifiable digital original. Photographs taken with its smartphone app are uploaded to its servers, where they enter a kind of cryptographic lockbox. “We make sure the image hasn’t been manipulated in transit,” Jeffrey McGregor, the company’s C.E.O., explained. “We look at geolocation data, at the nearby cell towers, at the barometric-pressure sensor on the phone, and verify that everything matches. We run the photo through a bunch of computer-vision tests.” If the image passes muster, it’s entered into the Bitcoin and Ethereum blockchain. From then on, it can be shared on a special Web page that verifies its authenticity. Today, Truepic’s biggest clients are insurance companies, which allow policyholders to take verified photographs of their flooded basements or broken windshields. The software has also been used by N.G.O.s to document human-rights violations, and by workers at a construction company in Kazakhstan, who take “verified selfies” as a means of clocking in and out. “Our goal is to expand into industries where there’s a ‘trust gap,’ ” McGregor said: property rentals, online dating. Eventually, he hopes to integrate his software into camera components, so that “verification can begin the moment photons enter the lens.” [1]
[0] https://truepic.com/
[1] https://www.newyorker.com/magazine/2018/11/12/in-the-age-of-...
From the New Yorker article I read:
"Truepic, a startup in San Diego, aims at producing a new kind of photograph—a verifiable digital original. Photographs taken with its smartphone app are uploaded to its servers, where they enter a kind of cryptographic lockbox. “We make sure the image hasn’t been manipulated in transit,” Jeffrey McGregor, the company’s C.E.O., explained. “We look at geolocation data, at the nearby cell towers, at the barometric-pressure sensor on the phone, and verify that everything matches. We run the photo through a bunch of computer-vision tests.” If the image passes muster, it’s entered into the Bitcoin and Ethereum blockchain. From then on, it can be shared on a special Web page that verifies its authenticity. Today, Truepic’s biggest clients are insurance companies, which allow policyholders to take verified photographs of their flooded basements or broken windshields. The software has also been used by N.G.O.s to document human-rights violations, and by workers at a construction company in Kazakhstan, who take “verified selfies” as a means of clocking in and out. “Our goal is to expand into industries where there’s a ‘trust gap,’ ” McGregor said: property rentals, online dating. Eventually, he hopes to integrate his software into camera components, so that “verification can begin the moment photons enter the lens.” [1]
[0] https://truepic.com/
[1] https://www.newyorker.com/magazine/2018/11/12/in-the-age-of-...
How does Truepic work? What's stopping me from spoofing sensor data on my client?
Probably nothing. But it solves 99% of the problems it's meant to solve. Casual insurance fraud, for example.
Preventing casual insurance fraud and guarding against deep fakes are very different problems, with a very stark contrast in terms of the effort and technical aptitude available.
Occasionally a security minded computer savy person might pull one over on the insurance company - but most clients aren't going to have the expertise or dedication to correctness. To contrast that - the pool of people who would try and produce a deep fake, especially in the political realm, have already decided to invest a good amount of investigation and effort and are likely rather technically minded.
So this would stop the script-kiddie equivalent actor in the deep fake realm, but I am more concerned about dedicated nefarious actors.
Occasionally a security minded computer savy person might pull one over on the insurance company - but most clients aren't going to have the expertise or dedication to correctness. To contrast that - the pool of people who would try and produce a deep fake, especially in the political realm, have already decided to invest a good amount of investigation and effort and are likely rather technically minded.
So this would stop the script-kiddie equivalent actor in the deep fake realm, but I am more concerned about dedicated nefarious actors.
The point is to have an immutable source to compare against. If you suspect a video is faked, you can find the verified original video that was used to make the fake and compare against it.
What if the fake video is the first one to be uploaded?
That’s where they use their fancy computer vision software to determine if the source has been tampered with.
And if they can’t determine that?
I think this is sort of where we realize that Oscar Wilde was actually the democratic candidate in the 2016 election - falsely attributed quotes have gotten out of hand and even relatively reputable sources have made this mistake... I suspect the same will be true of videos soon, your trust in the accuracy of the information will be solely tied to your trust in the organization - if you don't trust NYT you won't trust videos you see there and declare them "fake news", ditto for Fox and everyone else.
This is leading society to a very dangerous place where the truth is becoming a matter of opinion.
This is leading society to a very dangerous place where the truth is becoming a matter of opinion.
We've been there. It led to impartial professionals who, in person, report what they witnessed, and who made a career out of being truthful and exact.
The Bard is back again!
The Bard is back again!
Yes, it's an arms race. Surely this is obviously the case?
"What if the government can't determine if a 100 dollar bill is fake?"
Answer to both questions: those who have a vested interest in detecting fakes spend more money on fake-detection mechanisms.
"What if the government can't determine if a 100 dollar bill is fake?"
Answer to both questions: those who have a vested interest in detecting fakes spend more money on fake-detection mechanisms.
This is why I think a dolution will involve hardware signing, and maybe some integration with a.photoediting suite
I am so tempted to reply with a BS. Because this is a super hard problem, with hundreds of edge cases making it even harder, some tried to solve it in the past, with underwhelming results.
Yes, this is utter BS, sorry.
Yes, this is utter BS, sorry.
Declaring BS on what, TruePic as a business? The general problem is very hard, but the specific problem of taking photos with smartphones is tractable. Their approach makes sense, and in some cases can be made much easier with access to platform (e.g. android, ios) device verification. This is just the beginning of the broad adoption of identity verification services.
If you're gonna call BS, just do it!
I am curious to know more about this space because it's all very new to me. What failed solutions were tried in the past? Which issues does Truepic fail to address? And is it even possible to address these failures?
I am curious to know more about this space because it's all very new to me. What failed solutions were tried in the past? Which issues does Truepic fail to address? And is it even possible to address these failures?
Truepic doesn't solve the problem of you taking a picture of your neighbors flooded basement or broken windshield. GPS can't fix the problem as its not accurate enough, and for quite a few places is not odd if you park somewhat close to your neighbors house.
a few days ago, I was searching YouTube for a video on my TV and I found Bill Hader's impression of Arnold Schwarzenegger, which is impressive BTW, but I didn't realize what was going on with his face. At first I thought, wow, he's physiognomy is like Arnold, but something weird was going on. It took several trials until I realize that Arnold's face was superimposed on Bill's face. Spooky.
Same here. Jim Carrey's face seems to morph into John Wayne's and James Dean's when he does impressions of them. I thought the same was happening with Bill. I'm aware of deepfakes but yet it took several watches with me going "what the heck is going on here" to notice the almost seamless transitions.
I had a similar experience with that same video. It was suggested to me in the YT recommendations (I had watched Bill Hader on Late Show with Stephen Colbert previously).
I genuinely thought "wow, when he squints his eyes he really looks like Arnold". The transitions between Bill's own face and Arnold's were totally seemless to me.
I finally realized what was going on when I saw the channel's name, and it spooked me that I had fallen for it.
I genuinely thought "wow, when he squints his eyes he really looks like Arnold". The transitions between Bill's own face and Arnold's were totally seemless to me.
I finally realized what was going on when I saw the channel's name, and it spooked me that I had fallen for it.
> I genuinely thought "wow, when he squints his eyes he really looks like Arnold". The transitions between Bill's own face and Arnold's were totally seemless to me.
my experience, exactly
my experience, exactly
One thing I don't see talked about much is the Uncanny Valley. No other technology has made it past it and looking at these they definitely have not either. Is there any reason to think that "deepfakes" will leapfrog it somehow?
The problem is that a big percentage of people will not have the gut reaction that screams "there's something really unsettling about that person's face" when looking at a deepfake.
Just take a sample of your close friends or relatives. How many of them can tell 48fps video apart from 24fps? How many people leave Smooth Motion turned on on their TV's without even noticing it?
Hell, I remember people telling me that they could not tell the difference between Blu-ray and DVD when Blu-ray came out. Even in this forum, a lot of people have claimed that there's little difference between a 1080p monitor and a 4k monitor scaled at 200% (Retina). It's 2019. 1080p screens are completely unsuitable for looking at text, and yet many people can barely notice the difference.
That's what we have to worry about. The tech is good enough to weaponize right now. And I see no way out of it, except with education. Perhaps high schools should eventually include a visual and aural literacy course as part of the curriculum.
Just take a sample of your close friends or relatives. How many of them can tell 48fps video apart from 24fps? How many people leave Smooth Motion turned on on their TV's without even noticing it?
Hell, I remember people telling me that they could not tell the difference between Blu-ray and DVD when Blu-ray came out. Even in this forum, a lot of people have claimed that there's little difference between a 1080p monitor and a 4k monitor scaled at 200% (Retina). It's 2019. 1080p screens are completely unsuitable for looking at text, and yet many people can barely notice the difference.
That's what we have to worry about. The tech is good enough to weaponize right now. And I see no way out of it, except with education. Perhaps high schools should eventually include a visual and aural literacy course as part of the curriculum.
Yes, these may be still in it, but they're climbing out. They're not leapfrogging it, they're just getting slowly better, inching further until they get past the valley.
I can't say that though... had these videos been sent to me and it was mentioned "Arnold lost weight" or "Brad Pitt reshot the scene after shedding some pounds" I'd have probably been less of a sceptic and in a quick pass this "fake news" could have got a pass from me or others. It's an interesting time and if you're doing cursory browsing of news bites for the day this may fly right past...
The Fight Club one I couldn't tell was I looking at Edward Norton or Brad Pitt the whole time. Definitely in the valley for me but I guess I was looking for it.
Generally the best way I've found to detect deep fake video is to listen to the audio.
When we have deepfake video _and_ deepfake audio we'll be in trouble. At least for moving images.
When we have deepfake video _and_ deepfake audio we'll be in trouble. At least for moving images.
https://arxiv.org/pdf/1802.06006.pdf
These authors worked/work at baidu.
https://www.vice.com/en_us/article/3k7mgn/baidu-deep-voice-s...
These authors worked/work at baidu.
https://www.vice.com/en_us/article/3k7mgn/baidu-deep-voice-s...
This is multi-layered problem with no simple answers but many comments here touch on solutions, note there will not be 1 solution. Here are a couple of concepts I know are being worked on
- Secure Data at the Source -
Public Created - I have seen some demo's of phone apps that put video and pictures from phone directly on the blockchain with gps, time stamps etc.. Also have heard of dongles being added to phones
Professionally Created at Device - have read about hardware getting processors to secure data directly on the blockchain at the device.
Created at Editing Software - This could be connected to the identity of the original on the blockchain and then the edited version has notes with it and added to the blockchain as an edited version of the first.
Security cameras for Customs and Border Patrol are getting blockchain capabilities added per this quote - " Factom (blockchain) has integrated their technology into two brands of cameras used by CBP at border locations to ensure data collected from these cameras is tamper proof" - http://www.activecyber.net/dhs-st-lays-out-a-broad-yet-innov...
IoT Devices - This micro processor can be added to an IoT (raspberry pi and the like) device and secure data the source - https://iot-sas.tech/
Professionals (media) and citizens can then have proof of what and when they created the video and potential a certification can be added in manor and tool for others to check the video is in fact the original.
-- Identify Fakes -- Two articles from universities working on it https://jsis.washington.edu/news/deep-fakes-fake-news-and-wh...
https://medium.com/jsk-class-of-2019/six-lessons-from-my-dee...
None of the above completely solves the issue but with implementing these and having standards the public can start to trust what they are watching is authentic.
- Secure Data at the Source -
Public Created - I have seen some demo's of phone apps that put video and pictures from phone directly on the blockchain with gps, time stamps etc.. Also have heard of dongles being added to phones
Professionally Created at Device - have read about hardware getting processors to secure data directly on the blockchain at the device.
Created at Editing Software - This could be connected to the identity of the original on the blockchain and then the edited version has notes with it and added to the blockchain as an edited version of the first.
Security cameras for Customs and Border Patrol are getting blockchain capabilities added per this quote - " Factom (blockchain) has integrated their technology into two brands of cameras used by CBP at border locations to ensure data collected from these cameras is tamper proof" - http://www.activecyber.net/dhs-st-lays-out-a-broad-yet-innov...
IoT Devices - This micro processor can be added to an IoT (raspberry pi and the like) device and secure data the source - https://iot-sas.tech/
Professionals (media) and citizens can then have proof of what and when they created the video and potential a certification can be added in manor and tool for others to check the video is in fact the original.
-- Identify Fakes -- Two articles from universities working on it https://jsis.washington.edu/news/deep-fakes-fake-news-and-wh...
https://medium.com/jsk-class-of-2019/six-lessons-from-my-dee...
None of the above completely solves the issue but with implementing these and having standards the public can start to trust what they are watching is authentic.
The scare over this is overblown.
1. If the image quality is good and if you pause the video, the artifacts are pretty obvious.
2. Making meme videos is not a problem. You have to think about serious scenarios and ask yourself whether they were impossible before. In a lot of cases they were possible before without using AI. People edited interviews and staged recorded events since forever.
1. If the image quality is good and if you pause the video, the artifacts are pretty obvious.
2. Making meme videos is not a problem. You have to think about serious scenarios and ask yourself whether they were impossible before. In a lot of cases they were possible before without using AI. People edited interviews and staged recorded events since forever.
Memesters are at the end of the tech hierarchy. They're using stuff that's cheap or free.
That doesn't mean everybody is.
That doesn't mean everybody is.
I think where this will shine, creative people who want to make movies but not enough money to cast more than maybe a handful of people.
How about training a neutral net on fake videos to make it able to detect faked videos.
What about training an adversarial neural net to make faked videos pass that neural net?
See the problem?
See the problem?
easier said than done my friend.
whattya mean? you just train it. it's a neural network. /s
What are some of the libraries out there to do this stuff with?
Not knocking it all, but much like CRT’s used to provide free anti-aliasing, it seems as though part of the blending is achieved by the compression artifacts.
our benevolent media is worried about losing control of the propaganda machine..
I really don't see a reason to celebrate our accelerating slide into solipsism.
I find that putting a deeply faked face of an enemy on a video where someone is brutally killed or executed to be strangely pleasing.