Meta’s Adversarial Threat Report, Third Quarter 2022(about.fb.com)
about.fb.com
Meta’s Adversarial Threat Report, Third Quarter 2022
https://about.fb.com/news/2022/11/metas-adversarial-threat-report-q3-2022/
54 comments
There is at least one company in the US whose product is a platform that exactly mimics FB’s UI so that US military can train without having to have use real world FB accounts. Other users can role-play as members of the public responding to the pretend posts. I can’t remember the name of the company I am thinking of, but I interviewed someone who’d left it. I’ll see if I can dig it up.
Gotta say, ethics aside, something like that would be so much fun to build. No doubt ridiculously lucrative, too. Basically just: Here's $100 million, stand up a punching bag for us.
The report from Graphika has additional info about the US activity: https://public-assets.graphika.com/reports/graphika_stanford...
> Our joint investigation found an interconnected web of accounts on Twitter, Facebook, Instagram, and five other social media platforms that used deceptive tactics to promote pro-Western narratives in the Middle East and Central Asia. The platforms’ datasets appear to cover a series of covert campaigns over a period of almost five years rather than one homogeneous operation.
> These campaigns consistently advanced narratives promoting the interests of the United States and its allies while opposing countries including Russia, China, and Iran. The accounts heavily criticized Russia in particular for the deaths of innocent civilians and other atrocities its soldiers committed in pursuit of the Kremlin’s “imperial ambitions” following its invasion of Ukraine in February this year. To promote this and other narratives, the accounts sometimes shared news articles from U.S. government-funded media outlets, such as Voice of America and Radio Free Europe, and links to websites sponsored by the U.S. military. A portion of the activity also promoted anti-extremism messaging.
> ... At the same time, Twitter and Meta’s data reveals the limited range of tactics IO actors employ; the covert campaigns detailed in this report are notable for how similar they are to previous operations we have studied. The assets identified by Twitter and Meta created fake personas with GAN-generated faces, posed as independent media outlets, leveraged memes and short-form videos, attempted to start hashtag campaigns, and launched online petitions: all tactics observed in past operations by other actors.
> Importantly, the data also shows the limitations of using inauthentic tactics to generate engagement and build influence online. The vast majority of posts and tweets we reviewed received no more than a handful of likes or retweets, and only 19% of the covert assets we identified had more than 1,000 followers. The average tweet received 0.49 likes and 0.02 retweets. Tellingly, the two mostfollowed assets in the data provided by Twitter were overt accounts that publicly declared a connection to the U.S. military.
> Our joint investigation found an interconnected web of accounts on Twitter, Facebook, Instagram, and five other social media platforms that used deceptive tactics to promote pro-Western narratives in the Middle East and Central Asia. The platforms’ datasets appear to cover a series of covert campaigns over a period of almost five years rather than one homogeneous operation.
> These campaigns consistently advanced narratives promoting the interests of the United States and its allies while opposing countries including Russia, China, and Iran. The accounts heavily criticized Russia in particular for the deaths of innocent civilians and other atrocities its soldiers committed in pursuit of the Kremlin’s “imperial ambitions” following its invasion of Ukraine in February this year. To promote this and other narratives, the accounts sometimes shared news articles from U.S. government-funded media outlets, such as Voice of America and Radio Free Europe, and links to websites sponsored by the U.S. military. A portion of the activity also promoted anti-extremism messaging.
> ... At the same time, Twitter and Meta’s data reveals the limited range of tactics IO actors employ; the covert campaigns detailed in this report are notable for how similar they are to previous operations we have studied. The assets identified by Twitter and Meta created fake personas with GAN-generated faces, posed as independent media outlets, leveraged memes and short-form videos, attempted to start hashtag campaigns, and launched online petitions: all tactics observed in past operations by other actors.
> Importantly, the data also shows the limitations of using inauthentic tactics to generate engagement and build influence online. The vast majority of posts and tweets we reviewed received no more than a handful of likes or retweets, and only 19% of the covert assets we identified had more than 1,000 followers. The average tweet received 0.49 likes and 0.02 retweets. Tellingly, the two mostfollowed assets in the data provided by Twitter were overt accounts that publicly declared a connection to the U.S. military.
> To promote this and other narratives, the accounts sometimes shared news articles from U.S. government-funded media outlets, such as Voice of America and Radio Free Europe, and links to websites sponsored by the U.S. military.
This feels a bit shoddy for us gov. Its not like there is a shortage of sources saying russia is evil. I would have assumed that the us military would put more effort into appearing neutral in their secret propaganda campaign instead of being so obvious.
This feels a bit shoddy for us gov. Its not like there is a shortage of sources saying russia is evil. I would have assumed that the us military would put more effort into appearing neutral in their secret propaganda campaign instead of being so obvious.
I don’t understand how the us military hasnt trained a model on engaging posts yet. With the amount of compute they probably have to throw at this you’d think they would have a sausage factory thats at least as good as your average reddit karma whore.
They probably have. They probably didn't get picked up by the "inauthentic" behavior spotters because they look authentic.
Does the US military have the power to suppress this kind of thing? Like, if they did get picked up, can they ensure that gets silently dropped?
I find it unlikely that this is the only one. This is the only one they let FB take down.
We probably have the Countering Foreign Propaganda and Disinformation Act to thank for this.
* US: 39 accounts
* China: 81 accounts
* Russia: 1,633 accounts
Either Russia invests orders of magnitude more in these coordinated inauthentic behavior campaigns, or they are just that much worse at flying under the radar.
Either Russia invests orders of magnitude more in these coordinated inauthentic behavior campaigns, or they are just that much worse at flying under the radar.
Typically you build up accounts and pages over a long time period and try to fly under the radar until you're big enough to affect change without getting caught. In this case, my read is Russia is much better at growing before getting caught, probably due to more experience with influence operations.
Or russians used more accounts per campaign. Or facebook puts more effort into finding russian campaigns. Or russian campaigns are inherently easier to identify, or trying to achieve more difficult goals.
> or they are just that much worse at flying under the radar.
At least some of Russia's operations are likely less covert and more "look what we can do". They're not _really_ trying to hide; if you're trying to produce political disruption, then "look, this political disruption was caused by semi-deniable foreign interference" makes for even more political disruption.
Remember Guccifer 2.0? That was a GRU operation, apparently unmasked because the GRU officer 'forgot' to use a VPN (thus leaving an IP address which literally belonged to the GRU in some logs). And, I mean, it's _possible_ that that's what really happened, but it seems far more likely that it was deliberate.
At least some of Russia's operations are likely less covert and more "look what we can do". They're not _really_ trying to hide; if you're trying to produce political disruption, then "look, this political disruption was caused by semi-deniable foreign interference" makes for even more political disruption.
Remember Guccifer 2.0? That was a GRU operation, apparently unmasked because the GRU officer 'forgot' to use a VPN (thus leaving an IP address which literally belonged to the GRU in some logs). And, I mean, it's _possible_ that that's what really happened, but it seems far more likely that it was deliberate.
Knowledge of this kind of disruption leads to more disruption? Why? What theories support this? Has this been shown empirically?
> Has this been shown empirically?
I have no idea. However, if you expect national intelligence agencies to only be interested in stuff that has been shown empirically, then I have some very bad news for you (to take one of the _funnier_ examples, the CIA spent nearly 20 years researching "psychic remote viewing", ie magic: https://en.wikipedia.org/wiki/Stargate_Project).
That said, you'd expect that this would work (the double-whammy disruption, that is, not the magician-spies).
Take the "Guccifer 2.0" example, for instance. "Guccifer 2.0", that is the Russian military intelligence agency, stole and leaked emails from the DNC, causing political disruption. It was then revealed, in pretty short succession, that Trump lackey Roger Stone had been in communication with "Guccifer 2.0", and that "Guccifer 2.0" was a GRU officer. This caused further political disruption, and didn't obviously cost Russia anything, so, really, why would they _not_ reveal it?
And it does seem to be a trend; it's not like this is the only example of Russia saying "we totally deny these claims [wink]".
I have no idea. However, if you expect national intelligence agencies to only be interested in stuff that has been shown empirically, then I have some very bad news for you (to take one of the _funnier_ examples, the CIA spent nearly 20 years researching "psychic remote viewing", ie magic: https://en.wikipedia.org/wiki/Stargate_Project).
That said, you'd expect that this would work (the double-whammy disruption, that is, not the magician-spies).
Take the "Guccifer 2.0" example, for instance. "Guccifer 2.0", that is the Russian military intelligence agency, stole and leaked emails from the DNC, causing political disruption. It was then revealed, in pretty short succession, that Trump lackey Roger Stone had been in communication with "Guccifer 2.0", and that "Guccifer 2.0" was a GRU officer. This caused further political disruption, and didn't obviously cost Russia anything, so, really, why would they _not_ reveal it?
And it does seem to be a trend; it's not like this is the only example of Russia saying "we totally deny these claims [wink]".
Fair points.
Stargate: wow! With that kind of return on investment on $20 M, psychic readings for $10 a pop sound like a bargain in comparison. At worst, the readers are good at reading some signals hiding in plain sight and prompting some self reflection.
Stargate: wow! With that kind of return on investment on $20 M, psychic readings for $10 a pop sound like a bargain in comparison. At worst, the readers are good at reading some signals hiding in plain sight and prompting some self reflection.
When you think about the volume of divisive political messages that have been coming through over the last few years, you have to assume that it’s Russia driven. They’ll amplify anything that divides the US.
You're assuming this sample is representative.
I mean, it's representative of everything Meta found in the third quarter, no?
but what are they incentivised to find?
Adversarial threats...?
What exactly are you implying?
What exactly are you implying?
Parents are implying that Facebook is incentivized to find anything they can, and so inevitably finds the easiest things to find.
As a consequence, their samples would represent the dumbest/simplest psyops, rather than a random sampling of all psyops.
In the same way that tax auditors tend to "find" a lot of basic tax fraud, but almost no sophisticated, structured, high income tax fraud.
As a consequence, their samples would represent the dumbest/simplest psyops, rather than a random sampling of all psyops.
In the same way that tax auditors tend to "find" a lot of basic tax fraud, but almost no sophisticated, structured, high income tax fraud.
That doesn't contradict the first comment, which says they either invest a lot more OR "are very bad at flying under the radar"
Or they use completely different tactics.
You can use a single extremely influential account or a swarm of small ones to reach equivalents levels of influence.
You can use a single extremely influential account or a swarm of small ones to reach equivalents levels of influence.
They could be fake Russian bots. It wouldn't be the first time Democrats prop up fake Russian bots that are really obviously Russian bots and are saying things in favor of the Republicans.
https://heavy.com/news/2017/10/roy-moore-alabama-twitter-rus...
https://siliconangle.com/2018/12/26/reid-hoffman-apologizes-...
https://heavy.com/news/2017/10/roy-moore-alabama-twitter-rus...
https://siliconangle.com/2018/12/26/reid-hoffman-apologizes-...
[deleted]
Or the limited FB resources were concentrating in that area?
Or, combination thereof.
Or, combination thereof.
Traditionally, the strategy for the US when it comes to exerting influence internationally has been to facilitate regime changes by force, either through the military apparatus or by funding and training opposition groups. This has left the US as a "pointilist empire" with military bases strewn throughout the world.
China's strategy has been to use its massive workforce and economy, buying US debt, investing (and thus buying out) infrastructure in developing nations and becoming an integral part of the supply chain for most of Western industry.
Russia's strategy has primarily been to create a neverending stream of disinformation, misinformation and half-truths. Instead of creating a false narrative, it creates tens or hundreds of them without fully committing to any of them so the "objective truth" just becomes one of many possible truths and any minor factual mistake leads people to "question the narrative". Opposition groups in foreign countries are funded, but only to an extent that provides a cover of plausible deniability for them. Individuals are assinated in the most blatantly obvious way but with full denial of any involvement. Internal opposition is jailed with selectively enforced laws that seem relevant enough to be plausible but seem to be based on bogus evidence the deeper you dig. Russia's goal is not to get you to believe a lie, Russia's goal is to get you to question reality itself.
Basically, for the US and China disinformation is just another weapon in the arsenal. For Russia it's just what Russia does.
PS: The invasion of Ukraine is a bit of a counterexample to this but given how much it is dragging out, that just demonstrates how bad they are at solving conflicts "the American way".
Of course you could make the argument that all countries use all of these techniques to exert influence. The annexation of Crimea was as much a military operation as it was a disinformation campaign. The US is a vital part of the supply chain for many of its allies. China infamously engages in information warfare even against its own population. But what I'm arguing is what each country's primary weapon of choice is. And for Russia it's to attack the concept of reality itself.
China's strategy has been to use its massive workforce and economy, buying US debt, investing (and thus buying out) infrastructure in developing nations and becoming an integral part of the supply chain for most of Western industry.
Russia's strategy has primarily been to create a neverending stream of disinformation, misinformation and half-truths. Instead of creating a false narrative, it creates tens or hundreds of them without fully committing to any of them so the "objective truth" just becomes one of many possible truths and any minor factual mistake leads people to "question the narrative". Opposition groups in foreign countries are funded, but only to an extent that provides a cover of plausible deniability for them. Individuals are assinated in the most blatantly obvious way but with full denial of any involvement. Internal opposition is jailed with selectively enforced laws that seem relevant enough to be plausible but seem to be based on bogus evidence the deeper you dig. Russia's goal is not to get you to believe a lie, Russia's goal is to get you to question reality itself.
Basically, for the US and China disinformation is just another weapon in the arsenal. For Russia it's just what Russia does.
PS: The invasion of Ukraine is a bit of a counterexample to this but given how much it is dragging out, that just demonstrates how bad they are at solving conflicts "the American way".
Of course you could make the argument that all countries use all of these techniques to exert influence. The annexation of Crimea was as much a military operation as it was a disinformation campaign. The US is a vital part of the supply chain for many of its allies. China infamously engages in information warfare even against its own population. But what I'm arguing is what each country's primary weapon of choice is. And for Russia it's to attack the concept of reality itself.
> "This network originated in Russia and targeted primarily Germany, and also France, Italy, Ukraine and the United Kingdom. The operation centered around a large network of websites carefully impersonating legitimate news organizations in Europe. There, they would post original articles that criticized Ukraine, praised Russia and argued that Western sanctions on Russia would backfire. They would then promote these articles, memes and YouTube videos on Facebook, Instagram, Telegram, Twitter, petitions websites Change[.]org and Avaaz, and LiveJournal"
Although this is appreciated, no matter what they do, and after Cambridge Analytica, trust in Facebook/Meta went downhill. To top it off, Mark’s vision of VR and how stubborn he is with the infamous Metaverse will really just push FB to bankruptcy. Unless an activist investor pushes them to fire him and change course.
This is like Wells Fargo trying to regain trust, same thing. You had one chance…
This is like Wells Fargo trying to regain trust, same thing. You had one chance…
> To top it off, Mark’s vision of VR and how stubborn he is with the infamous Metaverse will really just push FB to bankruptcy. Unless an activist investor pushes them to fire him and change course.
Let's make this testable. Bankruptcy by when? Saying, e.g. "at some point" or "eventually" is very problematic to falsify.
I've noticed statements of this kind often have a lot of caveats; e.g. ...
> unless an activist investor pushes them to fire him and change course
... typically there are many other unspoken/unwritten caveats/exceptions as well. When this happens, the statement becomes unfalsifiable. In such cases, I treat the statement more as a normative criticism (here, "Mark's vision of VR and how stubborn he is") rather than a testable prediction.
Let's make this testable. Bankruptcy by when? Saying, e.g. "at some point" or "eventually" is very problematic to falsify.
I've noticed statements of this kind often have a lot of caveats; e.g. ...
> unless an activist investor pushes them to fire him and change course
... typically there are many other unspoken/unwritten caveats/exceptions as well. When this happens, the statement becomes unfalsifiable. In such cases, I treat the statement more as a normative criticism (here, "Mark's vision of VR and how stubborn he is") rather than a testable prediction.
Funny how you defend this company, you must work there and in denial, or a Zucker fan.
I don’t have time for all your arguments; will answer one of them:
Regarding timing for bankruptcy - META lost ~65% of its COVID-Driven valuation in one year. Mainly driven by investors not trusting Mark’s vision of the Metaverse.
Excluding COVID-driven boom, META has lost ~33% of its solid valuation based on its core product foundation in the past five years.
If we follow this trend, and nothing external happens as I previously explained on activist investors, or acquisitions, META could be filing for bankruptcy in as soon as 1 year or a as long as 5 years.
I don’t have time for all your arguments; will answer one of them:
Regarding timing for bankruptcy - META lost ~65% of its COVID-Driven valuation in one year. Mainly driven by investors not trusting Mark’s vision of the Metaverse.
Excluding COVID-driven boom, META has lost ~33% of its solid valuation based on its core product foundation in the past five years.
If we follow this trend, and nothing external happens as I previously explained on activist investors, or acquisitions, META could be filing for bankruptcy in as soon as 1 year or a as long as 5 years.
Tell me more about your claim that an org, person, or brand gets only one chance at trust. What exactly do you mean?
What about the passage of time? Memories can be reshaped; public opinion even more so.
Is your claim falsifiable?
What about the passage of time? Memories can be reshaped; public opinion even more so.
Is your claim falsifiable?
This feels like it's a drop in the ocean. 100 accounts is nothing as far as bad actors go. It probably set the attackers back a few dollars.
With that in mind and with other comments here talking about how these attacks mirror others, and were somewhat naive even in their methodology, maybe they intended to lose these accounts to just evaluate some Meta detection systems. Apparently Stanford is writing up a full report on this incident which I’m sure will be well studied by the military departments involved.
any link on stanford's report? When will it be published? It sounds pretty interesting
Depends on the impact distribution, I'd imagine? I.e they could go after the most impactful bad actors which, assuming a power law (typical for social networks), would have vastly more reach than other accounts. Or it could be that they're catching the low hanging fruit (careless accounts). Wish they'd share some numbers (i.e followers/reach of these accounts etc).
I don't know what to expect exactly, but these numbers seem quite small. Wonder if bad actors are moving to other channels
Not only do the numbers look small, the explanations seem very poor. For example:
>We began our investigation after reviewing public reporting...
So what Meta is saying here is that despite Russia having started a land war in Europe, Meta didn't do any pro-active investigation and anti-influence operations. Instead it waited until public reporting uncovered that their service had acted as a useful idiot for fascists. Think about the level of information and data that Meta has access to that it could use to detect and prevent attacks, and instead your average journalist is more likely to actually uncover these attacks.
>We began our investigation after reviewing public reporting...
So what Meta is saying here is that despite Russia having started a land war in Europe, Meta didn't do any pro-active investigation and anti-influence operations. Instead it waited until public reporting uncovered that their service had acted as a useful idiot for fascists. Think about the level of information and data that Meta has access to that it could use to detect and prevent attacks, and instead your average journalist is more likely to actually uncover these attacks.
>Together, these two approaches worked as an attempted smash-and-grab against the information environment, rather than a serious effort to occupy it long-term.
I.e disintermediation: fascinating to see evidence of this in the field, and to see evidence of the investment made in this at the nation state level.
I.e disintermediation: fascinating to see evidence of this in the field, and to see evidence of the investment made in this at the nation state level.
'We found 2 amateur-hour networks that didn't have much engagement + 1 network we were tipped to "after reviewing public reporting into a portion of this activity by investigative journalists in Germany."'
Wow. If this is what they come up with for a quarterly PR piece, it doesn't signal FB actually cares past the lowest-hanging fruit.
Wow. If this is what they come up with for a quarterly PR piece, it doesn't signal FB actually cares past the lowest-hanging fruit.
The transparency is much appreciated.
[deleted]
meta2023(1)
[deleted]
>>>
1. United States: We removed 39 Facebook accounts, 16 Pages, two Groups and 26 accounts on Instagram for violating our policy against coordinated inauthentic behavior. This network originated in the United States and focused on a number of countries including Afghanistan, Algeria, Iran, Iraq, Kazakhstan, Kyrgyzstan, Russia, Somalia, Syria, Tajikistan, Uzbekistan and Yemen. The operation ran across many internet services, including Twitter, YouTube, Telegram, VKontakte and Odnoklassniki. It included several clusters of fake accounts on our platforms, some of which were detected and disabled by our automated systems prior to our investigation. The majority of this operation’s posts had little to no engagement from authentic communities.
We found this activity as part of our internal investigation into suspected coordinated inauthentic behavior in the region. We’ve shared information about this network with independent researchers at Graphika and the Stanford Internet Observatory, who have published their findings about this network’s activity across the internet on August 24, 2022. Although the people behind this operation attempted to conceal their identities and coordination, our investigation found links to individuals associated with the US military.