Twitter analysis: identifying a propaganda bot network(bellingcat.com)
bellingcat.com
Twitter analysis: identifying a propaganda bot network
https://www.bellingcat.com/news/2019/09/03/twitter-analysis-identifying-a-pro-indonesian-propaganda-bot-network/
72 comments
If it's possible to do this analysis why does twitter not routinely shutdown these networks?
More than possible, it's been relatively simple to do for practically a decade. Case in point, here's one of my papers looking at twitter bots from 2011: https://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/viewP...
For a recent (last week!) review checkout https://arxiv.org/abs/1910.01720 (by a former colleague)
For a recent (last week!) review checkout https://arxiv.org/abs/1910.01720 (by a former colleague)
Same reason Facebook bends over backwards to make major figures in politics happy, or companies like Blizzard want to make certain governments happy, etc.
It's money. It's always money.
It's money. It's always money.
Can you point to some examples of Facebook bending over backwards to make major figures in politics happy?
Something besides the Warren allegations; MSNBC, NBC, ABC, Twitter, YouTube, and many others also ran the ad.
Something besides the Warren allegations; MSNBC, NBC, ABC, Twitter, YouTube, and many others also ran the ad.
Facebook recently explained that it does not intend to send politicians’ posts or ads to its third-party fact-checkers.
In a letter responding to the Biden campaign, Facebook said that politicians are not allowed to share a previously debunked viral hoax in ads, but their direct speech was ineligible for fact-checking.
“Our approach is grounded in Facebook’s fundamental belief in free expression, respect for the democratic process, and the belief that, in mature democracies with a free press, political speech is already arguably the most scrutinized speech there is,” Facebook’s head of global elections policy Katie Harbath wrote in the letter.
https://www.reuters.com/article/us-usa-election-facebook/bid...
facebook/twitter, even mainstream medeia/media in general love characters like trump. runs their numbers up and in addiction to reporters incentived to post fake news/clickbait as much as they can
In a letter responding to the Biden campaign, Facebook said that politicians are not allowed to share a previously debunked viral hoax in ads, but their direct speech was ineligible for fact-checking.
“Our approach is grounded in Facebook’s fundamental belief in free expression, respect for the democratic process, and the belief that, in mature democracies with a free press, political speech is already arguably the most scrutinized speech there is,” Facebook’s head of global elections policy Katie Harbath wrote in the letter.
https://www.reuters.com/article/us-usa-election-facebook/bid...
facebook/twitter, even mainstream medeia/media in general love characters like trump. runs their numbers up and in addiction to reporters incentived to post fake news/clickbait as much as they can
> We rely on third-party fact-checkers to help reduce the spread of false news and other types of viral misinformation, like memes or manipulated photos and videos. We don’t believe, however, that it’s an appropriate role for us to referee political debates and prevent a politician’s speech from reaching its audience and being subject to public debate and scrutiny. That’s why Facebook exempts politicians from our third-party fact-checking program.
https://newsroom.fb.com/news/2019/09/elections-and-political...
Also yes, I know they have had this in place since October 2016 with the whole "newsworthiness" thing.
To me that panders to any politician who wants to lie to manipulate the public through the surgical approach which is demographics marketing that exists on the FB platform (I'm a marketer - it's an outstanding platform). This is what I mean by "bends over backwards to make major figures in politics happy" - LOTS of politicians have taken advantage of this in their advertising. Anecdotally, I believe I have seen it first hand, and I see it as "kissing politician's butts for more engagement".
I think it's wrong - I think they should hold every advertiser to the same standards regardless if it's political or not. I see "newsworthiness" as a cop-out to say "we'll keep it up if we decide to keep it up". It puts FB in a scary position of being the thought police, and I just don't like that.
I'm not taking a side - please don't get my HN account deleted.
https://newsroom.fb.com/news/2019/09/elections-and-political...
Also yes, I know they have had this in place since October 2016 with the whole "newsworthiness" thing.
To me that panders to any politician who wants to lie to manipulate the public through the surgical approach which is demographics marketing that exists on the FB platform (I'm a marketer - it's an outstanding platform). This is what I mean by "bends over backwards to make major figures in politics happy" - LOTS of politicians have taken advantage of this in their advertising. Anecdotally, I believe I have seen it first hand, and I see it as "kissing politician's butts for more engagement".
I think it's wrong - I think they should hold every advertiser to the same standards regardless if it's political or not. I see "newsworthiness" as a cop-out to say "we'll keep it up if we decide to keep it up". It puts FB in a scary position of being the thought police, and I just don't like that.
I'm not taking a side - please don't get my HN account deleted.
So if Trump runs an ad stating "Vote for me, Donald J. Trump, I made the economy boom and the stock market reach all time highs" then it should be banned by Facebook (it's after all provably false)? Is that what I'm hearing?
That would be utterly terrifying if Facebook said their policy was to take down political/campaign ads which contain "misinformation". Ripe for abuse.
That would be utterly terrifying if Facebook said their policy was to take down political/campaign ads which contain "misinformation". Ripe for abuse.
I'm out this is getting too political. Sorry I just don't want to piss off the mods by bringing a specific politician into the convo.
You might be able to consider their hesitation to take action on Trump's Facebook page even though it breaks their ToS and is exactly the thing they are "trying" to fight right now [0]. Idk if that's something you'd consider to qualify as keeping political figures happy, but I think it's right on the cusp of that at least.
[0] https://www.vox.com/policy-and-politics/2019/10/9/20906612/t...
[0] https://www.vox.com/policy-and-politics/2019/10/9/20906612/t...
On the other hand, my twitter account was banned almost immediately after creating it this year. They said it was for “breach of terms of service” - and they won’t even let me delete the empty account now and declared they won’t respond to my support requests.
I think it was my VPN. I just wanted to thank somebody for a helpful tweet
I think it was my VPN. I just wanted to thank somebody for a helpful tweet
Facebook is similarly pretty bad: https://popular.info/p/the-tiny-alaskan-public-broadcaster-5...
Because the false positive rates are still rather high as there's no universal and foolproof definition of what behavior reliably tags people as bots.
just like spam email, and yet hardly any get past Gmail's filters.
The difference in impact between an email accidentally getting lost in the spam folder, and having your social media account getting banned, can be a rather vast one.
Because all those bots look good on the recurring visitor stats and the provocation is good for engagement
The analysis in this post is only on two rather small hashtags (a very small percentage of the whole of Twitter).
It also involves a lot of "eyeballing", particularly the part where the author looks at the graph and notices a weird pattern. That's very hard to do automatically.
It also involves a lot of "eyeballing", particularly the part where the author looks at the graph and notices a weird pattern. That's very hard to do automatically.
> That's very hard to do automatically.
When you run a worldwide network that has significant potential for abuse and spreading misinformation I would consider it to be your duty to monitor it and guard against these kinds of things. If it costs too much then maybe you shouldn't be in business or should change your business model so that you can afford to clean up the damage your business is doing to society.
When you run a worldwide network that has significant potential for abuse and spreading misinformation I would consider it to be your duty to monitor it and guard against these kinds of things. If it costs too much then maybe you shouldn't be in business or should change your business model so that you can afford to clean up the damage your business is doing to society.
Issue is if you run such automation with aggresive settings that would pick up this - you also increase false-positives and those for such a business could have a more negative impact than a few small bot nets.
So you do need people somewhere in the loop to handle these things.
What would be nice though would be if social media offered rewards for identifying such abuse, after all - they happily pay bug bounties, why not abuse bounties! That would get them lots of zero-hour/non-contract workers doing their job for them and big companies love that it seems and to be able to do that in a PR way that has positives. Well, it works for BUG bounties, why not incentivise the populus to police the populus against abuse as well.
So you do need people somewhere in the loop to handle these things.
What would be nice though would be if social media offered rewards for identifying such abuse, after all - they happily pay bug bounties, why not abuse bounties! That would get them lots of zero-hour/non-contract workers doing their job for them and big companies love that it seems and to be able to do that in a PR way that has positives. Well, it works for BUG bounties, why not incentivise the populus to police the populus against abuse as well.
> you also increase false-positives
...
> if social media offered rewards for identifying such abuse
couldn't this also be gamed for false positives, and create an incentive for groups to form smear-for-profit entities which try to spin small communities as 'bad' to the benefit of others?
...
> if social media offered rewards for identifying such abuse
couldn't this also be gamed for false positives, and create an incentive for groups to form smear-for-profit entities which try to spin small communities as 'bad' to the benefit of others?
When life is a game, everything is gameable. However, as humans are good at adapting, you are playing one large mass of people against a lesser mass of bad players. So whilst you may get gaming of the system, that would still stand out to others who would investigate.
But let us not forget, things like posting IP and client and other such details that may prove useful in identifying such abuse, may well elude public investigation. So it would be these niche area's in which Twitter could then focus upon.
However, as an example of a slightly comparable problem - wiki editing and the weighted credibility of editors over time, we may well end up creating a hidden social media based upon a small subset of society and perhaps yielding a bias in some direction or another in judgment of what is abuse.
But let us not forget, things like posting IP and client and other such details that may prove useful in identifying such abuse, may well elude public investigation. So it would be these niche area's in which Twitter could then focus upon.
However, as an example of a slightly comparable problem - wiki editing and the weighted credibility of editors over time, we may well end up creating a hidden social media based upon a small subset of society and perhaps yielding a bias in some direction or another in judgment of what is abuse.
agree all is gameable - this is why I argue social media should provide tools for users to filter their own content rather than being everyones nanny
Totaly, that would work. Let people impose their world view upon themselves via filters instead of imposing it upon everybody, would solve so much and tick all the free speech boxes.
But with the ways social media churns upon spam/botnets, it would place a huge load upon people self filtering.
But with the ways social media churns upon spam/botnets, it would place a huge load upon people self filtering.
Exactly, of course they’d have automated it by now if they could. This isn’t like spam mail, it’s propaganda that looks a whole lot like legitimate dissent or information, that’s the whole point. They won’t be full of spammy referral links and easily identifiable keywords for products or scams.
Yes, tech companies do not get to hide behind “we can’t do it because with humans.” Apple spends a significant amount of resources on human evaluation, and Twitter should not be allowed to shrug its shoulders and say “we are trying, but detection is hard.”
It is not unreasonable to take the top 5k accounts/hash tags and manually evaluate the top replies and retweets. Human evaluation can be very streamlined with clean evaluation rules, and these human evaluations get fed back to machine learning models.
It is not unreasonable to take the top 5k accounts/hash tags and manually evaluate the top replies and retweets. Human evaluation can be very streamlined with clean evaluation rules, and these human evaluations get fed back to machine learning models.
Apple charges one thousand dollars for a phone; twitter is free. Is it unreasonable to expect a lower quality of service? Twitter didn't make its first annual profit until last year, whereas Apple has staggering amounts of cash burning a hole in its pocket.
And after all that, Apple's recent products have been somewhat lackluster. Catalina is currently a dumpster-fire; a friend installed it and had his machine just sit frozen for a while along with a huge number of other problems. Keyboard design, too. Doesn't sound to perfect to me.
And after all that, Apple's recent products have been somewhat lackluster. Catalina is currently a dumpster-fire; a friend installed it and had his machine just sit frozen for a while along with a huge number of other problems. Keyboard design, too. Doesn't sound to perfect to me.
However, Apple’s failures don’t affect anyone that hasn’t bought into the ecosystem.
Twitter’s failures on the other hand affect everyone - misinformation spread through their platform can and does affect society at large regardless of whether you’re using Twitter yourself.
Twitter’s failures on the other hand affect everyone - misinformation spread through their platform can and does affect society at large regardless of whether you’re using Twitter yourself.
It sounds like Twitter needs to charge more. If we (society?) cannot support a free website enough for it to be viable, so be it.
Spot on. So, the equation runs as follows: ethical people running such networks will realize that they can't do it profitably and properly and so will shut down. The unethical ones don't give a fuck, don't do it properly, make gobs of money and claim victory.
This happens much more often than you might think, there are a quite a few examples of this effect.
This happens much more often than you might think, there are a quite a few examples of this effect.
The allegation here is that its the government doing it. Do you really think Twitter wants to take on governments?
Shutting down bots that spread misinformation is not against gov. If they do, it will mean they acknowledge that gov is behind all these.
Can you point out what identifies these tweets' content as "misinformation", in a way that's consistently operationalizable among the massive scale of frontline human reviewers that you're expecting? On its face, the content of the tweets is "the Indonesian govt built roads in West Papua", which hardly qualifies as misinformation.
The author's analysis that this is more sinister than simply a promotional campaign is, of course, correct: the coopting of the genocide hashtag to promote the government and the use of fake, international-sounding accounts are both skeezy as all hell. But to get to that conclusion, you have a professional journalist doing a level of post-automation analysis that's too complex to operationalize at scale. It requires too much critical thinking and discretion, and throwing a million human reviewers at it will just result in the kind of capricious false positives that are also railed against on HN and elsewhere (perhaps rightfully), with an undoubtedly high amount of false negatives too.
There are constant ignorant-layman calls for magical tech moderation that only touches bad targets and never steps on the good little guy, but none of the people making these blithe demands ever stop and think about whether this is possible, beyond "they're rich and smart, they definitely know how to do it".
The author's analysis that this is more sinister than simply a promotional campaign is, of course, correct: the coopting of the genocide hashtag to promote the government and the use of fake, international-sounding accounts are both skeezy as all hell. But to get to that conclusion, you have a professional journalist doing a level of post-automation analysis that's too complex to operationalize at scale. It requires too much critical thinking and discretion, and throwing a million human reviewers at it will just result in the kind of capricious false positives that are also railed against on HN and elsewhere (perhaps rightfully), with an undoubtedly high amount of false negatives too.
There are constant ignorant-layman calls for magical tech moderation that only touches bad targets and never steps on the good little guy, but none of the people making these blithe demands ever stop and think about whether this is possible, beyond "they're rich and smart, they definitely know how to do it".
This. And also, the cynic in me tells me that closing millions of accounts does not look good in your quarterly report on user base growth/active users.
They also get political blowback when they do this. There is an inevitable flurry of "Twitter is silencing our voices" posts when they shut down botnets. Congressmen start making statements about it.
https://grrrgraphics.com/wp-content/uploads/2018/07/jack_dor... https://pbs.twimg.com/media/DTmD9bmVMAA9M19.jpg https://static.thefederalistpapers.org/wp-content/uploads/20...
https://grrrgraphics.com/wp-content/uploads/2018/07/jack_dor... https://pbs.twimg.com/media/DTmD9bmVMAA9M19.jpg https://static.thefederalistpapers.org/wp-content/uploads/20...
Does twitter really deserve the trust though? I remember a lot of people I know complaining about getting locked out of their twitter accounts a year or two ago, with twitter demanding a phone number for them "for security". Then a few days ago it comes out that they were using those phone numbers for advertising purposes. Do they really deserve the benefit of the doubt on this?
Perhaps, but I think you would agree that "shareholder value" is a relic of the 80s and a poor metric to optimize for.
Twitter could pay people to do the work.
That would affect their user numbers they give investors...
Sometimes they do: https://blog.twitter.com/en_us/topics/company/2019/info-ops-...
not often enough.
I would guess it's traffic + engagement.
Do propaganda bots increase user engagement metrics?
Twitter shuts them down when they reach the press. Journalists emailing them about botnets will get them shut down faster than actual security researchers filing detailed reports.
My theory: botnets that fly under the public's radar allow them to sandbag their engagement stats. Once said botnets become public knowledge, it becomes a liability to advertisers and users who now are going to be worried about spam, so the fear of net loss of engagement spurns them to take action.
My theory: botnets that fly under the public's radar allow them to sandbag their engagement stats. Once said botnets become public knowledge, it becomes a liability to advertisers and users who now are going to be worried about spam, so the fear of net loss of engagement spurns them to take action.
Not a bot, Twitter recently blocked my account that had almost no activity without providing any reason and now I cannot delete it which is in contravention of GDPR.
> I cannot delete it which is in contravention of GDPR
Go on. They’ve refused your removal request with appropriate ID sent to their office, or?
Go on. They’ve refused your removal request with appropriate ID sent to their office, or?
Appropriate ID is username/password if real-life ID was not previously linked to an account directly.
> They’ve refused your removal request with appropriate ID sent to their office, or?
I have sent my email address to their support system while being logged in and they did confirm it. So yes. And they have not even responded so clearly in contravention with GDPR.
> Go on.
With?
I have sent my email address to their support system while being logged in and they did confirm it. So yes. And they have not even responded so clearly in contravention with GDPR.
> Go on.
With?
Uh, from the article: why is he using sudo to run a python script?
sudo twint -u bellanow1 –media -o papuabots1 –csv
Because confusing state of version management in 2019: https://twitter.com/noneprivacy/status/1168955093934399493
> If you used sudo because you installed the package "system wide", try with "pip3 install --user twint" which will install twint in your home directory, and so you'll not be asked to use sudo
> If you used sudo because you installed the package "system wide", try with "pip3 install --user twint" which will install twint in your home directory, and so you'll not be asked to use sudo
I do not like the color of this bikeshed.
because sudo improves security over 'su' and raw root logins, obviously
Theory: he's a reporter, someone showed him the command, one time it didn't work and the other person said to use "sudo" possibly because he was trying to write a file somewhere he needed access to. Now he always uses sudo because it always works...
[deleted]
Surprising to see this on here. The ongoing genocide of the West Papuan people by Indonesia has largely been ignored by the West.
BTW the propaganda team behind this bot network has also been very active on reddit.
BTW the propaganda team behind this bot network has also been very active on reddit.
Though I can't keep up with every political conflict I'm particularly interested in the intersection of genocide, propaganda, and social media. It's OK to gmail me with tips or questions.
Every now and then bellingcat covers a "lesser" news story like this one, so they can pretend their main purpose isn't to push for regime change in Syria.
bellingcat made its mark documenting the MH17 shootdown. https://www.bellingcat.com/tag/mh17/ has the exhaustive history
Let's say they seem to be consistently on one side of the Russia - USA disputes, of which Syria has been one of the issues. Just a quick browsing of the articles published in the last few months show that the vast majority of the investigations is focused on various malfeasances of Russia, Syria or Iran.
Bellingcat investigations on Syria predate Russian involvement in Syria.
Bellingcat was founded in July 2014, while Russia has been siding with Assad from the start of the war- vetoing sanctions, providing weapons and preventing an open attack by the US in 2013.
Russia did not enter Syria until 2015. Bellingcat's founder was already publishing on Syrian conflict in 2012.
This narrative is falling apart lately with the Dutch parliament pushing unanimously to investigate the role of Ukrainian forces in the attack, Bellingcat is apparently just a pro-US propaganda tool.
I am not super up to speed on this news, but I had a quick look and my guess as to why you are being downvoted, is without context, it sounds like you are implying Ukraine may have had some role worth investigating in the shooting down of MH17.
That is not the case. It sounds like the consensus is that pro-Russian troops brought a missile over from Russia, and shot down MH17.
What the Dutch parliament is proposing an investigation into, is whether Ukraine should have officially shut down its airspace.
Implying that Ukraine is at fault for shooting down an airliner because they did not shut down the airspace seems like blaming someone for being raped and shot because they didn't lock their front door, which is patently absurd.
That is not the case. It sounds like the consensus is that pro-Russian troops brought a missile over from Russia, and shot down MH17.
What the Dutch parliament is proposing an investigation into, is whether Ukraine should have officially shut down its airspace.
Implying that Ukraine is at fault for shooting down an airliner because they did not shut down the airspace seems like blaming someone for being raped and shot because they didn't lock their front door, which is patently absurd.
It's not just chemtruthers and Assadists that are mad on Bellingcat. They stepped on plenty of toes elsewhere.
So?
You need to check the definition of "genocide" before saying that it is on going. There are human right abuses and neglect by the central government. These used to be happened all over Indonesia and still are but not just in Papua. The government is trying to fix the situation. There will be no another independence to another province of Indonesia
Are you a member of the Indonesian propaganda team?
Well, that is not the point but I am just a regular citizen
Definitely sound's like something a propagandist would say.
It looks like but I am not. You can verify my identity :D Genocide is a strong word. There's no intention of Indonesian government to eradicate any people today. The last time we did commit genocide was 1965 during the communist purge. Guess what? Australia, UK, and USA blessed the act.
> This exact sentence was published by many of the other bots in their own individual tweets using the hashtags #westpapuagenocide and #freewestpapua.
A clickbaity video "what the government is trying to hide" that contains pro-gov messages ("they've developed an app for personal finance management and are giving villages more opportunities", "they are helping the food industry") posted under those hashtags. Is that attempting to influence people that are looking up those hashtags (presumably expecting to find info on "the genocide"), or is that just to add noise to the hashtag?
It looks too high quality for just noise (so it's probably too expensive to produce a lot of that content), but on the other hand, I can't imagine someone saying "woah, there's a genocide going on in west papua? I better check twitter. Ah, here's a video ... mhh, they are deploying an app to help people with finances and there's something about the food industry? I'm now convinced there is no genocide". Like, is that some "if only people knew about our app, they wouldn't mind the crimes" thinking? Does that work?
A clickbaity video "what the government is trying to hide" that contains pro-gov messages ("they've developed an app for personal finance management and are giving villages more opportunities", "they are helping the food industry") posted under those hashtags. Is that attempting to influence people that are looking up those hashtags (presumably expecting to find info on "the genocide"), or is that just to add noise to the hashtag?
It looks too high quality for just noise (so it's probably too expensive to produce a lot of that content), but on the other hand, I can't imagine someone saying "woah, there's a genocide going on in west papua? I better check twitter. Ah, here's a video ... mhh, they are deploying an app to help people with finances and there's something about the food industry? I'm now convinced there is no genocide". Like, is that some "if only people knew about our app, they wouldn't mind the crimes" thinking? Does that work?