Twitter improves API usage for researchers(blog.twitter.com)
blog.twitter.com
Twitter improves API usage for researchers
https://blog.twitter.com/developer/en_us/topics/tips/2021/enabling-the-future-of-academic-research-with-the-twitter-api.html
45 comments
It's a shame access to this API is limited to academic institutions, as many social media/misinformation researchers are now independent or affiliated with journalistic institutions.
Misinformation and troll farms on Twitter start with unrestricted API usage to the masses.
Misinformation & troll farms appear to do just fine with the current restricted API.
What evidence is there that read-only API access will make it significantly easier for them (enough to outweigh the other upsides)?
What evidence is there that read-only API access will make it significantly easier for them (enough to outweigh the other upsides)?
So you're telling me API's are not useful while at the same time telling me that API's are useful. Trolls want API access for the same reason researchers want access. You could argue research is the first step to becoming an effective troll.
I don't see API access as being necessary for a small-scale trolling operation, and large scale operations have enough resources to work around the lack of API by scraping.
I'm not saying that API access is completely useless, I was raising the question of whether the potential (and relatively small) benefit to trolling outweighs the major upsides of API access being available for all.
I'm not saying that API access is completely useless, I was raising the question of whether the potential (and relatively small) benefit to trolling outweighs the major upsides of API access being available for all.
In what research contexts is API usage valid instead of scraping a view more similar to what people experience? If the Twitter site and API are retrospectively cleared of removed/suspended accounts with large impact, how does that affect retrospective studies?
Are there ethical implications of working with Twitter to gather data? Despite Twitter TOS, legal, IRB ok, are there informed consent issues in studying the artifacts of social media use?
Are there ethical implications of working with Twitter to gather data? Despite Twitter TOS, legal, IRB ok, are there informed consent issues in studying the artifacts of social media use?
Until now, none I think. The API only gave a partial view while scraping offered all tweets for a particular search term. The scraper had to be clever to juke the anti scraping systems but you would get a more complete data set than using the API.
And the streaming API was terrible. Even if there was no data on the stream you could consume tens of gigabytes of bandwidth a day. Dreadful.
And the streaming API was terrible. Even if there was no data on the stream you could consume tens of gigabytes of bandwidth a day. Dreadful.
One easy example is language, for example tracking the spread of new words or other language constructs. You don’t care how the site looks, you care about the text that was previously input.
I wonder how are they going to enforce their rules, e.g. non-commercial use. I assume this will require some monitoring to be effective. Large scale Twitter API access is typically pricey, malicious actors might try to buy or steal researcher's credentials to cut costs.
I recently tried to sign up for Twitter API and the process is nothing like what it used to be. You have to give them a lot of information to even qualify, such as what you're going to use it for. It used to be that those were just some fields you need to fill out and you could sign up immediately. But nowadays the application process requires a direct approval from their team, which means they're monitoring every API account like Apple does with their app store. And if you like about your usage you are probably liable
> You are either a master’s student, doctoral candidate, post-doc, faculty, or research-focused employee at an academic institution or university.
This is gross. Rather than using the internet as a democratizing force for education, they restrict the program to those already inside credential-granting institutions. So much great research has been done from outside the institution and yet Twitter is actively pushing outsiders to resort to scraping.
This is gross. Rather than using the internet as a democratizing force for education, they restrict the program to those already inside credential-granting institutions. So much great research has been done from outside the institution and yet Twitter is actively pushing outsiders to resort to scraping.
Is it me or is Twitter making a lot of announcements this month?
But not for suspended accounts
https://www.reuters.com/article/us-twitter-product/twitter-g...
https://www.reuters.com/article/us-twitter-product/twitter-g...
Twitter makes it easier for Researchers to use tweets and the Twitter API for research.
basing on those UI screens, then why research is always associated with academia?
there's a lot of strong people especially in CS who do not work with academia and still work on interesting stuff
there's a lot of strong people especially in CS who do not work with academia and still work on interesting stuff
Absolutely. The limitation to people associated with academic institutions is pretty old fashioned (and also, from another point of view, modern).
Because it's easy for Twitter to draw the line between commercial and non-commercial use?
Maybe it has to do with control. Student==likely low impact. Faculty==likely uncontroversial. API==looks like transparency.
I always wondered: how many of these tweets are just Justin Bieber fandom type posts, bots, spam, or other dross? Twitter is infamous for its bad signal to noise ratio. These researchers need to write algos to filter out all the noise
What's wrong with that? If you wanted to investigate, say, the rise of The Beatles - wouldn't you love to have access to the random thoughts of their fans in the 1960s?
Similarly, if you're researching bots and spam and how they manipulate people & markets - this is still useful.
Similarly, if you're researching bots and spam and how they manipulate people & markets - this is still useful.
Are there researchers out there using bot accounts to prospectively experiment with social media researchers?
A lot! I ran named entity recognition on the Twitter garden hose back in 2012. The top entities were Bieber and the Jonas brothers.
Is it better then the previous time that they changed it?
Researchers like Joan Donovan?
Could have saved you some work, the research results are that Twitter is very liberal, less conservative and nobody has seen a libertarian for days.
Where's your sources, numbers, methodology? I mean anyone can make a kneejerk statement based on their perception (read: bubble), but that's not science.
You should follow different people.
I'd think libertarians would love twitter, deplatforming is the free market at work and the government has no right to make them do business with anyone they choose not to
Twitter should be judged by the way it governs its platform. And from libertarian perspective it's governed poorly. Sure, under current laws they can get away with deplatforming in the way they do now, but there is nothing commendable or desirable about it for libertarians specifically.
And one might think liberals, putting liberty above order, would be aghast at silencing opponents to maintain order, and yet here we are. Our political theater has gotten pretty weird. The whole thing is looking more and more like unprincipled tribes vying for power.
"silencing opponents"
Nobody here is silenced. You do not have a right to a twitter account. People can, and do, make accounts elsewhere.
Literally anything Trump does can and is covered by the media; this is exactly the opposite of being "silenced".
Nobody here is silenced. You do not have a right to a twitter account. People can, and do, make accounts elsewhere.
Literally anything Trump does can and is covered by the media; this is exactly the opposite of being "silenced".
I was not talking about Twitter accounts. I do not care what Donald Trump does or does not do.
Silence is a strong word -- I'm certainly still able to hear news from Donald Trump, even if he was banned from Twitter.
As someone that doesn't try I still hear news about Trump but no longer from Trump. That's a big (welcome) change.
You're right about Trump. Most people don't have the platform he has.
Many dissenting voices have been removed from the conversation, and you wouldn't even know they are missing.
Many dissenting voices have been removed from the conversation, and you wouldn't even know they are missing.
Except I keep hearing about it constantly, everywhere, including Twitter. And yet I haven't seen a single one being banned for discussing fiscal policy, less regulation, conservative views on social programs etc. "Conservative voices [being] silenced" are almost always some variation of spamming evidently real-world damaging conspiracy theories or clear ToS violations.
I didn't say conservative - I said dissenting. Although I imagine many conservatives have learned to self-censor so they can remain part of the conversation there.
How about asserting that men and women are different biologically? Is that a conspiracy theory?
Or is it simply a dissenting viewpoint from the group-think of Twitter that was silenced...
https://thefederalist.com/2018/11/25/twitter-permanently-ban...
How about asserting that men and women are different biologically? Is that a conspiracy theory?
Or is it simply a dissenting viewpoint from the group-think of Twitter that was silenced...
https://thefederalist.com/2018/11/25/twitter-permanently-ban...
> In late 2018, Twitter changed its policy on hateful conduct and harassment to officially prohibit intentionally calling a trans person by the wrong pronouns or using their pre-transition names.[62] Beginning in August 2018, Murphy stated that her Twitter account was locked more than once after she tweeted about issues involving trans women.[63] Twitter permanently suspended Murphy's account in late November 2018, after she referred to Jessica Yaniv, a trans woman, as "him".[64][65][66] On February 11, 2019, Murphy filed a lawsuit against Twitter in response to her banning.[67] The suit was dismissed in early June, but Murphy stated that she intended to file an appeal.[68][69]
Clear ToS violation. Which proves my point.
Sidenote: The Federalist itself got its account banned from twitter for spreading covid misinformation.
Clear ToS violation. Which proves my point.
Sidenote: The Federalist itself got its account banned from twitter for spreading covid misinformation.
This thread started with:
> And one might think liberals, putting liberty above order, would be aghast at silencing opponents to maintain order, and yet here we are.
You aren't liberal, that's OK, and Twitter isn't a liberal platform, and definitely has a political bias towards the progressive left.
I'd rather make decisions for myself than let Twitter tell me what is or isn't misinformation.
> Sidenote: The Federalist itself got its account banned from twitter for spreading covid misinformation.
Thanks for proving my point
> And one might think liberals, putting liberty above order, would be aghast at silencing opponents to maintain order, and yet here we are.
You aren't liberal, that's OK, and Twitter isn't a liberal platform, and definitely has a political bias towards the progressive left.
I'd rather make decisions for myself than let Twitter tell me what is or isn't misinformation.
> Sidenote: The Federalist itself got its account banned from twitter for spreading covid misinformation.
Thanks for proving my point
> Twitter isn't a liberal platform, and definitely has a political bias towards the progressive left.
[citation needed] seeing as your previous example fell flat.
> > Sidenote: The Federalist itself got its account banned from twitter for spreading covid misinformation.
> Thanks for proving my point
I fail to see your point. They spread misinformation that would get people killed, and were banned for it.
I've not in my 30 years on the internet spent more than a passing moment on a BBS, forum, social media that let anything go, nor would I want to as I've seen what it devolves into.
You have not displayed any basis for your assertion that the moderation enforcement on twitter is biased.
What point did I prove for you?
[citation needed] seeing as your previous example fell flat.
> > Sidenote: The Federalist itself got its account banned from twitter for spreading covid misinformation.
> Thanks for proving my point
I fail to see your point. They spread misinformation that would get people killed, and were banned for it.
I've not in my 30 years on the internet spent more than a passing moment on a BBS, forum, social media that let anything go, nor would I want to as I've seen what it devolves into.
You have not displayed any basis for your assertion that the moderation enforcement on twitter is biased.
What point did I prove for you?
1. Not banned, posted 5 hours ago (stop spreading innocuous misinformation): https://mobile.twitter.com/fdrlst
2. What misinformation was spread?
Was it something like masks work or don’t work or something like Black Lives are more important than stopping the spread or outdoor activities are safe?
Ok, first search result says:
* The article, titled “How Medical ‘Chickenpox Parties’ Could Turn The Tide Of The Wuhan Virus,” argued that a “controlled voluntary infection” program could allow young people to return to work after contracting and recovering from the virus.*
https://www.nytimes.com/2020/03/25/us/chickenpox-parties-fed...
The Times quotes the MD whose conjecture published by The Federalist caused so much twit-concern and then goes on to quote criticism of two not-MDs (using their title of “Dr.” but not establishing that they are academics and not medical doctors). [1,2]
To keep things fair the do cite an academic who is also an MD who “said he would not endorse intentionally infecting people, but added that it was worth considering ways to increase herd immunity while protecting vulnerable patients.” [3]
How is it you can claim that “they spread misinformation that would get people killed?” Something controversial, sure. “Would get people killed?” Thats silly.
1 https://en.wikipedia.org/wiki/Arthur_Caplan
2 https://www.umass.edu/sphhs/person/faculty/andrew-lover
3 https://www.medschool.umaryland.edu/profiles/Morgan-Daniel/
2. What misinformation was spread?
Was it something like masks work or don’t work or something like Black Lives are more important than stopping the spread or outdoor activities are safe?
Ok, first search result says:
* The article, titled “How Medical ‘Chickenpox Parties’ Could Turn The Tide Of The Wuhan Virus,” argued that a “controlled voluntary infection” program could allow young people to return to work after contracting and recovering from the virus.*
https://www.nytimes.com/2020/03/25/us/chickenpox-parties-fed...
The Times quotes the MD whose conjecture published by The Federalist caused so much twit-concern and then goes on to quote criticism of two not-MDs (using their title of “Dr.” but not establishing that they are academics and not medical doctors). [1,2]
To keep things fair the do cite an academic who is also an MD who “said he would not endorse intentionally infecting people, but added that it was worth considering ways to increase herd immunity while protecting vulnerable patients.” [3]
How is it you can claim that “they spread misinformation that would get people killed?” Something controversial, sure. “Would get people killed?” Thats silly.
1 https://en.wikipedia.org/wiki/Arthur_Caplan
2 https://www.umass.edu/sphhs/person/faculty/andrew-lover
3 https://www.medschool.umaryland.edu/profiles/Morgan-Daniel/
> 1. Not banned, posted 5 hours ago (stop spreading innocuous misinformation): https://mobile.twitter.com/fdrlst
Not all bans are permanent (but thanks for the sanctimonious flair).
> 2. What misinformation was spread?
1) That you can have safe infections (many otherwise healthy people have died from covid, including kids).
2) That "voluntary" is even an applicable term in an epidemic. You are not merely risking your life, but that of others, regardless of what you think are safety measures.
And speaking of masks as you mentioned, they are generally not for your safety, but for the safety of everyone else, so they work best if a critical amount of people use them.
I find it very curious that you dive deep into the (at least) three people opposing the article, and not the MD posting it. An unlicensed MD who's been a business man for 25 years and was a skin doctor.
> How is it you can claim that “they spread misinformation that would get people killed?” Something controversial, sure. “Would get people killed?” Thats silly.
People die from covid, even outside of risk groups. That's not silly, that's a fact.
Not all bans are permanent (but thanks for the sanctimonious flair).
> 2. What misinformation was spread?
1) That you can have safe infections (many otherwise healthy people have died from covid, including kids).
2) That "voluntary" is even an applicable term in an epidemic. You are not merely risking your life, but that of others, regardless of what you think are safety measures.
And speaking of masks as you mentioned, they are generally not for your safety, but for the safety of everyone else, so they work best if a critical amount of people use them.
I find it very curious that you dive deep into the (at least) three people opposing the article, and not the MD posting it. An unlicensed MD who's been a business man for 25 years and was a skin doctor.
> How is it you can claim that “they spread misinformation that would get people killed?” Something controversial, sure. “Would get people killed?” Thats silly.
People die from covid, even outside of risk groups. That's not silly, that's a fact.
you would think from the amount of republican/conservative accounts they have banned that they have some AI or parser dedicated to banning these types of voices
It is true that the vast majority of "research" is done by non-academics. Lots of companies doing market research want to mine media data.
Still, I believe that this "social media research" is a bit overvalued. There was this wave of "social media is the primary source where information appear". But now many realized how freaking difficult to separate this data from the noise comparing to traditional news published by journalists.
Also, take a look on this article [2] about how Dataminr sells insights from Twitter data to foreign governments (2017). Seems like just a way to punish the opposition channels.
[1] https://newscatcherapi.com/
[2] https://www.theverge.com/2017/1/27/14412014/dataminr-twitter...