Content moderation is broken (2019)(eff.org)
eff.org
Content moderation is broken (2019)
https://www.eff.org/deeplinks/2019/04/content-moderation-broken-let-us-count-ways
71 comments
> A few paragraphs up, weren't they saying it was a bad thing that different rules apply to different people? Now they're saying it's a bad thing if the same rules apply to everyone?
Or they're saying that the rules themselves are problematic, and more sophistication is needed than just "this word is not allowed."
Or they're saying that the rules themselves are problematic, and more sophistication is needed than just "this word is not allowed."
> I'm kind of disappointed that the EFF never issued a correction or retraction for this claim.
What kind of correction or retraction would you be looking for? That is explicitly Twitter policy[1].
Trump may have eventually crossed a line. That doesn't mean his line was in the same place as other people's.
[1] https://help.twitter.com/en/rules-and-policies/public-intere...
What kind of correction or retraction would you be looking for? That is explicitly Twitter policy[1].
Trump may have eventually crossed a line. That doesn't mean his line was in the same place as other people's.
[1] https://help.twitter.com/en/rules-and-policies/public-intere...
Also someone proved this experimentally just by tweeting out the same things that Trump did and got banned https://arstechnica.com/tech-policy/2020/06/bot-banned-from-...
Of course the line isn't the same for everyone. Some people (such as presidents) are a lot more influential and consequential than others. Nobody cares what Joe Random thinks about something and if he crosses a line nobody cares if he's silenced.
Trump isn't the only one who had a different line, though his was probably looser than anyone else got. Any celebrity or public figure is going to have more leeway in what he or she is allowed to say.
Trump isn't the only one who had a different line, though his was probably looser than anyone else got. Any celebrity or public figure is going to have more leeway in what he or she is allowed to say.
> A few paragraphs up, weren't they saying it was a bad thing that different rules apply to different people.
No, the rule is you can’t use reclaimed slurs unless you’re part of the group targeted by them. So yeah, you don’t get to call someone who’s gay a fa__ot unless you are gay yourself. That’s like the whole point of reclaiming them.
The issue is that a dumb keyword filter can’t properly enforce this.
No, the rule is you can’t use reclaimed slurs unless you’re part of the group targeted by them. So yeah, you don’t get to call someone who’s gay a fa__ot unless you are gay yourself. That’s like the whole point of reclaiming them.
The issue is that a dumb keyword filter can’t properly enforce this.
It’s not just a dumb keyword filter that can’t enforce this. It’s not enforceable. Period.
Are we going to have the content moderation team guess the sexual preferences of random twitter accounts?
Are we going to have the content moderation team guess the sexual preferences of random twitter accounts?
This is, practically, unenforceable.
trump tweeted all kinds of things that would get a normal user banned. you should retract your comment.
My whole point is that he did get banned, and "with impunity" basically means "without getting banned or otherwise punished".
Only after he was no longer President. If he had won the election, he would still be spewing on Twitter. And now he is again.
> Only after he was no longer President.
No, he was still President when he got banned.
No, he was still President when he got banned.
I think moderation should be modeled by following / subscribing to different censors (taggers / flaggers). Then you can mix and match as you see fit.
For example:
- Blacklist using censor X for nsfw content with anything above level 3.
- Blacklist using censor Y for political content to filter any ad hominem
- Blacklist using censor Z on scientific studies and information to filter anything with small sample size or that’s politicized
Obviously rather complicated, but this is what I want. A censor doesn’t have to be an org, it could be a person or a network of people you trust / follow.
Yes bubbles will form. That’s fine, that’s our right to join and filter on a bubble. Each bubble will exhibit different behaviors, and much like nature the fittest will rise and “survive”.
For example:
- Blacklist using censor X for nsfw content with anything above level 3.
- Blacklist using censor Y for political content to filter any ad hominem
- Blacklist using censor Z on scientific studies and information to filter anything with small sample size or that’s politicized
Obviously rather complicated, but this is what I want. A censor doesn’t have to be an org, it could be a person or a network of people you trust / follow.
Yes bubbles will form. That’s fine, that’s our right to join and filter on a bubble. Each bubble will exhibit different behaviors, and much like nature the fittest will rise and “survive”.
What is the motivation to be a censor? Is it something that content consumers would pay for? Or would you monitize it by injecting your own ads based on the users expressed filter preferences?
This is the problem reddit is facing, and I think for anything at scale paying people is the only way forward.
The amount of work required just goes way beyond checking some posts here and then towards the day, and distributing the load towards a huge community of moderators would give other issues on consistency, internal battles and keeping up to date on changes.
To note the people getting paid don’t need to be employed by the platform, I’d totally see a community crowdfund a moderator to tag for them specific content.
The amount of work required just goes way beyond checking some posts here and then towards the day, and distributing the load towards a huge community of moderators would give other issues on consistency, internal battles and keeping up to date on changes.
To note the people getting paid don’t need to be employed by the platform, I’d totally see a community crowdfund a moderator to tag for them specific content.
[deleted]
Many people love the feeling of power they get from telling other people how to behave.
Yeah basically like a subscribe to curator list. It can be pub/sub based content and content filters. I think important to still visibly surface other echo chambers but allow oneself to opt in.
the problem is that the censors themselves are political and judge different things differently.
For example, I've seen many ad hominem attacks on one group of people but do the exact same thing to another more "popular" and "in" group and you get ban-hammered.
You can't really trust anyone. Unless you want biased content - if that is the case, then censor away and be happy in the bubble.
For example, I've seen many ad hominem attacks on one group of people but do the exact same thing to another more "popular" and "in" group and you get ban-hammered.
You can't really trust anyone. Unless you want biased content - if that is the case, then censor away and be happy in the bubble.
There are two big reasons for content moderation: to grow (or at least maintain) the userbase, and to protect the investors who own the platform.
Your idea would work wonderfully for the first point, assuming you had enough censors devoting enough time to the social networks you care about. If this scheme was adopted, people would be able to craft the perfect echo chamber, seeing only what they want to see, being exposed only to the ideas they want to think about. It would be the current algorithms on steroids.
But it would totally fail for the second point.
When $social_media_empire deplatforms someone calling for violence against the Jews, it's not (entirely) out of altruism, it's because their investors don't want to see headlines declaring $social_media_empire as the home of violent, holocaust-denying sociopaths.
It's not enough to hide $antisocial_behvior from people that don't want to encounter it, it's important to the platform that the platform not be seen as a home for that type of content. So, at some point, it becomes necessary to hide content even from the people who want to see it.
Your idea would work wonderfully for the first point, assuming you had enough censors devoting enough time to the social networks you care about. If this scheme was adopted, people would be able to craft the perfect echo chamber, seeing only what they want to see, being exposed only to the ideas they want to think about. It would be the current algorithms on steroids.
But it would totally fail for the second point.
When $social_media_empire deplatforms someone calling for violence against the Jews, it's not (entirely) out of altruism, it's because their investors don't want to see headlines declaring $social_media_empire as the home of violent, holocaust-denying sociopaths.
It's not enough to hide $antisocial_behvior from people that don't want to encounter it, it's important to the platform that the platform not be seen as a home for that type of content. So, at some point, it becomes necessary to hide content even from the people who want to see it.
You are right, but I think weighting too much on the extreme.
Inciting violence or holocaust denying is outside the spectrum of what can be accepted on any moderation platform (outside of the sheer legal issues, also for the reasons you are citing). Some people want those kind of speeches represented as well, but it just won’t happen, at least not on any platform with public and international exposure.
With anything illegal off the table, sthe debate is more on how to deal with things ranging from flat earthers, hate on fat, furries, sex, anti-capitalism, hyper-capitalism, bigotry, grifting, crypto bros etc
In that “somewhat legal but possibily be deeply disturbing or unwelcoming from some” fringe, advertisers will also gain from being allowed to filter on what moderation strategies people are choosing. Basically it gives them ways to dictate specific rules for their content (e.g a diet product could be fine being advertised next to sex and fat-hate but want nothing to do with crypto-bros, etc.)
Inciting violence or holocaust denying is outside the spectrum of what can be accepted on any moderation platform (outside of the sheer legal issues, also for the reasons you are citing). Some people want those kind of speeches represented as well, but it just won’t happen, at least not on any platform with public and international exposure.
With anything illegal off the table, sthe debate is more on how to deal with things ranging from flat earthers, hate on fat, furries, sex, anti-capitalism, hyper-capitalism, bigotry, grifting, crypto bros etc
In that “somewhat legal but possibily be deeply disturbing or unwelcoming from some” fringe, advertisers will also gain from being allowed to filter on what moderation strategies people are choosing. Basically it gives them ways to dictate specific rules for their content (e.g a diet product could be fine being advertised next to sex and fat-hate but want nothing to do with crypto-bros, etc.)
The problem isn't moderation. It's monopoly.
If there were 5 Twitters all competing against one another, they would have to have customer service since you could move to another company. The companies would actually allocate the necessary resources and that would dampen the worst excesses.
Until somebody (US, EU, etc.) gets serious and starts busting up these companies that rely on "network effect", this will all continue.
If there were 5 Twitters all competing against one another, they would have to have customer service since you could move to another company. The companies would actually allocate the necessary resources and that would dampen the worst excesses.
Until somebody (US, EU, etc.) gets serious and starts busting up these companies that rely on "network effect", this will all continue.
Users are not Twitter's customers, they are their product. Customers get service by paying for it.
big platforms follow a concept like we've had for public places. Just like you cant visit the pub naked. The solution is to rent or buy your own turf. Just like the privacy of your hone you can go wild. Rules still apply but those are covered much better by the legal system. A platform can still rent space to you and they can chose to promote your content but if alternative aggregation is available their moderation can just be shooting from the hip
Speak for yourself, party pooper! https://sports.yahoo.com/drink-british-pubs-nude-thanks-1535...
Well sure, but there are many many pubs, some require a tux and a tie, some requier shoes and a shirt, and some allow you to be there fully naked. Opening a new pub is also (relatively) cheap, and getting new users is relatively easy (if users are interested in what you have to offer).
While current large social networks are not a real monopoly, they're not far away... if you want profiles, photos etc., you're pretty much stuck with facebook (this also counts instagram), forums are replaced by reddit and chat is concentrated on a few major platforms (whatsapp... so facebook again, whatever the current google chat is, viber, telegram and maybe signal).
While saying "fuck you guys, i'm building my own reddit, but with hookers, and blackjack!" is simple, and setting up some reddit-like software is simple, moving users there is hard... that goes double for stuff like twitter or facebook, where in same cases, you need to use those to even get official government info.
And even if you do succeed with making your own facebook, due to hookers and blackjack, even if perfectly legal (free spech and everything), hosting providers and domain registrars might turn off your service, because you're doing something perfectly legal, but they still don't like it.
Proving a monopoly would probably be impossible, but avoiding one of the big social networks is becoming harder and harder... and who knows what new service someone like facebook or google might buy... could be telegram, could be viber,.. you never know.
If you look further (eg china), it becomes ever harder to avoid those, since many real-life services (eg restaurants) accept payment only via WeChat ("ap pfor everything"), and if you get banned from there in china, you're even more fucked than if you lose your google account over here.
While current large social networks are not a real monopoly, they're not far away... if you want profiles, photos etc., you're pretty much stuck with facebook (this also counts instagram), forums are replaced by reddit and chat is concentrated on a few major platforms (whatsapp... so facebook again, whatever the current google chat is, viber, telegram and maybe signal).
While saying "fuck you guys, i'm building my own reddit, but with hookers, and blackjack!" is simple, and setting up some reddit-like software is simple, moving users there is hard... that goes double for stuff like twitter or facebook, where in same cases, you need to use those to even get official government info.
And even if you do succeed with making your own facebook, due to hookers and blackjack, even if perfectly legal (free spech and everything), hosting providers and domain registrars might turn off your service, because you're doing something perfectly legal, but they still don't like it.
Proving a monopoly would probably be impossible, but avoiding one of the big social networks is becoming harder and harder... and who knows what new service someone like facebook or google might buy... could be telegram, could be viber,.. you never know.
If you look further (eg china), it becomes ever harder to avoid those, since many real-life services (eg restaurants) accept payment only via WeChat ("ap pfor everything"), and if you get banned from there in china, you're even more fucked than if you lose your google account over here.
Content moderation clearly found the line and blew past it. Those who have been taking advantage of this seem immensely upset they are losing this power. Really playing all their cards down and are talking about leaving the game and going out back.
What seems inevitable is the US government starts hosting public forums to talk; but being bound by free speech. Virtually anything goes unless it's an actual crime. What happens after this?
What seems inevitable is the US government starts hosting public forums to talk; but being bound by free speech. Virtually anything goes unless it's an actual crime. What happens after this?
> What seems inevitable is the US government starts hosting public forums to talk; but being bound by free speech. Virtually anything goes unless it's an actual crime. What happens after this?
Everyone gets tired of 19 out of every 20 posts being about why they need Viagra, and goes back to MySpace.
Everyone gets tired of 19 out of every 20 posts being about why they need Viagra, and goes back to MySpace.
>Everyone gets tired of 19 out of every 20 posts being about why they need Viagra, and goes back to MySpace.
So lets say advertising is considered free speech and you may freely do it. In essence it would be a tragedy of the commons. The out of control advertising would eliminate the usefulness of the forum.
So lets say advertising is considered free speech and you may freely do it. In essence it would be a tragedy of the commons. The out of control advertising would eliminate the usefulness of the forum.
How so? You just wouldn't follow the accounts that posted that kind of thing.
So when exactly did you go into the coma you just awoke from, 1993? No sort of forum can operate solely on content users explicitly follow. That's just unfederated blogs. On an online forum there's always going to be some public stream that pulls content from all user posts. Without moderation it almost instantly devolves to bigots and spammers. We've seen this routinely for the past 30 years of online forums.
So "just don't follow" is not only meaningless advice but it's painfully naive.
So "just don't follow" is not only meaningless advice but it's painfully naive.
> Be kind. Don't be snarky.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
> Please don't post shallow dismissals
Perhaps you should give these a read and edit your comment accordingly: https://news.ycombinator.com/newsguidelines.html
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
> Please don't post shallow dismissals
Perhaps you should give these a read and edit your comment accordingly: https://news.ycombinator.com/newsguidelines.html
Don't a lot of people use Twitter's chronological timeline view and completely ignore features like Discover? Isn't that basically what I'm describing?
I think this is how most people use Twitter, especially if they go so far as to install any extensions which block ads, clear CSS, etc. - I use Minimal Twitter which has been pretty good. The only time I see posts from people I'm not following is when someone I do follow is responding, quote-tweeting, or retweeting.
If you use Twitter as is, there are a lot of things promoted for you but it's not any sort of public feed, it's (theoretically) based on who people you follow follow, who they like tweets from consistently, that sort of thing. No Viagra ads are going to be showing up in that.
If you use Twitter as is, there are a lot of things promoted for you but it's not any sort of public feed, it's (theoretically) based on who people you follow follow, who they like tweets from consistently, that sort of thing. No Viagra ads are going to be showing up in that.
It’s too bad there’s no way to use hybrid approaches or use some kind of learning, but for machines! No heuristics to rank and prioritize content.
We better tell every big tech company that what they’ve been doing forever is impossible to do if they don’t also de platform those deplorables.
We better tell every big tech company that what they’ve been doing forever is impossible to do if they don’t also de platform those deplorables.
What happens after this?
We know exactly what happens because it's been attempted. It fills up with spam and screaming bigots and nobody uses it beside spammers and screaming bigots. Which is why the the only 'inevitable' thing about such a government-run forum is that it won't happen.
We know exactly what happens because it's been attempted. It fills up with spam and screaming bigots and nobody uses it beside spammers and screaming bigots. Which is why the the only 'inevitable' thing about such a government-run forum is that it won't happen.
Lets say bigotry does take hold. Could this not be a valid measurement? Would this not become a fantastic resource to law enforcement, mental health officials, etc? Perhaps it ends up serving a better purpose than what the forum might represent.
As I said, that's already been built several times over. Research? Sure, that's done too.
https://fsi.stanford.edu/news/sio-new-gab-report
https://fsi.stanford.edu/news/sio-new-gab-report
> We know exactly what happens because it's been attempted.
When have we ever had a government-run social network before? And why aren't physical, offline public squares full of screaming bigots?
When have we ever had a government-run social network before? And why aren't physical, offline public squares full of screaming bigots?
When have we ever had a government-run social network before?
We've had many failed attempts at 'anything as long as it's not breaking a law' online spaces.
why aren't physical, offline public squares full of screaming bigots?
Because we impose severe social costs on people for that sort of thing. The screaming bigots are only bold and screaming when they feel they can avoid them. An 'anything goes' pseudonymous online forum is effectively a gift to them and mostly them - most people prioritize other things in their online social spaces far above 'anything goes'.
We've had many failed attempts at 'anything as long as it's not breaking a law' online spaces.
why aren't physical, offline public squares full of screaming bigots?
Because we impose severe social costs on people for that sort of thing. The screaming bigots are only bold and screaming when they feel they can avoid them. An 'anything goes' pseudonymous online forum is effectively a gift to them and mostly them - most people prioritize other things in their online social spaces far above 'anything goes'.
>And why aren't physical, offline public squares full of screaming bigots?
Because the security guards responsible for the event probably throw them out. If you behave like a crazy person at a public debate or gathering you'll probably get removed from the venue or delegated to the protest area with the other crazy people and their signs.
If you're attenting a townhall, a community meeting, a church gathering or what have you you've always been expected to behave like a civilized human being, moderation is omnipresent in the good old offline world. This anthing goes mentality on the internet has no precedent.
Because the security guards responsible for the event probably throw them out. If you behave like a crazy person at a public debate or gathering you'll probably get removed from the venue or delegated to the protest area with the other crazy people and their signs.
If you're attenting a townhall, a community meeting, a church gathering or what have you you've always been expected to behave like a civilized human being, moderation is omnipresent in the good old offline world. This anthing goes mentality on the internet has no precedent.
I don't mean those kinds of places. I mean places that are 100% truly public squares, where what they're doing is legally protected and they can't be kicked out.
Such places don't exist outside of the fever dreams of libertarians and minarchists. Governments have an interest in maintaining order and civil society even in public spaces, and the rule of law always applies.
https://en.wikipedia.org/wiki/Forum_(legal)
https://en.wikipedia.org/wiki/Freedom_of_speech_in_the_Unite...
https://en.wikipedia.org/wiki/Forum_(legal)
https://en.wikipedia.org/wiki/Freedom_of_speech_in_the_Unite...
> And why aren't physical, offline public squares full of screaming bigots?
They are, but only now and then.
The reason is that, for most (not all) communities, there are only a few screaming bigots within easy reach of the town square, so their demonstrations often look like this[0].
With teh Internets tubes, you can gather together all those little groups of nutters, into one big nutball, so it looks like there's a lot more of them, per capita, than there actually are.
[0] https://www.youtube.com/watch?v=nu-0HDBJHc8
They are, but only now and then.
The reason is that, for most (not all) communities, there are only a few screaming bigots within easy reach of the town square, so their demonstrations often look like this[0].
With teh Internets tubes, you can gather together all those little groups of nutters, into one big nutball, so it looks like there's a lot more of them, per capita, than there actually are.
[0] https://www.youtube.com/watch?v=nu-0HDBJHc8
yeah aside from all that basically every community has regulations regarding disturbing the peace and needing approval for rallies etc. so anybody that got up on a soapbox in the middle of the park and started yelling slurs would probably be hauled away pretty quick.
> What happens after this?
4chan
4chan
In theory, everyone needs a platform, to be heard. It seems to me that one question in that context is this:
How can you give people a platform, but at the same time, tame the conversation?
How can you give people a platform, but at the same time, tame the conversation?
Back in the golden age of phpBBs, I would give posters 1 notice when veering OT or getting nasty, then transfer the (sub-)thread to our 'FlameWar' room. There, they could, and would, be as nasty as they liked to each other. I failed to stem the flow of toxicity, but merely re-directed it. It worked out pretty well, and ironically, the more toxic categories were the most popular viewed threads for trolls and lurkers, alike. Go figure.
The town square doesn't exist in the American dream, capitalists would rather the public pay an admissions fee. Anyways, publicly funding anything of the sort is communism, right? When compared to Europe, Americans can't even manage public transit because monied interests and NIMBYism interfere. And the public good from transit is way more demonstrable than say, a forum for your racist uncle air his conspiracy theories.
I always wondered why platforms try to uphold the illusion that they are fair. Why not just say, "There are no rules. There is no fairness. If we think you make our site better you are welcome, if you don't we may or may not ban you if we feel like it. No we won't explain ourselves, because we have no rules, we do whatever we want. Don't piss us of, especially not before we have had our morning coffee."
Then you would, forgo all the arguments, the arm chair lawyering, all the whataboutism, and trolls wouldn't have rules to push up against. No one could ever "technically not break any rules" by being an ass.
When companies try to act like a law institution, they get all the politics of making and enforcing laws dropped in their lap.
Then you would, forgo all the arguments, the arm chair lawyering, all the whataboutism, and trolls wouldn't have rules to push up against. No one could ever "technically not break any rules" by being an ass.
When companies try to act like a law institution, they get all the politics of making and enforcing laws dropped in their lap.
> Why not just say, "There are no rules. There is no fairness. If we think you make our site better you are welcome, if you don't we may or may not ban you if we feel like it. No we won't explain ourselves, because we have no rules, we do whatever we want. Don't piss us of, especially not before we have had our morning coffee."
That is, essentially, what the terms of service for every social media platform (that no one ever bothers to read) have always said.
That is, essentially, what the terms of service for every social media platform (that no one ever bothers to read) have always said.
It isn't what they say in the guidelines people read or when they issue press releases or speak in the media though. They speak as if they are serious, trying to navigate complex terrain, be fair, strike a balance, yadda yadda yadda...standard corporate half-truths.
Because you can’t come on a press event, tell them to read the TOS and go home. The message is adapted to the media and what they want to hear. I think the companies you are rooting for will never be the ones actually giving PR responses.
Because companies like to engage in censorship when opinions arise that go against what's popular. That's why HN is one of the best places to discuss, free speech is allowed wholly and there is no censorship or heavy-handed moderation at all.
The same censorship exists when you can’t read a message because it is too gray or white.
There is also censorship when a comment is flagged even without actual reason.
Another particularly annoying form of censorship is being unable to defend a position when providing something controversial which limits your submission rate.
HN often comes out with very tone deaf responses and takes, so I don’t think that it is the best argument. HN is also a rather niche community that caters to a very specific audience.
There is also censorship when a comment is flagged even without actual reason.
Another particularly annoying form of censorship is being unable to defend a position when providing something controversial which limits your submission rate.
HN often comes out with very tone deaf responses and takes, so I don’t think that it is the best argument. HN is also a rather niche community that caters to a very specific audience.
It's pretty clear at this point that the people working the censorship departments (I mean, Trust and Safety) have started to become like HOA leaders and enforcing random rules that don't clearly benefit the users.
tpmx(3)
I'm kind of disappointed that the EFF never issued a correction or retraction for this claim.
> users being censored for engaging in counterspeech or for using reclaimed terms
A few paragraphs up, weren't they saying it was a bad thing that different rules apply to different people? Now they're saying it's a bad thing if the same rules apply to everyone?