The unrealized potential of federation(drewdevault.com)
drewdevault.com
The unrealized potential of federation
https://drewdevault.com/2020/09/20/The-potential-of-federation.html
201 comments
> Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities?
Invite only. If I'm running a federated service for me and my family, the moderation is going to be astronomically better than anything big providers are doing because a couple minutes of moderation by me every week is astronomically more on a per user basis than what someone like Google or Facebook is doing.
Invite only. If I'm running a federated service for me and my family, the moderation is going to be astronomically better than anything big providers are doing because a couple minutes of moderation by me every week is astronomically more on a per user basis than what someone like Google or Facebook is doing.
That's a nonstarter for a service that's for anyone but the geekiest of geeks, though. If you have to be close to someone with both the tech know-how and interest to host an instance of a federated service, the service is effectively off-limits to approximately everyone. Non-technical users have to be able to use the service and have a good experience doing so.
That's part of the point, though — can a federated service become easy or self-managing enough for people to run without needing sysadmin expertise?
Drew's point is that people who can have or acquire that expertise are force multipliers — it's easy _enough_ that one person can run a node that carries less-technical users into the ecosystem. But I don't think we're really there yet, and worse the value isn't there in federated social media to offset the costs of time, money, and change.
We've seen distributed networks capture enough imagination and be usable enough to get mass adoption, but they did things like distribute tremendous proprietary media libraries on the cheap and without permission.
Services like Mastodon aren't as easy to spin up for a group as, say, running Napster was, or even seeding a torrent is now. But people, even non-techincal ones, either put up with those services' occasional unreliability or recruited more technical users to work them on their behalf because the relative value was so great.
So to me the problem with federation is making the value exceed the cost. It's a squeeze from both sides. We won't get there until an ActivityPub service is as easy to stand up as running an app on your device, AND the value of running one is large enough to make it worth the trouble.
It feels like messaging should be where the value end should have already taken off, but federated group chat is still a rat's nest of incompatible or feature-starved services. Federated social is still just on the wrong side of complexity and the value it provides honestly isn't that great to a mass audience.
Drew's point is that people who can have or acquire that expertise are force multipliers — it's easy _enough_ that one person can run a node that carries less-technical users into the ecosystem. But I don't think we're really there yet, and worse the value isn't there in federated social media to offset the costs of time, money, and change.
We've seen distributed networks capture enough imagination and be usable enough to get mass adoption, but they did things like distribute tremendous proprietary media libraries on the cheap and without permission.
Services like Mastodon aren't as easy to spin up for a group as, say, running Napster was, or even seeding a torrent is now. But people, even non-techincal ones, either put up with those services' occasional unreliability or recruited more technical users to work them on their behalf because the relative value was so great.
So to me the problem with federation is making the value exceed the cost. It's a squeeze from both sides. We won't get there until an ActivityPub service is as easy to stand up as running an app on your device, AND the value of running one is large enough to make it worth the trouble.
It feels like messaging should be where the value end should have already taken off, but federated group chat is still a rat's nest of incompatible or feature-starved services. Federated social is still just on the wrong side of complexity and the value it provides honestly isn't that great to a mass audience.
> But I don't think we're really there yet, and worse the value isn't there in federated social media to offset the costs of time, money, and change.
Not for 99% of the users, for now. But there is an increasing faction that is taking issue with what any company does with their data.
At first, caring about one's own data was mostly a political thing. Now, abuse of data is getting so ridiculous that how much time you spend in the supermarket queue may depend on the aggregated data supermarkets have on you (and yes, supermarkets often aggregate their customer data via third-party reputation aggregator companies). Not to mention credit interest rates, insurance premiums, visas, and airport security checks. Soon people will too realize that taking their data off from ad networks is a good way to save money. At that point, one either goes silent or searches for a limited, well-protected social network.
Not for 99% of the users, for now. But there is an increasing faction that is taking issue with what any company does with their data.
At first, caring about one's own data was mostly a political thing. Now, abuse of data is getting so ridiculous that how much time you spend in the supermarket queue may depend on the aggregated data supermarkets have on you (and yes, supermarkets often aggregate their customer data via third-party reputation aggregator companies). Not to mention credit interest rates, insurance premiums, visas, and airport security checks. Soon people will too realize that taking their data off from ad networks is a good way to save money. At that point, one either goes silent or searches for a limited, well-protected social network.
can a federated service become easy or self-managing enough for people to run without needing sysadmin expertise?
Sandstorm is/was most of the way there; it makes installing server apps really easy (assuming a hosting provider installs Sandstorm for you).
Sandstorm is/was most of the way there; it makes installing server apps really easy (assuming a hosting provider installs Sandstorm for you).
What kind of service is Sandstorm? I'd like to know more but the term is too broad to search for, I need more keywords.
have you tried 'sandstorm server'?
Invite-only with a chain of responsibility. That is, if I invite you, I'm responsible for you behavior; if you screw up and get banned, I may lose privileges or get banned with you.
Perhaps extend that one or two levels into the graph, with exponential decay. So if you invite someone, and that someone turns out to be a spammer, they get banned, you get banned, and I lose posting privileges for a week. And the person who invited me loses some privileges for a day. Etc. Add some level of manual review and algorithmic detection of clusters of accounts that exist only to invite other accounts that invite spammers, and...
... well, did anyone ever try this out? What are the obvious holes in this system that I'm missing?
Perhaps extend that one or two levels into the graph, with exponential decay. So if you invite someone, and that someone turns out to be a spammer, they get banned, you get banned, and I lose posting privileges for a week. And the person who invited me loses some privileges for a day. Etc. Add some level of manual review and algorithmic detection of clusters of accounts that exist only to invite other accounts that invite spammers, and...
... well, did anyone ever try this out? What are the obvious holes in this system that I'm missing?
I'm not really aware of anything like this, besides how mastodon instances connect to each other, and I think this idea has massive promise.
It's worth not just "a shot", but lots of earnest attempts.
It's worth not just "a shot", but lots of earnest attempts.
[deleted]
That's basically how lobste.rs works. The invite graph is also public (see https://lobste.rs/u). It works fine, but I wouldn't derive too much data from it, since it is a small community, and spammers / vote rings are rare enough that a manual approach works.
Another feature that I wish other places adopted is a public moderation log (https://lobste.rs/moderations) where users can see all the actions by the mod team. This helps keep the mods accountable.
Another feature that I wish other places adopted is a public moderation log (https://lobste.rs/moderations) where users can see all the actions by the mod team. This helps keep the mods accountable.
It would be interesting to see a multi-tenant social network: you start a network for your family/household/whatever as easily as if you were signing up to Facebook or whatever, and then you connect to friends by sending a request to them at their instance. Mastodon/pixelfed may be like 90% already: just need to come up with a good ux for automated deployments and a way to monetize it
Your assumption is that hosting is way more difficult than posting. What if it weren't?
I research in that direction: http://mro.name/ShaarliGo#install--update
I research in that direction: http://mro.name/ShaarliGo#install--update
> Next, identity. If grandpa can post, so can the bots
Yes, protection against a million fake accounts is the key scientific problem to solve (Sybil attack).
Our university lab has worked on this problem for 15+ years, its hard. Early indications are that ActivityPub and federation are not going to scale. Bot attacks are too ferocious for amateur admins. Big instances attract more people, however a single instance fails to scale.
Nobody knows how to build trustworthy self-organising systems with "self-governance". Somehow you need to deduce the reputation of others from their signed interaction history. Their privacy needs to be protected, but you can still calculate a reputation score. After 20 years of scientific progress around "distributed reputation systems" with 100+ scientific articles they have one thing in commons: they don't protect from the Sybil attack.
Server federation is not going to help if there is no trust fabric to build upon. The effortless onboarding plus mobile-first requirement of today means you even need to have smartphones talking to eachother directly, P2P-style. <plug> thankfully students in our lab now have carrier-grade NAT traversal operational between Android devices. Google: "Technology Stack for Decentralized Mobile Services" (quotes required).
Yes, protection against a million fake accounts is the key scientific problem to solve (Sybil attack).
Our university lab has worked on this problem for 15+ years, its hard. Early indications are that ActivityPub and federation are not going to scale. Bot attacks are too ferocious for amateur admins. Big instances attract more people, however a single instance fails to scale.
Nobody knows how to build trustworthy self-organising systems with "self-governance". Somehow you need to deduce the reputation of others from their signed interaction history. Their privacy needs to be protected, but you can still calculate a reputation score. After 20 years of scientific progress around "distributed reputation systems" with 100+ scientific articles they have one thing in commons: they don't protect from the Sybil attack.
Server federation is not going to help if there is no trust fabric to build upon. The effortless onboarding plus mobile-first requirement of today means you even need to have smartphones talking to eachother directly, P2P-style. <plug> thankfully students in our lab now have carrier-grade NAT traversal operational between Android devices. Google: "Technology Stack for Decentralized Mobile Services" (quotes required).
How does that work with interacting with other nodes in the federation? Now you have the whole huge userbase to deal with again, and only you to moderate.
If you're not talking to wide internet, what's the point of federating?
If you are, while i think email has proven that moderation is much much easier at scale if the attackers are spammers (astroturfing is a bit of a different story though)
If you are, while i think email has proven that moderation is much much easier at scale if the attackers are spammers (astroturfing is a bit of a different story though)
>1. Among other things, because of a dead-simple user experience that lets even grandpa post content after a few clicks. Federation, if it adds any level of complexity past the grandpa UX, would defeat the goal.
That's the point though, design for the lowest common denominator has been the modus operandi for the giants because it gets them the most profit. There is no proof that this is the way to create the most "useful" social network. Often I'm tired of the "organic spam" that users with easy access create non-stop in a lot of the communities I'm in. A technical barrier to entry isn't necessarily a bad thing in this case.
That's the point though, design for the lowest common denominator has been the modus operandi for the giants because it gets them the most profit. There is no proof that this is the way to create the most "useful" social network. Often I'm tired of the "organic spam" that users with easy access create non-stop in a lot of the communities I'm in. A technical barrier to entry isn't necessarily a bad thing in this case.
Being networks, a fundamental definition of "usefulness" is directly (or exponentially, rather) related to the size of the network. It's not the only aspect of value, but it is exponential, ad hard to beat.
One of the reasons amoral commercial efforts like fb won is because this modus operandi happens to harmonize standard commercial MO with what a network is.
There's definitely value in alternative MOs. As you say, a technical barrier to entry can have its uses. Wikipedia is a nice example. I don't think simplifying the editor UX to an intuitive one would necessarily be useful. But wikipedia needs a throttle on editing.
I'm not sure this relates to federation though. Niche communities don't need to be federated.
One of the reasons amoral commercial efforts like fb won is because this modus operandi happens to harmonize standard commercial MO with what a network is.
There's definitely value in alternative MOs. As you say, a technical barrier to entry can have its uses. Wikipedia is a nice example. I don't think simplifying the editor UX to an intuitive one would necessarily be useful. But wikipedia needs a throttle on editing.
I'm not sure this relates to federation though. Niche communities don't need to be federated.
>a fundamental definition of "usefulness" is directly (or exponentially, rather) related to the size of the network. It's not the only aspect of value, but it is exponential, ad hard to beat.
No that is not true. Judging from my experience with various subreddits anything above 100k quickly becomes trash. You don't need as many users as possible.
>One of the reasons amoral commercial efforts like fb won is because this modus operandi happens to harmonize standard commercial MO with what a network is.
For what facebook wants to be i.e. a profile for all your contacts that is indeed the case. For hobby/activity related communities that's not true.
>I'm not sure this relates to federation though. Niche communities don't need to be federated.
It does relate to federation because right now if I open a new subreddit I am bound to the wishes of the company and they can change the UI in such a massive way that any community that benefits from long-form text posts quickly dysfunctional. I have noticed the gradual decline in content and comment quality on reddit since the redesign and the push for mobile app usage and it's slowly creeping into every subreddit out there.
With a federated alternative to reddit I could launch a server on my own and have control over how the community evolves over time( or rather, people who have the community's best interests in mind can in general). I hope someone takes what Mastodon does and make a reddit alternative.
No that is not true. Judging from my experience with various subreddits anything above 100k quickly becomes trash. You don't need as many users as possible.
>One of the reasons amoral commercial efforts like fb won is because this modus operandi happens to harmonize standard commercial MO with what a network is.
For what facebook wants to be i.e. a profile for all your contacts that is indeed the case. For hobby/activity related communities that's not true.
>I'm not sure this relates to federation though. Niche communities don't need to be federated.
It does relate to federation because right now if I open a new subreddit I am bound to the wishes of the company and they can change the UI in such a massive way that any community that benefits from long-form text posts quickly dysfunctional. I have noticed the gradual decline in content and comment quality on reddit since the redesign and the push for mobile app usage and it's slowly creeping into every subreddit out there.
With a federated alternative to reddit I could launch a server on my own and have control over how the community evolves over time( or rather, people who have the community's best interests in mind can in general). I hope someone takes what Mastodon does and make a reddit alternative.
I do sympathize with the argument (I've made it myself, in the past), but ...
... the problem with that is there are a lot of damned smart people who are good at what they're good at precisely because they view "wrangling tech" as a complete waste of their time.
Artists and musicians are a really good example, and are the key reason I use Twitter (pretty much the entire indie game industry lives on that platform), but this principle also extends to doctors, lawyers, scientists, teachers, historians, practically everyone in all the sciences, trades, and humanities ... except computer programmers. I've had aspirations towards being a polymath, and one of the bitter things I've learned about the programming-autodidact death march is how much of what someone learns isn't important fundamental knowledge, but is just byzantine details that will evaporate half a decade later when the software in question goes obsolete. It's excruciating to waste time learning something like that (which will become worthless) when I could be learning core programming fundamentals -- or could be learning something in a different field like drawing that will stay valid for the rest of my life.
I was a desktop developer looking-in-from-the-outside at web development, and I timed my entry into the field specifically to avoid the hot mess that was the API differences between netscape (yes, Netscape Communicator 4.5) and IE 6 at the time. As a programmer with enough of a nose for bad code smell, I took one look at that and thought to myself "this is completely insane." I could have knuckled down, busted the books and gotten all those api minutiae down just pat. Just in time for jQuery (or future browser versions) to show up and render them irrelevant forever.
It wasn't insane because it was a mess - it was insane because it was definitely going to go away, and soon. And once it did go away all the time I'd spent learning and mastering it would have been utterly wasted, sure - but wasted at the expense of not doing something else that's more important. That's time I could have spent learning the piano. Or a foreign language.
Now - not all programming is like that - eventually you've gotta bite the bullet and decide something's "good enough to get the job done". Many tech stacks actually end up in a nice sweet spot where they're coherent enough to learn quickly, and stable enough (in the sense of an API) to get out of the way and let you approach long-term projects without the ground shifting under your feet. But if you just blindly vacuum up proficiencies as they come off the treadmill you'll quickly discover you're doing nothing else with your life.
--
This is the relation most humans have with tech. Especially the smart ones who are really into something else.
It's a means to an end - and can work wonderful magic. But it demands a price.
... the problem with that is there are a lot of damned smart people who are good at what they're good at precisely because they view "wrangling tech" as a complete waste of their time.
Artists and musicians are a really good example, and are the key reason I use Twitter (pretty much the entire indie game industry lives on that platform), but this principle also extends to doctors, lawyers, scientists, teachers, historians, practically everyone in all the sciences, trades, and humanities ... except computer programmers. I've had aspirations towards being a polymath, and one of the bitter things I've learned about the programming-autodidact death march is how much of what someone learns isn't important fundamental knowledge, but is just byzantine details that will evaporate half a decade later when the software in question goes obsolete. It's excruciating to waste time learning something like that (which will become worthless) when I could be learning core programming fundamentals -- or could be learning something in a different field like drawing that will stay valid for the rest of my life.
I was a desktop developer looking-in-from-the-outside at web development, and I timed my entry into the field specifically to avoid the hot mess that was the API differences between netscape (yes, Netscape Communicator 4.5) and IE 6 at the time. As a programmer with enough of a nose for bad code smell, I took one look at that and thought to myself "this is completely insane." I could have knuckled down, busted the books and gotten all those api minutiae down just pat. Just in time for jQuery (or future browser versions) to show up and render them irrelevant forever.
It wasn't insane because it was a mess - it was insane because it was definitely going to go away, and soon. And once it did go away all the time I'd spent learning and mastering it would have been utterly wasted, sure - but wasted at the expense of not doing something else that's more important. That's time I could have spent learning the piano. Or a foreign language.
Now - not all programming is like that - eventually you've gotta bite the bullet and decide something's "good enough to get the job done". Many tech stacks actually end up in a nice sweet spot where they're coherent enough to learn quickly, and stable enough (in the sense of an API) to get out of the way and let you approach long-term projects without the ground shifting under your feet. But if you just blindly vacuum up proficiencies as they come off the treadmill you'll quickly discover you're doing nothing else with your life.
--
This is the relation most humans have with tech. Especially the smart ones who are really into something else.
It's a means to an end - and can work wonderful magic. But it demands a price.
I don't think this argument is really that relevant. Imo for a federated network to be successful it doesn't need to be huge, just big enough to sustain itself and keep the conversations going. Besides that the control over rules and the experience of the network that federation can provide can be beneficial. The only reason why Mastodon doesn't have an indie music server is because of the network effect of Twitter.
I think also a certain amount of non freedom helps.
One of the big things facebook did is you couldn't customize your page. No autoplaying midis with seizure inducing gifs. Also helps with brand cohesion.
Federation gives people freedom. Quite frankly users cannot be trusted to maintain a good experience if given freedom.
One of the big things facebook did is you couldn't customize your page. No autoplaying midis with seizure inducing gifs. Also helps with brand cohesion.
Federation gives people freedom. Quite frankly users cannot be trusted to maintain a good experience if given freedom.
So instead of we’ve got rampant racism, conspiracy theories, fake news and attempts to sway elections in that direction all over the world. If that’s the good experience, bring back the midis!
Most of us here presumably recall the early iterations of Twitter and Facebook. Both succumbed to dark patterns a very long time ago. Your argument seems to be a slow death is preferable to a fast one.
I think the "feed" aspect needs to be considered also.
The power of Facebook, Twitter and such is largely their feed. The feed is an answer to point #1: grandpa simplicity. The feed is the thing that needs to be protected from points #2 & 3: identity and culture.
The feed is that will motivate participants, content creators, etc. IE, many will create content for the feed. Reddit/Digg/HN upvoting were a simple way of doing the feed that proved this point and paved the way for Facebook & Twitter. A lot of mass adoption services created after went extreme. Feed only. The Tinder UX is a feed of datable people, for example. Spotify's importance is being solidified by being a feed. Importance in spotify's context is largely "being the place where music gets popular."
One way or another, the feed issue must be addressed. New content UIs will either have a successful feed or transcend the feed some way.
I'm not sure how this relates to federation. A feed seems a centralizing element by nature. Successful federated systems of the past (the the WWW) tended to leave the feed ("discovery," in a pre-feed context) to someone else... which turned out to be google, fb, etc.
However the labyrinth is is designed, what matters is how the feed works.
The power of Facebook, Twitter and such is largely their feed. The feed is an answer to point #1: grandpa simplicity. The feed is the thing that needs to be protected from points #2 & 3: identity and culture.
The feed is that will motivate participants, content creators, etc. IE, many will create content for the feed. Reddit/Digg/HN upvoting were a simple way of doing the feed that proved this point and paved the way for Facebook & Twitter. A lot of mass adoption services created after went extreme. Feed only. The Tinder UX is a feed of datable people, for example. Spotify's importance is being solidified by being a feed. Importance in spotify's context is largely "being the place where music gets popular."
One way or another, the feed issue must be addressed. New content UIs will either have a successful feed or transcend the feed some way.
I'm not sure how this relates to federation. A feed seems a centralizing element by nature. Successful federated systems of the past (the the WWW) tended to leave the feed ("discovery," in a pre-feed context) to someone else... which turned out to be google, fb, etc.
However the labyrinth is is designed, what matters is how the feed works.
I totally agree, the interface has to be simple.
First movers always take the majority of the market share if there are no compelling reasons to use a similar service.
First movers always take the majority of the market share if there are no compelling reasons to use a similar service.
I also agree. "Federated" is not a killer feature. Simplicity is, and that needs to be a priority.
'Federated' is not even a feature.
'Privacy' itself is barely a feature - people do care, but it's rarely #1 on their list.
Techies viewing this from the more strategic, systems perspective have to get into the headspace of individuals to see it from a 'user centric' perspective - because the answer to many of the questions lie there.
For starters, very few people care that Facebook for example is 'centralized' - they care about the material value offered to them 'right now' which is to say, access to their friends, posts, information etc.. Everything is is a distant second. Issues that do not directly affect their experience an even more distant third.
'Privacy' itself is barely a feature - people do care, but it's rarely #1 on their list.
Techies viewing this from the more strategic, systems perspective have to get into the headspace of individuals to see it from a 'user centric' perspective - because the answer to many of the questions lie there.
For starters, very few people care that Facebook for example is 'centralized' - they care about the material value offered to them 'right now' which is to say, access to their friends, posts, information etc.. Everything is is a distant second. Issues that do not directly affect their experience an even more distant third.
I remember attending a talk where the conclusion was that you should never market a product as "green" or "eco-friendly". The reason being that if that's your value proposition then you are now limiting your product to people who care about "green" and "eco-friendly". It's better to market a product that is really good but happens to be eco-friendly.
The same seems to be pretty much true of "federation".
The same seems to be pretty much true of "federation".
I totally disagree. There must a high bar to entry, for useful discussions, instead of talks about tits and shits. Look at HN, for example.
> for useful discussions
That rules out my children's grandparents then.
That rules out my children's grandparents then.
>Among other things, because of a dead-simple user experience that lets even grandpa post content after a few clicks. Federation, if it adds any level of complexity past the grandpa UX, would defeat the goal.
You could have stopped there, that's pretty much the only reason. But that has little to do with federation though. And yeah the ugliness and bad UX problems of free software - software that's freely unusable - have been known for literal decades.
You could have stopped there, that's pretty much the only reason. But that has little to do with federation though. And yeah the ugliness and bad UX problems of free software - software that's freely unusable - have been known for literal decades.
>> Why are giants attracting 95%+ of users?
Because they get free money from the Federal reserve through cheap loans and government contracts and money attracts people. They can use the money to saturate the media with their brand until everyone ends up using them. Then they can use the money to crush all potential competitors.
Because they get free money from the Federal reserve through cheap loans and government contracts and money attracts people. They can use the money to saturate the media with their brand until everyone ends up using them. Then they can use the money to crush all potential competitors.
Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder.
These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services.
So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.
These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services.
So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.
> Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder.
Email would beg to differ. Google and Microsoft may have a lot of users, but interoperability is astoundingly high.
Yes, there is a high barrier to sending emails from a new provider, and yes, Google/Microsoft have a large chunk of the market. Still, there's little barrier to using ProtonMail or Fastmail or whatever free host you want, and everything interoperates shockingly well. You don't typically have to think about "what if this person has gmail?" or "I don't know if I can send to that host from outlook".
Email would beg to differ. Google and Microsoft may have a lot of users, but interoperability is astoundingly high.
Yes, there is a high barrier to sending emails from a new provider, and yes, Google/Microsoft have a large chunk of the market. Still, there's little barrier to using ProtonMail or Fastmail or whatever free host you want, and everything interoperates shockingly well. You don't typically have to think about "what if this person has gmail?" or "I don't know if I can send to that host from outlook".
Well most people use one of a few big email providers, not small individual mail servers run by tech-savvy friends or family.
I don't think that is necessarily a bad thing, but I think for federated services to catch on, you need big players that provide reliable service and a recognizable brand. The average person is not going to want to search through thousands of small federated servers to try and find one that works for them. And running a single server might be simple and inexpensive, but making it reliable and secure is not.
I don't think that is necessarily a bad thing, but I think for federated services to catch on, you need big players that provide reliable service and a recognizable brand. The average person is not going to want to search through thousands of small federated servers to try and find one that works for them. And running a single server might be simple and inexpensive, but making it reliable and secure is not.
> Well most people use one of a few big email providers, not small individual mail servers run by tech-savvy friends or family.
That's just because installing and maintaining your own email server is extremely complicated [1]. Lots of people maintained their own Kazaa/Limewire servers for their friends (and the bigger piracy community) back in the day, because it was as simple as double clicking setup.exe and installing the software, and then configuring it.
Email does require higher reliability than Limewire. But I imagine if someone made a 100$ box that I could plug into my router, and so could 3 of my friends, and that had an email server installed on it or could be easily by clicking a installation file, and the 3 could act in concert to provide email services to the 50 people, maybe you would see more uptake.
[1] I am a tech-savy user who has been looking for a simple solution for a decade now.
That's just because installing and maintaining your own email server is extremely complicated [1]. Lots of people maintained their own Kazaa/Limewire servers for their friends (and the bigger piracy community) back in the day, because it was as simple as double clicking setup.exe and installing the software, and then configuring it.
Email does require higher reliability than Limewire. But I imagine if someone made a 100$ box that I could plug into my router, and so could 3 of my friends, and that had an email server installed on it or could be easily by clicking a installation file, and the 3 could act in concert to provide email services to the 50 people, maybe you would see more uptake.
[1] I am a tech-savy user who has been looking for a simple solution for a decade now.
> That's just because installing and maintaining your own email server is extremely complicated
It's not anymore [0]. However, good look on getting Microsoft's or other's mail servers to accept your e-mail. That's a big reason why even tech savvy users give up on hosting their own mail server. I've been running my own mail server for over two years now and the situation hasn't improved. My last problem was that a mail server from a government agency straight up refused the SMTP connection and contacting their postmaster mailbox yielded no response.
(For anyone wondering: It's not a problem with the configuration of my mail server, I've checked it about 10 times now. It passes all MxToolbox checks. However, I'm using an .xyz domain which might contribute to my problem.)
[0] mailcow.email
It's not anymore [0]. However, good look on getting Microsoft's or other's mail servers to accept your e-mail. That's a big reason why even tech savvy users give up on hosting their own mail server. I've been running my own mail server for over two years now and the situation hasn't improved. My last problem was that a mail server from a government agency straight up refused the SMTP connection and contacting their postmaster mailbox yielded no response.
(For anyone wondering: It's not a problem with the configuration of my mail server, I've checked it about 10 times now. It passes all MxToolbox checks. However, I'm using an .xyz domain which might contribute to my problem.)
[0] mailcow.email
Come on, this is the installation process [0]. If it's more complicated than apt/yum install mailcow, followed by a nice friendly browser based configuration page, it's too complicated for widespread usage.
This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.
[0] https://mailcow.github.io/mailcow-dockerized-docs/i_u_m_inst...
This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.
[0] https://mailcow.github.io/mailcow-dockerized-docs/i_u_m_inst...
> If it's more complicated than apt/yum install mailcow, followed by a nice friendly browser based configuration page, it's too complicated for widespread usage.
It's a bit more complicated than that but the simplest solution I know.
> This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.
This is unfortunately impossible. You can have a perfectly valid configuration including reverse DNS, SPF, DKIM and DMARC, yet Microsoft's email servers will still let your mails vanish.
It's a bit more complicated than that but the simplest solution I know.
> This includes, having out of the box configuration, that makes the email acceptable by Microsoft/Gmail etc.
This is unfortunately impossible. You can have a perfectly valid configuration including reverse DNS, SPF, DKIM and DMARC, yet Microsoft's email servers will still let your mails vanish.
> Well most people use one of a few big email providers, not small individual mail servers run by tech-savvy friends or family.
Yes, but importantly, those few big providers federate. That means that the barrier to entry for the little guys is much lower. This situation, while not ideal, is vastly better than the non-federated alternative.
Yes, but importantly, those few big providers federate. That means that the barrier to entry for the little guys is much lower. This situation, while not ideal, is vastly better than the non-federated alternative.
While they do federate, it's a challenge to get them to accept mails from your own mail server. Microsoft's mail servers are notorious for accepting your mails with a "250 Queued for delivery" and then letting them vanish in a black hole. I've also ran into problems with mail servers of government agencies.
So yes, while technically possible, self-hosting your mail server sadly isn't viable anymore for the little guy.
Email was already too big to fail by the time Facebook and Google got big enough to be able to kill it.
Looking at stuff created in the last 10-15 years, are there any federation successes? I can't think of any. (Blockchain excluded for the reason stated in the article)
Looking at stuff created in the last 10-15 years, are there any federation successes? I can't think of any. (Blockchain excluded for the reason stated in the article)
What's the definition of success? I spend far too much time on the Fediverse (mastodon) and if success is measured by the amount of time I spend there, it's the most successful service I use.
Of course, there are few if any political leaders or CEO's posting there, but that's not a definition of success to me.
I have thought about what would happen if someone like, say, Musk would proclaim that they're quitting Twitter and setting up a Mastodon instance instead. I'm not sure the Fediverse would be better off if that happened.
Of course, there are few if any political leaders or CEO's posting there, but that's not a definition of success to me.
I have thought about what would happen if someone like, say, Musk would proclaim that they're quitting Twitter and setting up a Mastodon instance instead. I'm not sure the Fediverse would be better off if that happened.
Wide adoption. Especially for social network type apps, which are rather useless without my friends in them.
To each their own, clearly. I don't think I ever joined a social network because my friends used it (possible exception of FB, but I'm not actually using that one. There could be correlation there).
I have met many new friends on all the networks that I have joined however.
I have met many new friends on all the networks that I have joined however.
Urbit is also not a federated system, but rather, a true distributed system.
Each urbit instance is a peer. The stars and galaxies provide discovery and initial routing, which is a pretty neat solution, they're basically fancy DNS servers.
What I think is exciting about urbit, is that each identity can host services. So you can launch a chat room, and it's yours, other peers can show up and say things in it, but this is very different from federation, where there is a sharp distinction between administrators and users.
Anyone who comments in your chat room can have a chat of their own, and the cryptographic identity of you and them is identical and consistent, wherever in the fleet you may encounter each other.
Stability and performance have improved, but there's a long way to go before a smooth one-click user experience is a reality.
But the basic design is solid, and it has a chance precisely because it isn't federated, but distributed. Secure Scuttlebutt is another truly distributed system, of very different design; it's locked in some bad architecture but I found it quite pleasant to interact with.
Don't expect anything useful from Holochain, ever. Just... I don't want to get into inside baseball, but, nah. Never gonna happen.
Each urbit instance is a peer. The stars and galaxies provide discovery and initial routing, which is a pretty neat solution, they're basically fancy DNS servers.
What I think is exciting about urbit, is that each identity can host services. So you can launch a chat room, and it's yours, other peers can show up and say things in it, but this is very different from federation, where there is a sharp distinction between administrators and users.
Anyone who comments in your chat room can have a chat of their own, and the cryptographic identity of you and them is identical and consistent, wherever in the fleet you may encounter each other.
Stability and performance have improved, but there's a long way to go before a smooth one-click user experience is a reality.
But the basic design is solid, and it has a chance precisely because it isn't federated, but distributed. Secure Scuttlebutt is another truly distributed system, of very different design; it's locked in some bad architecture but I found it quite pleasant to interact with.
Don't expect anything useful from Holochain, ever. Just... I don't want to get into inside baseball, but, nah. Never gonna happen.
Yeah, for me a federated system taken to its logical conclusion turns into p2p system.
SSB is neat, but ineed could use rearchitecting. I could imagine ssb-like system + handshake to work quite OK for certain things.
Thanks for info on Holochain. I've been only lurking in there and reading docs, so my understanding is rather shallow and theoretical.
SSB is neat, but ineed could use rearchitecting. I could imagine ssb-like system + handshake to work quite OK for certain things.
Thanks for info on Holochain. I've been only lurking in there and reading docs, so my understanding is rather shallow and theoretical.
The current shape of the internet itself (BGP peering among autonomous systems, a decent number of large tier 1 ISPs, etc) is, on its face, a relatively successful federation. The applications running atop that federation are, as you say, centralized, but much of that is because residential ISPs forbid customers forbid servers and even if they didn’t (they haven’t always) residential customers are prone to accidentally running botnets of various types.
> Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder.
Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.
Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.
Usenet died from a combination of web forums being a better experience for most people, and a saturation with warez binary traffic becoming so expensive that ISPs did not want to pay for it.
> So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying.
What about GNUnet? They’re trying to build pretty much what you described, from the ground up. https://gnunet.org/en/
What about GNUnet? They’re trying to build pretty much what you described, from the ground up. https://gnunet.org/en/
I'm aware of GNUnet, and been following it loosely for too long to remember. It's been around for like 2 decades?
Superficially it looks to me like bunch of theoreticians overcomplicated everything, and it will never be practical. Probably great project to advance your academic carrier though. But maybe I'm wrong.
Superficially it looks to me like bunch of theoreticians overcomplicated everything, and it will never be practical. Probably great project to advance your academic carrier though. But maybe I'm wrong.
I am not sure this is self-evident; it could be that there is no end state, but rather a cycle of centralized and federated systems, and we simply haven't seen enough of the cycle to identify it, yet. As a parallel, consider systems of government in the last 2500 years or so.
Could you share examples from the governance cycle?
I was thinking of the rise and fall of democracy from before the Roman empire and through the present day. Pick any time and look back only a few centuries, and the answer to "What is the natural end state of government?" looks very different. Zoom out a bit and it's clear that if there is an end state, we haven't reached it yet.
Please note: I am being very careful with the claims I am making here. I have a little more than a layperson's understanding of Rome's history (years of Latin classes paying off at last!) but not nearly enough to call myself an authority on governments through the ages.
My point -- my only[0] point -- is that "The Internet started off federated and has become centralized / nobody has gotten federation to stick yet" is not enough of an argument to support "federation is doomed to failure".
My comment does not totally disagree with its parent, which seems to think that alternative approaches may succeed. We certainly agree that it's important to learn from the past.
I'm having trouble wrapping this up, but don't want to spend more time, so I'll leave it on that somewhat unfinished note.
[0] edit: I suppose there is also a secondary point that systems design is, like governance, an inherently social issue. This makes any analysis a lot more complex than if we restrict ourselves to purely technical merits.
Please note: I am being very careful with the claims I am making here. I have a little more than a layperson's understanding of Rome's history (years of Latin classes paying off at last!) but not nearly enough to call myself an authority on governments through the ages.
My point -- my only[0] point -- is that "The Internet started off federated and has become centralized / nobody has gotten federation to stick yet" is not enough of an argument to support "federation is doomed to failure".
My comment does not totally disagree with its parent, which seems to think that alternative approaches may succeed. We certainly agree that it's important to learn from the past.
I'm having trouble wrapping this up, but don't want to spend more time, so I'll leave it on that somewhat unfinished note.
[0] edit: I suppose there is also a secondary point that systems design is, like governance, an inherently social issue. This makes any analysis a lot more complex than if we restrict ourselves to purely technical merits.
I generally agree. Though just to be clear: Rome was never considered a democracy.
If you want a classic example, some of the Greek city states might count. Though keep in mind that many of the Greeks considered sortition (ie random selection) the hallmark of the democracy, and considered that voting would just lead to aristocracy or oligarchy.
The German Free Imperial Cities are an interesting object of study. And there's lots to be learned from eg China as well.
If you want a classic example, some of the Greek city states might count. Though keep in mind that many of the Greeks considered sortition (ie random selection) the hallmark of the democracy, and considered that voting would just lead to aristocracy or oligarchy.
The German Free Imperial Cities are an interesting object of study. And there's lots to be learned from eg China as well.
> Though just to be clear: Rome was never considered a democracy.
I did intentionally avoid clarifying this (also, technically the US is a republic) because I was trying to paint in broad strokes — I don't think my point needed details to be convincing and so I didn't want to get bogged down in them. I'm happy for the clarification as a reply, though :) A better phrase, which I would have used if I'd thought of it at the time, would be "representative government".
I did intentionally avoid clarifying this (also, technically the US is a republic) because I was trying to paint in broad strokes — I don't think my point needed details to be convincing and so I didn't want to get bogged down in them. I'm happy for the clarification as a reply, though :) A better phrase, which I would have used if I'd thought of it at the time, would be "representative government".
> considered that voting would just lead to aristocracy or oligarchy.
Well, looking at the US at least, they weren't wrong.
Well, looking at the US at least, they weren't wrong.
Perhaps. I actually think some variant of sortition is superior to most voting systems to eg fill up a parliament; but to take the US as an example to argue that voting is bad would make the job too easy.
Eg the German electoral system is also a bit messy, but seems to work well in practice. That might be a better yardstick to measure what voting can do, before we dismiss it.
(My favourite sortition variant right now:
- everyone can register as a candidate, if they want to - every voter can pick one candidate to put on their ballot - we select a few hundred ballots at random to fill up a parliament
For completeness sake, add some mechanism to handle a single candidate being selected by more than one randomly drawn ballot. Eg you could draw again, or you could give that candidate extra weight in parliamentary votes, or you could form a chain and add a person of the double-candidates choice to parliament, or you could have voters write down multiple candidates in order, and just pick a later one on the voters list etc.
Parliament can then select the executive, like it is already done in the UK or in Germany.)
Eg the German electoral system is also a bit messy, but seems to work well in practice. That might be a better yardstick to measure what voting can do, before we dismiss it.
(My favourite sortition variant right now:
- everyone can register as a candidate, if they want to - every voter can pick one candidate to put on their ballot - we select a few hundred ballots at random to fill up a parliament
For completeness sake, add some mechanism to handle a single candidate being selected by more than one randomly drawn ballot. Eg you could draw again, or you could give that candidate extra weight in parliamentary votes, or you could form a chain and add a person of the double-candidates choice to parliament, or you could have voters write down multiple candidates in order, and just pick a later one on the voters list etc.
Parliament can then select the executive, like it is already done in the UK or in Germany.)
My post was a little tongue in cheek. I don't think a voting sysyem necessarily leads to an aristocracy/oligarchy. But it certainly can, and it needs to be carefully crafted to avoid that.
Well, voting leads to specialisation of labour, and thus a political caste.
The sortition system I sketched would still reward specialisation of labour, but there would be fewer threshold effects: if you convince thirty friends friends to ballot for you, you'd have a ten times higher chance to make it into parliament compared to only convincing three friends.
With voting you have threshold effects. In all but the smallest towns, getting thirty people to vote for you has exactly the same outcome as zero votes.
The sortition system I sketched would still reward specialisation of labour, but there would be fewer threshold effects: if you convince thirty friends friends to ballot for you, you'd have a ten times higher chance to make it into parliament compared to only convincing three friends.
With voting you have threshold effects. In all but the smallest towns, getting thirty people to vote for you has exactly the same outcome as zero votes.
A cycle of democracies / republics / authoritarian regimes? A cycle of smaller independent states / large empires?
This has repeated a lot in the history of the western civilization.
This has repeated a lot in the history of the western civilization.
E-mail doesn't work too? Federation works, if some walled garden obsessive lock-in freaks (like Compuserve and AOL of old or today's analogies of Facebook and the like) don't manage to mess things up.
Email sucks.
Spam is ramptant. Running a server is a nightmare.
Standards are a tar pit.
It "works" only for fairly loose definitions of "works".
Spam is ramptant. Running a server is a nightmare.
Standards are a tar pit.
It "works" only for fairly loose definitions of "works".
Have you tried to setup a mail server these days? There's plenty of nearly turn key solutions. Even from scratch on Debian, Fedora, or Ubuntu is pretty easy these days, a few apt-gets and a few additional tweaks and away you go.
Using spam-assassin, clamav, and a few recommended tweaks (greylisting, HELO rules, blocking .ads, .party, and friends) and I'm down near zero spam.
Using spam-assassin, clamav, and a few recommended tweaks (greylisting, HELO rules, blocking .ads, .party, and friends) and I'm down near zero spam.
How's your delivery? Anything that runs the risk of my mail getting blackholed by a clients provider is a nonstarter (even if it's only a few days every few years).
Been fine, I have noticed others with terrible results. My best guess is that large email provides use domain age. So new domains have terrible problems with delivery. My domain is many years old.
Still, it works.
Centralized servers are just as hard to run, if not harder. You just don't experience those ops yourself without working there.
Email is totally feature frozen as well. There is infinite room to improve email but it will never happen because it requires the participation of too many parties. With centralized services there is always someone who has the final say and can force change.
As an example, we still haven't solved the wrapping problem that's been known for 40 years and has a fairly simple solution.
Is this the line length limit you are talking about?
Well, there isn't really any limit as such, but yes.
Sure. But it's still far better than any of the alternatives.
If these problems are so obviously ineluctable, why talk around them instead of detailing or even naming any?
You mean, other than Mastodon, which has a few million users and growing.
Mastodon is proof at least that federation doesn't fix the issue of censorship. There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster.
And if your not part of the main group, what's even the point of federation? It's practically "servers" at that point.
And if your not part of the main group, what's even the point of federation? It's practically "servers" at that point.
It does fix the issue of censorship, in that 'blacklisted' instances can still have a world of their own and use the technology or protocol.
No amount of technology is going to fix the 'a majority of people don't like you or interact with you' problem though.
No amount of technology is going to fix the 'a majority of people don't like you or interact with you' problem though.
> Mastodon is proof at least that federation doesn't fix the issue of censorship. There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster.
That's nonsense.
I run my own instance and don't use any sort of ban list, and I can interact just fine with both the "main" instances, and a whole host of smaller, more niche, instances.
That's nonsense.
I run my own instance and don't use any sort of ban list, and I can interact just fine with both the "main" instances, and a whole host of smaller, more niche, instances.
Well I'm really glad that that trusted circle of instances exists, I don't want anti-Semitism or racism in my timeline. So far octodon.social has been a step up from twitter for me, because it isn't designed to monetize outrage. The model where admins have to police their community or risk being de-federated with the mainstream means that an instance is limited to the scale they can effectively moderate, which is exactly where other social media platforms break for me.
So basically Mastodon is moderated in exactly the same way as Twitter, with the same set of rules. How is Mastodon more interesting than Twitter then?
the only way you can imagine something like Twitter being more interesting is more content Twitter prohibits?
it's more interesting because the UX encourages different styles of posting (no QRTs for dunks, for one) that people are developing into slightly different social idioms. also no ads, chrono timeline, pick your own image crops...
it's more interesting because the UX encourages different styles of posting (no QRTs for dunks, for one) that people are developing into slightly different social idioms. also no ads, chrono timeline, pick your own image crops...
>There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster.
I run my own instance. As far as I can tell, this is false. There are some ban lists floating around, though I'm yet to see a common, standard policy of blocking instances because they don't subscribe to a particular blocklist.
I run my own instance. As far as I can tell, this is false. There are some ban lists floating around, though I'm yet to see a common, standard policy of blocking instances because they don't subscribe to a particular blocklist.
My grandma will never use Mastodon, but she uses Facebook. Tech isn’t the only audience you need.
I think tech is a tiny minority on Mastodon. From what I can see there are far more queer folk and lefty ppl. Your grandma will never use Mastodon because there are no network effects luring her there, but the UX is about as good as a normal centralized social site.
sorry, but your grandmother doesn't have a few hundred thousand to a few million followers.
media organizations, pols, and public sector institutions do. Your grandma doesn't need to use Mastodon and it's not about her. It's about those who will be followed and have something important to transmit.
media organizations, pols, and public sector institutions do. Your grandma doesn't need to use Mastodon and it's not about her. It's about those who will be followed and have something important to transmit.
> Federated systems just don't work in practice because of inherent problems with them
The entire form of current global human society is federated so you can't simply state it does not work. For the internet so far we have a few empires that battle it out, but I don't think they are manageable long term and this has started to become apparent.
The entire form of current global human society is federated so you can't simply state it does not work. For the internet so far we have a few empires that battle it out, but I don't think they are manageable long term and this has started to become apparent.
> because of inherent problems with them that don't have good technical&social solutions yet
Would you care to mention them?
Would you care to mention them?
[deleted]
You would more likely need a new form of people.
Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda.
For the same reason, there won't be a federated social network...
Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.
Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda.
For the same reason, there won't be a federated social network...
Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.
For those who dislike federated services, keep in mind: email is a federated system, and it remains arguably one of the largest social networks in the world.
Email's federated nature has kept it alive for decades, and will ensure that it long outlives more proprietary forms of social networking.
Federated systems already work in practice and at scale.
edit: Some other examples with widespread adoption:
- The phone network (PSTN)
- The internet (via BGP)
- The postal service (federated between countries via the UPU)
- Podcasts (and RSS more generally)
Am I missing any?
Email's federated nature has kept it alive for decades, and will ensure that it long outlives more proprietary forms of social networking.
Federated systems already work in practice and at scale.
edit: Some other examples with widespread adoption:
- The phone network (PSTN)
- The internet (via BGP)
- The postal service (federated between countries via the UPU)
- Podcasts (and RSS more generally)
Am I missing any?
Unfortunately 2 of those I almost entirely get spam and business/commercial interactions through. On my cell phone I've gotten one call in the last 6 months that wasn't spam and it was an automated notification from the Sheriff about a permit I'd applied for. I haven't gotten a single useful call from someone not on the whitelist of my contact page in so long it's tempting to just autosend all unrecognized numbers to voicemail.
Email in particular I don't think really fits the mold of social network. I think principally because there's not really a public post concept it's all essentially point to point communications. There are some abstractions like mailing lists but they're still an express list of people who will get it. I guess that might not be so different from a 100% private profile but the lack of any public functionality excludes it from the social media category.
Email in particular I don't think really fits the mold of social network. I think principally because there's not really a public post concept it's all essentially point to point communications. There are some abstractions like mailing lists but they're still an express list of people who will get it. I guess that might not be so different from a 100% private profile but the lack of any public functionality excludes it from the social media category.
What about SMS? I don't know where you live, but here in America everyone I know uses it.
The use of SMS is a US oddity. The rest of the world is using almost only messenger apps: Line or WeChat if you’re in Asia, WhatsApp or telegram in Europe or South America.
euro here, used sms heavily until the ott messenger craze set in my cicles (~2015).
nowadays mostly for contacts that either dont use a smartphone or dont have mobile internet connection (yes! those exist).
are u sure u are using sms? asking because apple overlays imessage and iphone penetration is said to be highest in usa.
nowadays mostly for contacts that either dont use a smartphone or dont have mobile internet connection (yes! those exist).
are u sure u are using sms? asking because apple overlays imessage and iphone penetration is said to be highest in usa.
In Indonesia the only SMS I get is spam. It's so annoying that I just disable notifications for SMS.
I largely only text my partner and that's through Whatsapp. My friend group has a Discord we all use.
> - Podcasts (and RSS more generally)
Podcasts exactly exhibit what I fear for with federated systems. As podcasters move to "Spotify exclusive", Spotify still calling those podcasts, and those podcasts not being available through your general podcast subscription app, could lead to most podcasts being de facto centralised again.
The same could happen to the Fediverse: some player adds some feature that makes large swaths of people migrate there, those features are not interoperable, and then that service turns off federation altogether...
Podcasts exactly exhibit what I fear for with federated systems. As podcasters move to "Spotify exclusive", Spotify still calling those podcasts, and those podcasts not being available through your general podcast subscription app, could lead to most podcasts being de facto centralised again.
The same could happen to the Fediverse: some player adds some feature that makes large swaths of people migrate there, those features are not interoperable, and then that service turns off federation altogether...
Personally email is a bad comparison in my opinion. The way it is federated makes easy to fossilize the protocol and the "social" nature is at odds with the federation due to the spam problem.
And in general I consider it a terrible method of communication.
And in general I consider it a terrible method of communication.
> arguably one of the largest social networks in the world
At the same time, I know several people who gave up their email addresses and only use Facebook now. Maybe the next iteration of social media behemoth will be even more effective at that.
At the same time, I know several people who gave up their email addresses and only use Facebook now. Maybe the next iteration of social media behemoth will be even more effective at that.
How do they reset their Facebook password?
Technically they still have an email account, so I imagine that would be how. But they don't otherwise ever use the account, never even check it.
Though to be honest I expect they just use their phone if they need to reset their password.
Though to be honest I expect they just use their phone if they need to reset their password.
> [email] remains arguably one of the largest social networks in the world.
What percentage of email traffic do you think is social in nature (and not just notifications from an actual, separate social network)? I'm guessing <0.1%.
What percentage of email traffic do you think is social in nature (and not just notifications from an actual, separate social network)? I'm guessing <0.1%.
Federated and decentralized solutions have significant UX issues compared to leading products in the area. They're not only harder to use and slower, but are much worse at converting and keeping users to begin with, since they're not ran like businesses are, which take in more funding and try hard to optimize all of these user metrics as KPI. It will be a long time before this can change I think, users still do not (and won't) care if what they're using is federated or not. I say this as someone who loves the idea of federated platforms, but we cannot truly succeed until we have a way to convince large amounts of (normal) users to give them a try.
I agree about your UX notes, but I disagree with the success criteria. I've wholly quit social media apart from the mastodon instance I'm on. There are enough quality people I'm connected with now that it's a success for me.
Am I missing out on content from Facebook/Twitter/Instagram because I'm not connected indirectly to everyone in their gardens? Sure, but I'd argue it's low quality content that was only a drain on my mental health.
I guess I'm trying to say, I think it's about the individual success and not the % of market share.
Am I missing out on content from Facebook/Twitter/Instagram because I'm not connected indirectly to everyone in their gardens? Sure, but I'd argue it's low quality content that was only a drain on my mental health.
I guess I'm trying to say, I think it's about the individual success and not the % of market share.
I've enjoyed them myself too, and I still use things like IRC on a daily basis. The two impluses I see to get federation to a larger market share are A) to 'save' people from the platforms they're currently stuck on, but also B) get rid of, or at least diminish, their network effects.
You could argue maybe A) won't work or isn't possible or needed, but I feel B) myself a lot. For example I meet someone and want to talk to them, but it's likely they might suggest we chat via Skype or Discord or something like that, and if I don't want to use that service I'm often out of luck.
That, personally, is what bothers me the most - that network effects seem inherently anti-competitive because they force me to use a platform I wouldn't otherwise want to use, only because they purposefully make their software unable to communicate with competing platforms.
For all of the open protocols and specifications we have from HTTP to TLS to HTML and literally one thousand others, we just re-use them over and over to create closed systems that can't talk to each other, which feels like going backwards. Sure it's great all of our computers speak TCP, UDP, TLS, HTTP, and so on. But they don't all speak iMessage, Zoom, Skype, Discord, Reddit, Twitter, Mastdodon, Matrix, Slack, etc etc. I still feel constantly locked-in to all of these platforms just when I want to communicate to someone that happens to be on it, and the area is certainly not progressing because the fight to gain this huge market share of users is never-ending.
Ideally, that is what I would love to end via federation. If I could actually use some type of FOSS and open protocol, and actually talk to any other person I wanted to with it, and I wasn't forced to use a certain platform, service, company, and give them all of my data in the process. I don't see any way for this to be achieved, but I can always dream.
You could argue maybe A) won't work or isn't possible or needed, but I feel B) myself a lot. For example I meet someone and want to talk to them, but it's likely they might suggest we chat via Skype or Discord or something like that, and if I don't want to use that service I'm often out of luck.
That, personally, is what bothers me the most - that network effects seem inherently anti-competitive because they force me to use a platform I wouldn't otherwise want to use, only because they purposefully make their software unable to communicate with competing platforms.
For all of the open protocols and specifications we have from HTTP to TLS to HTML and literally one thousand others, we just re-use them over and over to create closed systems that can't talk to each other, which feels like going backwards. Sure it's great all of our computers speak TCP, UDP, TLS, HTTP, and so on. But they don't all speak iMessage, Zoom, Skype, Discord, Reddit, Twitter, Mastdodon, Matrix, Slack, etc etc. I still feel constantly locked-in to all of these platforms just when I want to communicate to someone that happens to be on it, and the area is certainly not progressing because the fight to gain this huge market share of users is never-ending.
Ideally, that is what I would love to end via federation. If I could actually use some type of FOSS and open protocol, and actually talk to any other person I wanted to with it, and I wasn't forced to use a certain platform, service, company, and give them all of my data in the process. I don't see any way for this to be achieved, but I can always dream.
It reminds me of a rant by an anime youtuber / analyst, who complained that various video streaming platforms (amazon, netflix, crunchyroll, ...) are not innovative in design, but their only selling point is their closed catalogues. And. It. Drove. Him. Mad.
Okay, so you should subscribe to 5 services to see all the latest anime you should see. You want to support the artists. But, some of them have horrible experiences, and you are locked into it because you want to _legally_ watch this show.
It's that if your platform is exclusive enough, you also become very slow to innovate.
We've seen time and again that most innovation comes from constrained settings. But big corps just want to maximize money, they don't care about improvements. That is IMHO the block stopping us from actually tackling a lot of identified and solveable problems we face this century.
Okay, so you should subscribe to 5 services to see all the latest anime you should see. You want to support the artists. But, some of them have horrible experiences, and you are locked into it because you want to _legally_ watch this show.
It's that if your platform is exclusive enough, you also become very slow to innovate.
We've seen time and again that most innovation comes from constrained settings. But big corps just want to maximize money, they don't care about improvements. That is IMHO the block stopping us from actually tackling a lot of identified and solveable problems we face this century.
I think you just described email.
My favorite video game is Halo 3. It never died, I just left.
My favorite video game is Halo 3. It never died, I just left.
KPIs that are being optimized result in increasingly user-hostile products, like Facebook and Twitter.
There certainly are UX problems that the current federated social media projects have. They're not technically unsolvable, it's just that current federated social media projects don't want to solve them for some reason. That's what I'm trying to address with my project (that's not yet ready to be demonstrated, sorry).
The primary problem that the current federated social media projects are facing is that there's no global user search. At all. You sign up for a Mastodon account on the instance of your liking, and now what? Sure you can follow Eugen and like the cat pictures he reposts. Cats are cute. But how do you find people you already know without asking every one of them whether they have a Mastodon account? You don't. And that's how you not build a useful social media service. Every single social media service that has had any semblance of success offered an easy way to bootstrap your social graph — be it a detailed global search feature and rich profiles to go with it or the ability to import your friend/contact list from somewhere else. That alone should be telling.
There are technical solutions to this, like making a DHT out of instance servers that could be used to link the user IDs on centralized platforms to ActivityPub URLs. Even if said centralized platforms refuse API access to instance admins, there's still machine-readable GDPR data export to take advantage of. It can be done. It's just going to take some time.
There certainly are UX problems that the current federated social media projects have. They're not technically unsolvable, it's just that current federated social media projects don't want to solve them for some reason. That's what I'm trying to address with my project (that's not yet ready to be demonstrated, sorry).
The primary problem that the current federated social media projects are facing is that there's no global user search. At all. You sign up for a Mastodon account on the instance of your liking, and now what? Sure you can follow Eugen and like the cat pictures he reposts. Cats are cute. But how do you find people you already know without asking every one of them whether they have a Mastodon account? You don't. And that's how you not build a useful social media service. Every single social media service that has had any semblance of success offered an easy way to bootstrap your social graph — be it a detailed global search feature and rich profiles to go with it or the ability to import your friend/contact list from somewhere else. That alone should be telling.
There are technical solutions to this, like making a DHT out of instance servers that could be used to link the user IDs on centralized platforms to ActivityPub URLs. Even if said centralized platforms refuse API access to instance admins, there's still machine-readable GDPR data export to take advantage of. It can be done. It's just going to take some time.
The problems are big. What are the plans for your not-yet-ready project? Are they purely technical like what you stated in the final paragraph?
To me they're more technical than not. I've worked at VK for 5 years, so I think I know a thing or two about building a competitive, easy to use, and user-respecting social media service. It's also somewhat of a curse because I keep seeing all the flaws other fediverse projects have but refuse to address.
[deleted]
> "This [federation] system also makes it hard for marketing and spam to get a foothold"
> "You’re certainly familiar with another federation which is not based on ActivityPub: email."
Self-contradicting article?
> "You’re certainly familiar with another federation which is not based on ActivityPub: email."
Self-contradicting article?
Email is particularly broken regarding spam for at least two reasons:
- it's a distributed system and not really a federated one: you can set up your own SMTP agent and send mails to anyone without prior registration to a federation.
- anyone can impersonate anyone: if I want to send an email pretending I'm [email protected] there's no built-in mechanism to prevents that.
Theses aren't fundamental features of federations, and came from a design which just overlooked how many bad actors email would attract when it became mainstream.
- it's a distributed system and not really a federated one: you can set up your own SMTP agent and send mails to anyone without prior registration to a federation.
- anyone can impersonate anyone: if I want to send an email pretending I'm [email protected] there's no built-in mechanism to prevents that.
Theses aren't fundamental features of federations, and came from a design which just overlooked how many bad actors email would attract when it became mainstream.
> it's a distributed system and not really a federated one: you can set up your own SMTP agent and send mails to anyone without prior registration to a federation.
One could say publishing an MX record on your domain is that registration.
One could say publishing an MX record on your domain is that registration.
You can set up your own Mastodon or Matrix server and send messages, I am not sure which federation you're thinking of that requires "registration".
The way the impersonation problem is dealt with in ActivityPub is very similar to DKIM in the SMTP world: the sender uses a cryptographic signature (HTTP signature) to prove that he is allowed to send messages from that actor. Of course this doesn't prevent someone from sending spam from their domain, in either case.
The way the impersonation problem is dealt with in ActivityPub is very similar to DKIM in the SMTP world: the sender uses a cryptographic signature (HTTP signature) to prove that he is allowed to send messages from that actor. Of course this doesn't prevent someone from sending spam from their domain, in either case.
> I am not sure which federation you're thinking of that requires "registration".
Which makes me wonder, how come we call these systems "federated" in the first place?
Federations I'm familiar with feature a central authority composed of representatives from the member states. That authority controls admissions and membership, and usually has some level of additional authority over members, along with capability to enforce it. None of that is present in these "federated" systems.
Which makes me wonder, how come we call these systems "federated" in the first place?
Federations I'm familiar with feature a central authority composed of representatives from the member states. That authority controls admissions and membership, and usually has some level of additional authority over members, along with capability to enforce it. None of that is present in these "federated" systems.
The "central authority" is the working group which authors the protocol specs, similar to how a federation of states' central authority would edict laws.
There is actually some debate as to whether it's really a federation if the central authority can override member states (as per https://en.wikipedia.org/wiki/Federation)
There is actually some debate as to whether it's really a federation if the central authority can override member states (as per https://en.wikipedia.org/wiki/Federation)
Since many federated systems use DNS, technically ICANN is the central authority.
But looking at how the terminology is used in practice, "federated" refers to systems with N users per host while "P2P" describes systems with one user per host.
But looking at how the terminology is used in practice, "federated" refers to systems with N users per host while "P2P" describes systems with one user per host.
You can fight spam by refusing to federate with servers which are often spamming. That acts as a deterrent for servers to host spam since they'll lose participation. However, this only works if there are enough of other servers to federate with so that you can actually choose your peers.
The status quo with email is not like that, unfortunately. Either you federate with GMail and accept the possible spam, or you disrupt significant part of communications of your users by refusing to accept email from Google's mail servers, with Google not losing basically anything (and having no intrinsic reason to fight spam). So it becomes a loss-meh situation instead of win-loss.
The status quo with email is not like that, unfortunately. Either you federate with GMail and accept the possible spam, or you disrupt significant part of communications of your users by refusing to accept email from Google's mail servers, with Google not losing basically anything (and having no intrinsic reason to fight spam). So it becomes a loss-meh situation instead of win-loss.
>You can fight spam by refusing to federate with servers which are often spamming
Awesome, so users will naturally gravitate to the servers that spent the most $$$ money fighting spam. You end up with Gmail all over again.
I don't know why you are using Gmail as the spammer in this situation. Arguably the 2nd biggest reason Gmail got so big is at the time their spam detection was lightyears ahead of everyone else. Then as they got larger they killed off smaller servers that didn't do the magic song and dance, and those users couldn't talk to most people as everyone was on Gmail. There's nothing stopping this from happening again on some new federation platform.
This is my biggest gripe with federated services. They all start from the point where "the Internet is centralized" and offer some proto-Internet to undo it. The Internet started federalized and the easiest solution to many of the problems that came with it was centralization. It's always unclear how you prevent this from happening again.
Awesome, so users will naturally gravitate to the servers that spent the most $$$ money fighting spam. You end up with Gmail all over again.
I don't know why you are using Gmail as the spammer in this situation. Arguably the 2nd biggest reason Gmail got so big is at the time their spam detection was lightyears ahead of everyone else. Then as they got larger they killed off smaller servers that didn't do the magic song and dance, and those users couldn't talk to most people as everyone was on Gmail. There's nothing stopping this from happening again on some new federation platform.
This is my biggest gripe with federated services. They all start from the point where "the Internet is centralized" and offer some proto-Internet to undo it. The Internet started federalized and the easiest solution to many of the problems that came with it was centralization. It's always unclear how you prevent this from happening again.
users will naturally gravitate to the servers that spent the most $$$ money fighting spam
Or hopefully all servers have good enough open-source anti-spam built in so that it isn't a centralizing force.
Or hopefully all servers have good enough open-source anti-spam built in so that it isn't a centralizing force.
From what I've seen anti spam works almost entirely by two methods. Only accepting messages from trusted servers (which rules out having too many small servers), and information asymmetry where the spam artists don't know what signals you are looking for (which rules out open source).
> information asymmetry where the spam artists don't know what signals you are looking for (which rules out open source).
Machine learning mitigates this somewhat - doesn't really matter if the code's open source if even the people who wrote it have no idea what's actually happening in the black box.
Of course, that method of anti-spam has an entirely separate set of problems. But we've got those either way.
Machine learning mitigates this somewhat - doesn't really matter if the code's open source if even the people who wrote it have no idea what's actually happening in the black box.
Of course, that method of anti-spam has an entirely separate set of problems. But we've got those either way.
If the model weights are available you have the same problem.
If the model weights aren't available you won't get many instances of the server running.
If the model weights aren't available you won't get many instances of the server running.
What about open-source infrastructure with a closed-source antispam plug-in?
Closed source probably isn't enough, you probably need to not widely distribute the binary either. At which point you can't have many server instances to federate.
Almost - intrinsic to offering modern webmail, (and not simply selling SMTP as a service) is fighting spam. Empowered users aren't going to put up with receiving spam and can and will go elsewhere.
Spam is a problem on centralized social networks also--and they have advertisements, which are really just well-organized spam.
To the extent that this problem is email-specific, the ultimate root cause is a failure to deploy DMARC. Any new federated system will require a DMARC/SPF-like mechanism for preventing request forgery.
To the extent that this problem is email-specific, the ultimate root cause is a failure to deploy DMARC. Any new federated system will require a DMARC/SPF-like mechanism for preventing request forgery.
DMARC can effectively eliminate email spoofing. Unfortunately, that doesn't prevent spam, phishing, or malware spreading through email. In part because email addresses are largely hidden. In part because not a whole lot of people would notice if their scam mail come from [email protected] instead of from their bank.
Or a federated system that was designed before anyone imagined spam could be a problem, and thus can only deal with it in a backwards-compatible way by piling on ex post facto hacks. There's no reason to assume the designs of today will be informed by the same naïveté.
Email is a bit different from the new federations we're seeing today: there's less weaker guarantees of identity, and there's less emphasis on choosing what other instances you federate with. In the example of Mastodon, the cost of setting up a new server is also too high to make spam effective, since you need a persistent, reachable account in order to interact with other instances on behalf of it. And the social factor counts, too, newer federations like this tend to be opposed to corporate influence in general and this makes it hard for it to get the initial foothold.
Email has a lot of spam, but a federated system which keeps this in mind from the outset needn't suffer the same fate.
EDIT: Incorporating the replies to my comment, I also want to mention that the lack of large, centralized players like gmail also prevents spam. If there are a lot of intances, and they're all small, then cutting one off for bad behavior is a lot less difficult. Instance admins can be held accountable for the behavior of their users.
Email has a lot of spam, but a federated system which keeps this in mind from the outset needn't suffer the same fate.
EDIT: Incorporating the replies to my comment, I also want to mention that the lack of large, centralized players like gmail also prevents spam. If there are a lot of intances, and they're all small, then cutting one off for bad behavior is a lot less difficult. Instance admins can be held accountable for the behavior of their users.
That would imply that centralized solutions would have even less spam, since they have much stronger guarantees of identity. And yet sites like Twitter has plenty of spam. Even Hacker News gets spam! And this is not a popular place in the grand scheme of things.
How is distributed identity solved in the long term? Nobody would run a server to communicate spam; that's rarely how email spam is sent these days. Most of the spam I get is from accounts hosted on GMail and Yahoo!. So the weakest link is a single server that's community-respected, but doesn't have adequate spam protection. Federation makes that much harder, because the whole point is that every server is run separately.
Mastodon gets no spam today because it's smaller than Hacker News. There's relatively nobody on there.
How is distributed identity solved in the long term? Nobody would run a server to communicate spam; that's rarely how email spam is sent these days. Most of the spam I get is from accounts hosted on GMail and Yahoo!. So the weakest link is a single server that's community-respected, but doesn't have adequate spam protection. Federation makes that much harder, because the whole point is that every server is run separately.
Mastodon gets no spam today because it's smaller than Hacker News. There's relatively nobody on there.
> How is distributed identity solved in the long term?
Same way email handles it -- through the use of the domain name system. You'd expect that an a federated server with the address "https://mastodon.mit.edu/" would consist of those with a tie to the Massachusetts Institute of Technology (faculty or students). Your identity is provided by the system you are a member of and handled by your specific system administrator.
Same way email handles it -- through the use of the domain name system. You'd expect that an a federated server with the address "https://mastodon.mit.edu/" would consist of those with a tie to the Massachusetts Institute of Technology (faculty or students). Your identity is provided by the system you are a member of and handled by your specific system administrator.
That doesn't solve the spam problem, though? The issue from spam is that bad actors can sign up for any number of free-standing servers, without being blocked from the service as a whole. Blocking whole servers isn't the solution; that just leads to a weird scenario where you can't reach some of your contacts because they lucked out and picked the wrong server when signing up.
there's a lot of talk about object capabilities as a model to solve for this in the AP space. A lot of it goes over my head so I can't really go into details on it but there are some conference talks available where this is a topic.
background info:
https://github.com/WebOfTrustInfo/rwot9-prague/blob/master/t...
https://gitlab.com/spritely/ocappub
background info:
https://github.com/WebOfTrustInfo/rwot9-prague/blob/master/t...
https://gitlab.com/spritely/ocappub
I'm worried how mastodon is going to scale (in terms of moderation) - maybe some instance can experient with crowd sourced moderation.
Part of the point, as I understand it, is that moderation scales more-or-less linearly because each instance is ideally human-scale (that is, maybe having hundreds of members as opposed to millions) and responsible for its own moderation, unlike sites like Twitter or Facebook which have to pay a relatively small team of moderators to moderate millions of people's communications.
There's nothing saying any particular instance will moderate, but if they don't and it becomes a problem they'll quickly find themselves cut off from federating with other instances.
There's nothing saying any particular instance will moderate, but if they don't and it becomes a problem they'll quickly find themselves cut off from federating with other instances.
Email suffers from one of it's early defining operational principals which was to be backwards compatible forever. In this way, operators still have to decide if they should accept federation with weakly identified entities or close the door and federate only with high reputation sources. To add to that, some forms of it's adoption are difficult to work around with normal methods. Mailing lists are tough, since they spoof the senders address and break dkim. And once again, the decision between allowing the old way, and just blocking anyone who doesn't have proper dkim.
We see the complaints about this every time someone posts about an email server. It's hard to get the big guys to play ball. On the other hand, every time email comes up, people complain that there's too much spam.
We see the complaints about this every time someone posts about an email server. It's hard to get the big guys to play ball. On the other hand, every time email comes up, people complain that there's too much spam.
A server operator can blacklist domains if they want, preventing sending or receiving from spamsalot.com and can do so for whatever reason, including if they feel spamsalot.com is too weakly (or strongly) federated for that operator.
They can, but domains are easy to spoof, which leads us to operators having to decide to accept unsigned domains and risk bad actors getting through to the filters and milters or rejecting well meaning but unsigned email servers into federation.
That's a problem that can be fixed from the get go if you design a federated system that doesn't assume all actors are charitable. The major players could cut down on a huge amount of spam by only accepting signed domains, but email wasn't designed to force all email through only it's home server, so that's not as backwards compatible as you'd like.
That's not however a strict federation problem, it's more of an 'email was designed to assume everyone is a good actor, before we had much proof to the contrary' problem.
That's a problem that can be fixed from the get go if you design a federated system that doesn't assume all actors are charitable. The major players could cut down on a huge amount of spam by only accepting signed domains, but email wasn't designed to force all email through only it's home server, so that's not as backwards compatible as you'd like.
That's not however a strict federation problem, it's more of an 'email was designed to assume everyone is a good actor, before we had much proof to the contrary' problem.
Email is one to one mostly. So reputation should really also be based on individual entities. The normal way to make that possible is to have email clients sign messages by default with the senders identity of choice.
The technology exists in the current system. We just need to activate it. This isn't a technical problem.
The technology exists in the current system. We just need to activate it. This isn't a technical problem.
note that you do not need a full-blown Mastodon server to participate in the AP ecosystem. There are very little barriers to entry here - one is being able to process dynamic scripts and the other is a registered domain name.
I think the lowest cost solution to this would be a WordPress site on a shared host somewhere, with this plugin installed:
https://wordpress.org/plugins/activitypub/
I think the lowest cost solution to this would be a WordPress site on a shared host somewhere, with this plugin installed:
https://wordpress.org/plugins/activitypub/
Heh.
Honestly, I'd like to see more yammering about federation by people aware of the early federated systems of the internet and some perspective on their uses and failings.
Or, better, less yammering and more realizing. Preferably more so than small networks where Nazi Twitter is the biggest node.
Honestly, I'd like to see more yammering about federation by people aware of the early federated systems of the internet and some perspective on their uses and failings.
Or, better, less yammering and more realizing. Preferably more so than small networks where Nazi Twitter is the biggest node.
The biggest challenge I see with federated systems is the discovery process for finding a node you want to be associated with. This is a hard choice - will this server still be running in 6 months? How is it funded? Is it being actively maintained?
These questions are too much for laypeople, and I can't imagine a scenario in which your friendly local system admin is your service provider for a critical service in your life like Facebook. Could you imagine email working that way?
I see federated systems naturally ending in centralization in the form of a couple big megacorps that the average person trusts to not shut down tomorrow. Convenience and ease of use will win out. It happened with email, it arguably happened with xmpp back in the day. Change my view?
These questions are too much for laypeople, and I can't imagine a scenario in which your friendly local system admin is your service provider for a critical service in your life like Facebook. Could you imagine email working that way?
I see federated systems naturally ending in centralization in the form of a couple big megacorps that the average person trusts to not shut down tomorrow. Convenience and ease of use will win out. It happened with email, it arguably happened with xmpp back in the day. Change my view?
There's no evidence of this happening with, say, Mastodon. The culture is different, and isn't changing. If we know what we want and what we don't want from the start, we can keep centralization from gaining a foothold.
You can migrate a Mastodon account and the old server will forward on your follows.
Strangely I posed a question to HN just a little earlier about the future of sofware - https://news.ycombinator.com/item?id=24547534 - and you've answered that question as I was expecting. I have been formulating similar theories and ideas but it's unclear how feasible it is without a collective willing to take it the distance.
For me it starts with the basis of a communication protocol for services https://github.com/asim/mucp. And then moves onto something on a broader scale https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d3216.... I don't have all the answers yet, but I feel something is coming. An answer to our future needs and its not p2p as you say. P2P might be useful for trade but not services or long term ownership and privacy.
For me it starts with the basis of a communication protocol for services https://github.com/asim/mucp. And then moves onto something on a broader scale https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d3216.... I don't have all the answers yet, but I feel something is coming. An answer to our future needs and its not p2p as you say. P2P might be useful for trade but not services or long term ownership and privacy.
My issue with federation is that so far, everything still requires some centralisation. Take Matrix for instance. The most popular homeserver by a long, long shot is the one that the Matrix foundation runs themselves. This is same for Mastadon, PeerTube, etc. If you let this experiment play out long enough, you end up with a de facto centralised system.
I think P2P ends up being closer to the goal of distributed control because it doesn't tend towards centralisation over time. My issue with P2P though is that it still requires trackers for boostraping in most cases. DHT only gets you so far.
I think there will be some interesting advances once IPv6 becomes more widely deployed. Hopefully this time around multicast is widely deployed on the internet. Imagine being able to subscribe to a multicast group and instantly receive DHT nodes over the internet without a tracker? Not sure how this would work in practice, but I have heard of some research being done here. I think the future looks very good for P2P applications, whereas federated applications are still on shaky ground.
I think P2P ends up being closer to the goal of distributed control because it doesn't tend towards centralisation over time. My issue with P2P though is that it still requires trackers for boostraping in most cases. DHT only gets you so far.
I think there will be some interesting advances once IPv6 becomes more widely deployed. Hopefully this time around multicast is widely deployed on the internet. Imagine being able to subscribe to a multicast group and instantly receive DHT nodes over the internet without a tracker? Not sure how this would work in practice, but I have heard of some research being done here. I think the future looks very good for P2P applications, whereas federated applications are still on shaky ground.
Ultimately I think developers need to take some responsibility for causing and fixing de facto centralization by not steering new users to "default" servers and refusing signups once a server has excess market share.
Federated communication or social media works differently than P2P file sharing. In file sharing you're locating random anonymous peers but in social media that's the last thing that you want. When communicating with a specific person you need their identity anyway and you can use some system like DNS or a blockchain to find the server backing that identity.
Multicast on the public Internet is not going to happen due to state management cost, and it's not helpful for federation anyway due to the spam problem.
Federated communication or social media works differently than P2P file sharing. In file sharing you're locating random anonymous peers but in social media that's the last thing that you want. When communicating with a specific person you need their identity anyway and you can use some system like DNS or a blockchain to find the server backing that identity.
Multicast on the public Internet is not going to happen due to state management cost, and it's not helpful for federation anyway due to the spam problem.
Mastodon and Matrix seem quite different in practice, in this regard.
Who cares what the most popular Slack or Discord is? I'm sure there is one, I bet it's a lot like hanging out on ##hottub back in the day. Matrix is fantastic for organizations.
Mastodon is... I mean it's a Twitter clone at end of day. People want reach, no on should be too surprised that poasters tend to cluster in the biggest instance.
I think chats for companies/organizations/projects are a good fit for federation, and with the adoption of Matrix by the French and now the German government, Slack should be looking over its shoulder.
Who cares what the most popular Slack or Discord is? I'm sure there is one, I bet it's a lot like hanging out on ##hottub back in the day. Matrix is fantastic for organizations.
Mastodon is... I mean it's a Twitter clone at end of day. People want reach, no on should be too surprised that poasters tend to cluster in the biggest instance.
I think chats for companies/organizations/projects are a good fit for federation, and with the adoption of Matrix by the French and now the German government, Slack should be looking over its shoulder.
What I'm saying is that despite Matrix and Mastodon saying they are federated services, in practice they (and things like them) tend towards centralisation anyway. The main selling point of Matrix and Mastodon is that they aren't meant to be centralised. Slack and Discord are much better products (at the moment) if you want a centralised solution. I have not yet seen a service/protocol/product that claims to be federated/decentralised, and actually is in practice. There's always a central server that heavily dominates or some kind of central system required for the whole thing to operate.
And what I'm replying to you, is that my company is not the least bit interested in growing our Matrix instance any faster than the company itself.
Whereas if I ran a Mastodon instance, then of course, I would want as many users as my droplet could carry.
Whereas if I ran a Mastodon instance, then of course, I would want as many users as my droplet could carry.
> My issue with federation is that so far, everything still requires some centralisation. Take Matrix for instance. The most popular homeserver by a long, long shot is the one that the Matrix foundation runs themselves. This is same for Mastadon, PeerTube, etc. If you let this experiment play out long enough, you end up with a de facto centralised system.
Yeah this was Gagron's fault. Running a flagship instance was bound to result in this. I make it a point to educate people about not joining mastodon.social and finding another instance that better suits their interests.
Yeah this was Gagron's fault. Running a flagship instance was bound to result in this. I make it a point to educate people about not joining mastodon.social and finding another instance that better suits their interests.
Matrix does require some de facto centralisation for the identity server. But Mastodon really doesn't. I can set up a server right now and start federating immediately without involving any central system.
People often forget that "the Internet" itself (i.e. IP, BGP) operates on a federated basis.
Traditionally two of the issues with federation have been sharing namespaces and sharing logins. Once you need a single source of truth, the federation breaks and you need to rely on a central authority. There are some new decentralized protocols (Handshake's decentralized naming system is one) that provide a solution to these problems so I'm bullish on federation in general — I think a federated system could break into the mainstream given the recent geopolitical issues affecting centralized services.
Is it easier to just use federated identity that creates pointers to the origin account? e.g account exists on server A and a new signup to server B creates a link as acc://alice@server-a/blabla
Earlier in the month, the website OnlyFans set a price cap on what creators could charge for content in response to a celebrity selling photos that weren't as advertised. This affected the incomes of a lot of sex workers whose content contributed to their livelihoods. [1]
I'm not knowledgeable about sex work at all, but a lot of the discussion I read noted that it seemed troubling how OnlyFans had so much authority over how the content creators did business. It's not uncommon to hear how sex workers or people in the adult industry are at the whims of opaque moderation on social media too. [2]
Sex work can be an exploitative industry, but I think federation could bring a lot of power and control back to the content creators involved online.
Disclaimer: Again, I'm not an expert on the topic of sex work or knowledgeable about it at all. It'd be a lot better to get feedback of the people involved or know the industry better.
[1] https://www.bbc.com/news/business-53979625
[2] https://jezebel.com/instagram-s-solicitation-policies-are-ex...
I'm not knowledgeable about sex work at all, but a lot of the discussion I read noted that it seemed troubling how OnlyFans had so much authority over how the content creators did business. It's not uncommon to hear how sex workers or people in the adult industry are at the whims of opaque moderation on social media too. [2]
Sex work can be an exploitative industry, but I think federation could bring a lot of power and control back to the content creators involved online.
Disclaimer: Again, I'm not an expert on the topic of sex work or knowledgeable about it at all. It'd be a lot better to get feedback of the people involved or know the industry better.
[1] https://www.bbc.com/news/business-53979625
[2] https://jezebel.com/instagram-s-solicitation-policies-are-ex...
In the end, the problem sex workers have on the internet is that they are a huge fraud risk, with that risk coming from both the provider and the client. The mainstream payment providers won't touch it as a result. The less mainstream ones are expensive and still not easy to do business with.
Without easy setup of payment providers, federation is difficult. Any new instance in the federation needs to go through the same pain.
Meanwhile, what does federation actually solve? The only fans model (and most sex work) seems to be a social graph that is a single producer talking to a bunch of clients. The producer needs a server, clients need to connect to that server, where does federation come in?
Disclaimer: I'm not at all knowledgeable either - literally just know what I've read on hn on this subject.
Without easy setup of payment providers, federation is difficult. Any new instance in the federation needs to go through the same pain.
Meanwhile, what does federation actually solve? The only fans model (and most sex work) seems to be a social graph that is a single producer talking to a bunch of clients. The producer needs a server, clients need to connect to that server, where does federation come in?
Disclaimer: I'm not at all knowledgeable either - literally just know what I've read on hn on this subject.
The bottleneck is the Mastercard/Visa duopoly combined with the modern banking system. At the end of the day no amount of Patreon vs Shopify style business models for adult entertainment work fixes the fundamental payment risk problem. Onlyfans helped amortize the risk somewhat by spreading across multiple payment providers and controlling the UX tightly but even so they still had to change the pricing policy. Mindgeek/CCBill are the Paypal/Stripe of adult entertainment payment processing and it is not an easy field to "disrupt". You can put the best actuaries and machine learning engineers in a room and it would be doubtful if they could come up with a better risk model without fundamentally altering the method of transaction. (Crypto is often touted as a supposed fix for this sort of things but I am not knowledgeable enough about it to comment, somebody else please feel free to chime in.)
I followed somewhat the history of the various file sharing protocols, from Napster and OpenNap to Gnutella to Bittorrent.
None of them really stayed totally p2p. With Gnutella, nodes would slow network connections and weird NAT conditions would pop in and pop out. So eventually things changed, and some Gnutella nodes which were online for some time period and had a fast network connection would self-elect as super-nodes (or ultrapeers, or various names). In Gnutella, theoretically ever node was a server and a client, but eventually, Gnutella server/clients would hook into to these super-nodes.
If you want the maximum amount of content, both centralization and decentralization have their drawbacks. You tend to find elements of both (with Gnutella and Bittorrent too).
None of them really stayed totally p2p. With Gnutella, nodes would slow network connections and weird NAT conditions would pop in and pop out. So eventually things changed, and some Gnutella nodes which were online for some time period and had a fast network connection would self-elect as super-nodes (or ultrapeers, or various names). In Gnutella, theoretically ever node was a server and a client, but eventually, Gnutella server/clients would hook into to these super-nodes.
If you want the maximum amount of content, both centralization and decentralization have their drawbacks. You tend to find elements of both (with Gnutella and Bittorrent too).
I'm not convinced federation is immune to corporatization. The article claims that "it optimizes for a self-governing system of human beings talking to human beings, and not for corporations to push their products." Replace "corporations" with "people", how does federation stop people from using the platform to sell products? Especially when in the same breath, the author admits that operators may be pushed to "solict donations" from their users. Or maybe, ads?
Again, you end up with a system in which money is correlated with user experience, and the operators who most efficiently monetize their userbase will be the most successful.
Again, you end up with a system in which money is correlated with user experience, and the operators who most efficiently monetize their userbase will be the most successful.
The only practical issue standing in the way of decentralized services is a lack of business vision. No one has managed or bothered to propose viable business models for decentralized services.
Novel approaches to the business side of decentralized services will naturally allow for a re-think of the UX issue. (Business model -> default config for "grandpa")
~
An alternative solution approach requires faith in one's government. Can you trust it? If you can, then a national identity services (yes, the dreaded digital id) can provide the infrastructural back-bone to build federated systems.
At the end of the day, location and identity is -all- you need to "reach out and touch someone".
Novel approaches to the business side of decentralized services will naturally allow for a re-think of the UX issue. (Business model -> default config for "grandpa")
~
An alternative solution approach requires faith in one's government. Can you trust it? If you can, then a national identity services (yes, the dreaded digital id) can provide the infrastructural back-bone to build federated systems.
At the end of the day, location and identity is -all- you need to "reach out and touch someone".
> No one has managed or bothered to propose viable business models for decentralized services
I think that is for the better.
I think that is for the better.
Mastodon, Misskey and Pleroma are glorified clones of Twitter, and I'm sure there are many more like that. Pixelfed is a glorified clone of Instagram. A few others that I've looked at just don't seem appealing, at least not to me.
People need to stop harping over the technical appeal of the Fediverse, lest they only are interested in courting the technical crowd, which is actually fine with me. If that's what they want more of then they should not be concerned with who's a part of it. I think these federated platforms should not be interested in trying to appeal to anyone at all, because it's resulting in imitations of the exact platforms that we want to get away from. Under the hood they may unique, but all most people are going to care about is the front end.
Personally, my preference for a new social networking service would be very, very plain. Chat-orientated, share documents, comment on shared documents. No hearts or arrows or "Rebeeps, repoots, and boobops". Is there a way to federate message boards? I just want to talk, read and talk about what I've read. Is there a way to federate Hacker News?
People need to stop harping over the technical appeal of the Fediverse, lest they only are interested in courting the technical crowd, which is actually fine with me. If that's what they want more of then they should not be concerned with who's a part of it. I think these federated platforms should not be interested in trying to appeal to anyone at all, because it's resulting in imitations of the exact platforms that we want to get away from. Under the hood they may unique, but all most people are going to care about is the front end.
Personally, my preference for a new social networking service would be very, very plain. Chat-orientated, share documents, comment on shared documents. No hearts or arrows or "Rebeeps, repoots, and boobops". Is there a way to federate message boards? I just want to talk, read and talk about what I've read. Is there a way to federate Hacker News?
> Is there a way to federate Hacker News?
There's Lemmy https://github.com/LemmyNet/lemmy
There's Lemmy https://github.com/LemmyNet/lemmy
There are others too.
I remember talking to the author of Lemmy on Matrix before he made the github repo. Perhaps this has changed, but he didn't seem to have any actual plan for federation. The early work in the repo appeared to be a Vue or Angular frontend (I forget which) with a Reddit-like style with server-to-server communication left as a "TODO". The author seemed to me to be mostly preoccupied with writing essays about why communism is a good idea, so I assumed the author is "an ideas guy" and didn't expect any actual work to get done on the Reddit-clone.
Then one day, for no obvious reason, the repo received hundreds of stars within a few hours. It was a trending Rust repo and was recognized on TWiR, but there was no actual implementation of federation at all. Just a JS frontend, and I think a tiny bit of stub code in Rust.
I'm not saying there are upvote rings or botnets amongst any particular group, but I haven't paid attention to GitHub stars since then.
Then one day, for no obvious reason, the repo received hundreds of stars within a few hours. It was a trending Rust repo and was recognized on TWiR, but there was no actual implementation of federation at all. Just a JS frontend, and I think a tiny bit of stub code in Rust.
I'm not saying there are upvote rings or botnets amongst any particular group, but I haven't paid attention to GitHub stars since then.
There seems to be significant and active work on federation https://github.com/LemmyNet/lemmy/issues/647
When this started in Spring 2019, back when hundreds of github users decided to star this repo, there was actually nothing. Just stub diesel method calls and a JS front end. I think that's what's interesting. Looking at the issues you linked, I now see that a minimal federation implementation didn't actually happen until this year (and by someone else). I'm glad I didn't wait.
>federate Hacker News
Mailing lists can be somewhat like that.
I would think that most any pub-sub system with a minimalist protocol that supports text and maybe some file types would be adequate to build on. However, I think marketing and social factors have more to do with success and failure of these platforms than anything else. There are certainly a lot of capable programmers who have tried or who are trying to make a social media platform without pervasive user surveillance.
I would think that most any pub-sub system with a minimalist protocol that supports text and maybe some file types would be adequate to build on. However, I think marketing and social factors have more to do with success and failure of these platforms than anything else. There are certainly a lot of capable programmers who have tried or who are trying to make a social media platform without pervasive user surveillance.
I work on the design of novel federated systems for adverse environments, like the Internet, where you have to consider the full range of threats to system behavior, functionality, and economy. There is a pretty long list of theoretical trade offs and hard design problems that do not have practical solutions today, which makes it an interesting research area.
Some of the more interesting open questions for systems that want to strongly enforce the sovereignty of logical nodes:
How do you design inter-node protocol strategies that exhibit optimal, robust Nash equilibria in non-trivial real-world environments i.e. uses resources efficiently in all plausible configurations absent centralized control? This is fiendishly difficult because relevant properties of nodes scale in complex, interacting ways and game theory is not sufficiently developed to solve some of these cases in practical systems. In any sufficiently complex and real-time federated environment, you want these mechanics to be almost completely automagic. Currently, there is a bias toward restricting the number of logical nodes in federated systems because that means humans can plausibly deal with the brittleness, which implies centralization.
How do you prevent nodes from being attacked/exploited from both the outside and inside, including malware, DDOS, fraudulent data, exfiltration of private data, etc? This is a separate question from the intrinsic theoretical robustness of the protocol design in terms of resource and scaling efficiency. Also, how do you make a policy easy to describe and enforceable with strong guarantees? The state-of-the-art in high-assurance internetworking (which considers these kinds of problems) is quite far from where it needs to be address these issues convincingly.
How do you audit user behavior that spans multiple sovereign federation domains to limit abuse and misuse of the system while leaking minimal private state?
Some of these are just Hard Problems, unless you have a very simple/restricted system, that we will need to carefully design around to minimize human involvement; most federated system designs just ignore these issues. For others we have plausible approaches theoretically but no real technical implementations with sufficient generality or functionality. And then there are the fundamental trade offs and edge cases that we tend to ignore but which are really important at scale.
tl;dr: federated systems make distributed systems look easy. The reason they are not commonly used is that a correct design for a non-trivial system is deep into the frontier of computer science. We don’t have a complete theory for how to do it.
Some of the more interesting open questions for systems that want to strongly enforce the sovereignty of logical nodes:
How do you design inter-node protocol strategies that exhibit optimal, robust Nash equilibria in non-trivial real-world environments i.e. uses resources efficiently in all plausible configurations absent centralized control? This is fiendishly difficult because relevant properties of nodes scale in complex, interacting ways and game theory is not sufficiently developed to solve some of these cases in practical systems. In any sufficiently complex and real-time federated environment, you want these mechanics to be almost completely automagic. Currently, there is a bias toward restricting the number of logical nodes in federated systems because that means humans can plausibly deal with the brittleness, which implies centralization.
How do you prevent nodes from being attacked/exploited from both the outside and inside, including malware, DDOS, fraudulent data, exfiltration of private data, etc? This is a separate question from the intrinsic theoretical robustness of the protocol design in terms of resource and scaling efficiency. Also, how do you make a policy easy to describe and enforceable with strong guarantees? The state-of-the-art in high-assurance internetworking (which considers these kinds of problems) is quite far from where it needs to be address these issues convincingly.
How do you audit user behavior that spans multiple sovereign federation domains to limit abuse and misuse of the system while leaking minimal private state?
Some of these are just Hard Problems, unless you have a very simple/restricted system, that we will need to carefully design around to minimize human involvement; most federated system designs just ignore these issues. For others we have plausible approaches theoretically but no real technical implementations with sufficient generality or functionality. And then there are the fundamental trade offs and edge cases that we tend to ignore but which are really important at scale.
tl;dr: federated systems make distributed systems look easy. The reason they are not commonly used is that a correct design for a non-trivial system is deep into the frontier of computer science. We don’t have a complete theory for how to do it.
On the other hand, only if we plunge in will we learn to swim. The solutions will come when there is enough at stake to solve them, but right now since almost no one is using decentralisation/federation at scale on the consumer side, no one has the motivation to tackle the problems you have mentioned. Waiting for a perfect solution to arise before jumping in will ensure it will never happen.
Personally speaking I'd like to see a dead simple to self-host CMS that has built-in comments and RSS, making it trivially easy for people to spin up a simple website or blog or even a webpage, and connect it to a domain they have purchased (or even get one from within the CMS).
And by dead simple, I mean something grandpa can do. All they should need is enough skills to install an app, click through a welcome wizard, and start writing on its WYSIWYG editor.
And by dead simple, I mean something grandpa can do. All they should need is enough skills to install an app, click through a welcome wizard, and start writing on its WYSIWYG editor.
You can just open a word processor and save the files to /var/www/ in html format. You don't even need an installation wizard (although on most mainstream OSes you do have to push a button in a GUI somewhere to enable the webserver.) If (for some reason) your computer doesn't have a public address you can install TOR. People with normal browsers will be able to access the pages (even from search engines) via gateways.
I get what you're saying but grandpa is not going to be using Unix (Linux) so /var/www/ is out of question. I'm thinking more along the lines of Publii with a built-in Caddy, just to give an example.
If the computer doesn't have a publicly reachable IP then that's a much tougher obstacle. That can only be solved by voting with your wallet for ISPs that do give you a public IP with reachable ports. Net neutrality is critical for decentralisation and self-hosting.
If the computer doesn't have a publicly reachable IP then that's a much tougher obstacle. That can only be solved by voting with your wallet for ISPs that do give you a public IP with reachable ports. Net neutrality is critical for decentralisation and self-hosting.
This is a feature that works out of the box on OSX. Both /var/www and ~/public are places to put documents. The button to enable appache is in the settings dialog under "sharing" I think.
I only use that path because I have no idea what the windows equivalent is but it also ships with a similar feature.
I only use that path because I have no idea what the windows equivalent is but it also ships with a similar feature.
As far as I can tell, Windows doesn't ship with a web server bundled. You can however install IIS through "activate Windows features" dialogue but not many people use IIS outside enterprise I'd imagine.
Nice article. That said, to talk about the benefits of software federations without conceding that cryptocurrency is the ultimate foundation for such an ecosystem is delusional.
Money is the only incentive which works in our modern age. Any federated or decentralized system needs to start with money because money is the ultimate tool for control.
Money is the only incentive which works in our modern age. Any federated or decentralized system needs to start with money because money is the ultimate tool for control.
> This system also makes it hard for marketing and spam to get a foothold
> You’re certainly familiar with another federation [...]: email
> You’re certainly familiar with another federation [...]: email
This is an eloquent description of what my reasoning was when I started to work on an ActivityPub powered link aggregator[1]. I hope Drew will allow me to re-use some of his words to improve it.
[1] https://littr.me/about
[1] https://littr.me/about
I was slightly surprised to get all the way through the article without seeing a single mention of NNTP.
As a solution or as something that was tried before?
Spam as a term transitioned from, “flooding a communication channel with the same message” to “ads sent indiscriminately” due to a green card advertisement on NNTP.
What we need along with federation is federated moderation. There are some characterizations of content where we can agree to a label but not whether the label is bad. You might trust me to label gore accurately, I might trust you to label incitement to doxing. We might never agree on what constitutes baiting. Or the crossover point where a flame war should be filtered. So you might ban an individual who I think is “keepin it real” or vice versa. But ten of us shouldn’t have to do the exact same moderation in order to maintain a federation. N log n would be great, 3N would probably be tolerable.
Spam as a term transitioned from, “flooding a communication channel with the same message” to “ads sent indiscriminately” due to a green card advertisement on NNTP.
What we need along with federation is federated moderation. There are some characterizations of content where we can agree to a label but not whether the label is bad. You might trust me to label gore accurately, I might trust you to label incitement to doxing. We might never agree on what constitutes baiting. Or the crossover point where a flame war should be filtered. So you might ban an individual who I think is “keepin it real” or vice versa. But ten of us shouldn’t have to do the exact same moderation in order to maintain a federation. N log n would be great, 3N would probably be tolerable.
In retrospect NNTP and IRC, where every message is replicated to every server, are not a good style of federation.
Does IRC replicate every message to all servers? It's been many years since I was active on IRC, but I thought it was only sending messages along the network tree where needed. So a message to a user would be sent by the sender's server to the recipient's server, or, if not directly connected, to a server one hop closer to that server. And for messages to channels, those only go along the tree if there is a user in the channel from that side of the tree.
With today's IRC user count and today's server capabilities, you might have an easier time running on a single server, but there's maybe some latency benefits for users with geographically diverse servers (maybe with support for a server to server mesh instead of a tree, to avoid multi-hop message paths)
With today's IRC user count and today's server capabilities, you might have an easier time running on a single server, but there's maybe some latency benefits for users with geographically diverse servers (maybe with support for a server to server mesh instead of a tree, to avoid multi-hop message paths)
Just occurred to me that passports (the paper ones that you use to cross borders) are an example of a federated system.
Linux distros/brew/npm/maven/etc are federated software distribution.
Not really; most of them have centralized namespaces and servers. PPAs, Flatpak, and Deno are federated though.
[deleted]
Diaspora* and Matrix are using federation.
> Cryptocurrency is the digitally woke techbro’s ponzi scheme
The fiat monetary system is a ponzi scheme. Watch "Hidden Secrets of Money": https://youtu.be/iFDe5kUUyT0?t=62
I also recommend doing your own research after this. The evidence is clear and overwhelming.
Cryptocurrency is merely a response to this scam and its continued success helped to further prove that the financial system is a scam.
Even if you assumed that cryptocurrency is a scam, what kind of financial system enables a scam to thrive for 10+ years? In such a system, how can you tell the scams apart from the non-scams?
Clearly the so-called 'free markets' are not capable of distinguishing the two.
What's to say that most big tech corporations aren't scams? They get a lot of 'free money' from the government through massive contracts to build projects which only add more bureaucracy.
The fiat monetary system is a ponzi scheme. Watch "Hidden Secrets of Money": https://youtu.be/iFDe5kUUyT0?t=62
I also recommend doing your own research after this. The evidence is clear and overwhelming.
Cryptocurrency is merely a response to this scam and its continued success helped to further prove that the financial system is a scam.
Even if you assumed that cryptocurrency is a scam, what kind of financial system enables a scam to thrive for 10+ years? In such a system, how can you tell the scams apart from the non-scams?
Clearly the so-called 'free markets' are not capable of distinguishing the two.
What's to say that most big tech corporations aren't scams? They get a lot of 'free money' from the government through massive contracts to build projects which only add more bureaucracy.
1. Among other things, because of a dead-simple user experience that lets even grandpa post content after a few clicks. Federation, if it adds any level of complexity past the grandpa UX, would defeat the goal.
2. Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities? What if an account has to pass a quality threshold before it can publish in public, but can be used privately any time. HN has similar model, but here you limit public posting until 10 random other users have seen and upvoted the new user’s post. Add to that possible additional authentication for payment of a pro feature, or a human check mark, and you’ve put a hurdle on spam.
3. To keep culture from running amuck provide positive and negative feedback. Public content has a span problem, not so much private behavior. Content ratings and discussion licenses can come into play as well, where any group can limit posts to PG content, or discussion that gets moderated automatically or manually if it passes certain thresholds.
Happy to suggest more if needed. I run https://www.dreamlist.com