Apparently Bluesky has one centralized service, the "relay"(mastodon.online)
mastodon.online
Apparently Bluesky has one centralized service, the "relay"
https://mastodon.online/@mastodonmigration/114399534536933573
82 comments
Haven't we learned by now that approximately no one cares? Centralised services are vastly more popular than federated ones, the main reason being that they reduce the paradox / paralysis of choice when you're signing up for them. (Solve that problem properly and you may be on to something.)
Definitely agree that centralized services have a lot of advantages. Bluesky deserves some criticism for trying to have their cake and eat it too, though. They told a good story about being decentralized, and lots of people repeated it while ignoring technical experts pointing out it's not true. Even on HN, the claim that they're decentralized was repeated a lot.
No. This isn’t true. Bluesky has always been very transparent about exactly what flavor of decentralization they offer. The whole thing is swappable microservices you can host yourself, including the relay if you really care. The relay is an optimization to make their app offering work in a performant, scalable way.
The relay is described here and here: https://arxiv.org/pdf/2402.03239, https://atproto.com/guides/glossary#relay
The protocol Bluesky is built on is certainly decentralized. It’s really fucking annoying that people act like it’s not “real” decentralization, when in fact it’s just a different flavor of decentralization on the technical level compared to, say, the fediverse.
The relay is described here and here: https://arxiv.org/pdf/2402.03239, https://atproto.com/guides/glossary#relay
The protocol Bluesky is built on is certainly decentralized. It’s really fucking annoying that people act like it’s not “real” decentralization, when in fact it’s just a different flavor of decentralization on the technical level compared to, say, the fediverse.
Exactly. There's a lot of dogmatic hype with atproto, which is kinda giving me early crypto/NFT vibes. Any discussions or criticisms are quickly snuffed out or labeled as toxic or uninformed, or devolve into whataboutism.
A lot of the content on bluesky, but especially in its early days, is about how the protocol is great, its potential and what bright future it will lead us into. Their main investors are a crypto bro company. Their CEO has built her career around crypto. It's the same rhetoric.
Now it's about decentralized "verification". They still haven't defined what they're verifying except a vague term "the person posting is who they say they are", but it's not actual identity verification.
The endgame is probably monetizing the protocol by connecting it to some form of identity for crypto-bs or paywalling engagement via the verification.
A lot of the content on bluesky, but especially in its early days, is about how the protocol is great, its potential and what bright future it will lead us into. Their main investors are a crypto bro company. Their CEO has built her career around crypto. It's the same rhetoric.
Now it's about decentralized "verification". They still haven't defined what they're verifying except a vague term "the person posting is who they say they are", but it's not actual identity verification.
The endgame is probably monetizing the protocol by connecting it to some form of identity for crypto-bs or paywalling engagement via the verification.
Ironically, your whole comment is whataboutism.
I'm pretty sure the ATproto answer to that is "don't bother users with that decision", i.e. just point new users to somewhere (be it bsky.social or some other PDS provider) and call it a day. This "works" from a decentralized/federated point of view because ATproto supports migrating accounts/identities between PDSes even if the old PDS is offline or adversarial, and because account/identity hosting (i.e. the PDS) is decoupled from the app itself (i.e. the appview) - so even if you signed up as @foo.example.app instead of @foo.bsky.social, you can still log into Bluesky as @foo.example.app (and likewise, @bar.bsky.social can still log into Example App), and if Example App (or Bluesky, for that matter) ever kicks the bucket you can migrate your account (and its followers, and very likely its public content, and possibly its private content if that was backed up somewhere) to someplace else.
I thought I had it (mostly) solved with fediverser: users would go to one place, sign up with OAuth to the service they wanted to leave (Reddit, Twitter...) and the system would behind the scenes create an account on any of the participating instances. In doing so, it could leverage the user existing information (subreddits subscribed, users followed on Twitter) and find the corresponding subreddits/users that are already registered.
Turns out the biggest challenge was not in getting users, but in convincing admins to join the network. Instances with open registrations are already dealing with spammer accounts, and none of them was excited about the idea of this extra vector for having unvetted users on their services.
This doesn't mean that centralized services are safe, though. I am reasonably convinced that we can have "social media" that is less focused on "platforms" and more like what we (used to) have with web: companies and institutions owning their presence by running ActivityPub "servers" on their own domain, and a hotch-potch of community/commercial servers to serve the users who want "basic access".
But to get there, we need to stop thinking that the way to get rid of Facebook/Twitter/Reddit/Instagram/YouTube is by taking their templates and tacking "but federated!", and we need to really come up with a killer app on ActivityPub (or Solid, or Linked Data, or ActivityPods) to disrupt the whole thing entirely.
Turns out the biggest challenge was not in getting users, but in convincing admins to join the network. Instances with open registrations are already dealing with spammer accounts, and none of them was excited about the idea of this extra vector for having unvetted users on their services.
This doesn't mean that centralized services are safe, though. I am reasonably convinced that we can have "social media" that is less focused on "platforms" and more like what we (used to) have with web: companies and institutions owning their presence by running ActivityPub "servers" on their own domain, and a hotch-potch of community/commercial servers to serve the users who want "basic access".
But to get there, we need to stop thinking that the way to get rid of Facebook/Twitter/Reddit/Instagram/YouTube is by taking their templates and tacking "but federated!", and we need to really come up with a killer app on ActivityPub (or Solid, or Linked Data, or ActivityPods) to disrupt the whole thing entirely.
> Turns out the biggest challenge was not in getting users, but in convincing admins to join the network
The incentives for homeserver admins are extremely perverted and it's why the Mastodon network in particular is so dominated by ideological cliques.
Running a homeserver is thankless, laborious, and expensive, and the costs go up with each user. There's no money for it, so admins have to be compensated another way. Either they get off on the power, or have an ideological axe to grind and thus moral compensation, or they are altruistic and eventually burn out when they are subjected to the abuse of the job.
The financial story has to be solved. Admins must be paid to run services or the services get distorted to compensate them some other way.
The incentives for homeserver admins are extremely perverted and it's why the Mastodon network in particular is so dominated by ideological cliques.
Running a homeserver is thankless, laborious, and expensive, and the costs go up with each user. There's no money for it, so admins have to be compensated another way. Either they get off on the power, or have an ideological axe to grind and thus moral compensation, or they are altruistic and eventually burn out when they are subjected to the abuse of the job.
The financial story has to be solved. Admins must be paid to run services or the services get distorted to compensate them some other way.
> The financial story has to be solved
You are preaching to the converted. :) [0]
I might be wrong, but I think that my instance was the first to provide accounts only to paying subscribers, and even today there are only 2 others like mine.
[0] https://mastodon.communick.com/@raphael/114365227998082545
You are preaching to the converted. :) [0]
I might be wrong, but I think that my instance was the first to provide accounts only to paying subscribers, and even today there are only 2 others like mine.
[0] https://mastodon.communick.com/@raphael/114365227998082545
Yup. Mastodon has this issue. Can’t pick a big generic instance because it could get defederated for spam/being too normie; can’t pick a small instance because it could get defederated over some ridiculous drama [1][2].
Normal people (and even some not-normal people) don’t want to deal with that. “Instances” are a bad model.
[1]: https://tootworld.social/@lilythelonelygirl/1143999833553083...
[2]: https://tau-ceti.space/@[email protected]/114394202556951...
Normal people (and even some not-normal people) don’t want to deal with that. “Instances” are a bad model.
[1]: https://tootworld.social/@lilythelonelygirl/1143999833553083...
[2]: https://tau-ceti.space/@[email protected]/114394202556951...
Weird. Millions of people seem able to deal with it just fine.
Yeah mods of an instance have the power to do so for the sake of their users (in their view at least).
Your comment also implies their actions affect all other instances just by seldomly doing this action. Which isn't the case.
We can cherry pick all day, but one cannot take the whole Fediverse down like it happened with Bluesky. Which is the topic at hand.
Your comment also implies their actions affect all other instances just by seldomly doing this action. Which isn't the case.
We can cherry pick all day, but one cannot take the whole Fediverse down like it happened with Bluesky. Which is the topic at hand.
> Can’t pick a big generic instance because
Nobody is going to defederate from mastodon.social, but otherwise, this is all true.
Nobody is going to defederate from mastodon.social, but otherwise, this is all true.
It's very common to defederate from mastodon.social due to spam. It might be among the most de-federated instances.
People can and have.
I don't think the instances themselves are the problem. It's rather the whole defederation mess. If this had been left in the users' own hands rather than the server operators, it wouldn't have been an issue.
After all email is very federated and it works fine. And people have no issue grokking it. The implementation matters.
After all email is very federated and it works fine. And people have no issue grokking it. The implementation matters.
The main reason is that they have marketing of a corporate entity behind them and someone to sign a contract with.
I've seen many times - company switching from free, open-source, distributed solution to a worse, closed-source, coporate-backed solution just so they have someone to sign a contract with.
First time it was moving from self-hosted Jabber to MSN Messanger (is sucked, worked less reliably than self-hosted jabber, didn't worked on Linux, and was probably way more expansive in the long run). Then it was moving from self-hosted wikis to some B2B solution. Then it was self-hosted git to corporate github or sth similar.
I understand the theory behind outsourcing these things, especially if you're just starting. But if you already have the OS solution deployed and working - why switch?
I've seen many times - company switching from free, open-source, distributed solution to a worse, closed-source, coporate-backed solution just so they have someone to sign a contract with.
First time it was moving from self-hosted Jabber to MSN Messanger (is sucked, worked less reliably than self-hosted jabber, didn't worked on Linux, and was probably way more expansive in the long run). Then it was moving from self-hosted wikis to some B2B solution. Then it was self-hosted git to corporate github or sth similar.
I understand the theory behind outsourcing these things, especially if you're just starting. But if you already have the OS solution deployed and working - why switch?
The first startup I worked for had 1 IT guy among 12 people, and everything was in-house, including servers. The second grew to 50 people and did not ever employ an IT guy for internal work - most stuff was run on SaaS and the total cost was less than an IT guy salary. Any admin was done by taking engineering time, and no-one wanted to divert that away from product to do in-house stuff. Because when you're taking time away from product, it's not just the salary but the opportunity cost, because any investment in product is supposed to return a multiple.
What conclusions could you draw from this observation?
And you know what would be even cheaper (in the medium/long term) than the IT in-house or outsourcing to commercial SaaS? If the company took 10-20% of their budget to sponsor the development of FOSS alternatives.
Maybe, but I think a lot of people are more acutely aware of the risks of centralized services lately, and I think BlueSky likely got a popularity boost by pretending to be decentralized.
I wouldn't be so dismissive of the importance when people succumb to the realization that it's like VHS/Betamax, but instead of Bruce Willis in Die Hard, at stake is literally half of your friends and acquaintances.
Email has solved federation / decentralisation long before WWW.
Nobody is confused when you hand them a [email protected], [email protected], etc; I use my own [email protected] and sometimes get a blank stare, until people realise you can go to www.gmail.com to check your own inbox, and you can totally just type www.whatever.com into your browser.
Links like reddit.com/u/user, youtube.com/@user, already exist and are de facto a standard of some kind. If we stop trying to make @[email protected] a thing, the only obstacle is in convincing people that whatever.com/user is just another link you can click, and this is totally how they can reach you - send you a message.
Federation between servers is an entirely different topic, but for the purpose of this discussion we can assume it's just an implementation detail - just like SMTP is for GMail users.
I'm probably making this sound more trivial than it actually is, but IMO all you have to do is build up on existing paradigms and collective understanding.
Nobody is confused when you hand them a [email protected], [email protected], etc; I use my own [email protected] and sometimes get a blank stare, until people realise you can go to www.gmail.com to check your own inbox, and you can totally just type www.whatever.com into your browser.
Links like reddit.com/u/user, youtube.com/@user, already exist and are de facto a standard of some kind. If we stop trying to make @[email protected] a thing, the only obstacle is in convincing people that whatever.com/user is just another link you can click, and this is totally how they can reach you - send you a message.
Federation between servers is an entirely different topic, but for the purpose of this discussion we can assume it's just an implementation detail - just like SMTP is for GMail users.
I'm probably making this sound more trivial than it actually is, but IMO all you have to do is build up on existing paradigms and collective understanding.
It's funny you mention gmail and hotmail. That's exactly my point. Who is setting up their own email server when they could instead sign up for one of those? Is it even 1 in 100,000 email users?
(I say this as someone running my own email server, who periodically has problems sending to gmail.)
(I say this as someone running my own email server, who periodically has problems sending to gmail.)
> Who is setting up their own email server when they could instead sign up for one of those?
About the same number of people who set up their own WWW server, modulo the absolute PITA that is email.
Still, federation between GMail, Hotmail, iCloud, Yahoo, ProtonMail, etc does work. You don't have as many choices but it can still feel overwhelming if you're shopping for an alternative. People at Corp & Co use their @corpnco.com addresses daily. My personal email is connected to iCloud, but I can always point my MX elsewhere. Running your own Mastodon instance is costly - however fediverse already leans on smaller interconnecting communities.
Nobody sets up an AS just to run some fiber to their home. Decentralisation and federation just happen at different levels and scales.
About the same number of people who set up their own WWW server, modulo the absolute PITA that is email.
Still, federation between GMail, Hotmail, iCloud, Yahoo, ProtonMail, etc does work. You don't have as many choices but it can still feel overwhelming if you're shopping for an alternative. People at Corp & Co use their @corpnco.com addresses daily. My personal email is connected to iCloud, but I can always point my MX elsewhere. Running your own Mastodon instance is costly - however fediverse already leans on smaller interconnecting communities.
Nobody sets up an AS just to run some fiber to their home. Decentralisation and federation just happen at different levels and scales.
I would count is as decentralized enough if there are a few major players, you have Google, Microsoft, I assume Proton Mail works fine though don’t know as I only use it for burner accounts, iCloud Mail including Apple’s cool private email relay thing. (Maybe other countries have big providers I dunno.) You can use your own domain and switch between providers if needed, and use custom email clients… it’s all the benefits of decentralization to the end user.
[deleted]
It doesn't matter what instance you choose. An instance is a group, who is part of a bigger group.
But yeah, the majority of people won't ever understand that. And, in all honesty, that's fine.
Really, I'm cool with just having people around who care enough to grasp that. If you care that's cool, if not ¯\_(ツ)_/¯
Nevertheless, this discussion is a whole other topic. They are just pointing out a potential issue.
But yeah, the majority of people won't ever understand that. And, in all honesty, that's fine.
Really, I'm cool with just having people around who care enough to grasp that. If you care that's cool, if not ¯\_(ツ)_/¯
Nevertheless, this discussion is a whole other topic. They are just pointing out a potential issue.
It's possible to spin up alternative relays. For example: https://whtwnd.com/bnewbold.net/3kwzl7tye6u2y
The issue is that relays tend to require a lot more resources to operate than appviews and PDSes (though not necessarily as much as that blog post suggests; I recall posts of people running their own relays on RPi4s with NVMe drives), so it's common for alternative appviews to rely on Bluesky's relay instead of taking on the expense of spinning up a new one.
In any case, as that toot notes, the Bluesky outage was on a PDS level rather than a relay level. And thankfully it's much less expensive to run your own PDS; apparently those who do so weren't impacted.
The issue is that relays tend to require a lot more resources to operate than appviews and PDSes (though not necessarily as much as that blog post suggests; I recall posts of people running their own relays on RPi4s with NVMe drives), so it's common for alternative appviews to rely on Bluesky's relay instead of taking on the expense of spinning up a new one.
In any case, as that toot notes, the Bluesky outage was on a PDS level rather than a relay level. And thankfully it's much less expensive to run your own PDS; apparently those who do so weren't impacted.
Genuine question: if it's so easy and cheap to host a relay, why then "Free Our Feeds" initiative [0] looking to raise $4,000,000 [1] to establish a second relay [2]? Most of that money must be earmarked for administrative and human expenses then, right?
[0] https://freeourfeeds.com/
[1] https://www.gofundme.com/f/help-us-free-social-media-from-bi...
[2] https://freeourfeeds.com/ § FAQ § What will the money be used for?
[0] https://freeourfeeds.com/
[1] https://www.gofundme.com/f/help-us-free-social-media-from-bi...
[2] https://freeourfeeds.com/ § FAQ § What will the money be used for?
> if it's so easy and cheap to host a relay
That wasn't asserted anywhere. Quite the opposite: as I explained above, the expense is why few people have done it (and even fewer have done it in production). It's the PDSes which are (relatively) cheap and easy to self-host.
> why then "Free Our Feeds" initiative [0] looking to raise $4,000,000 [1] to establish a second relay [2]?
Per the section you cite, they're doing a lot more with that money than running a second relay: they're spinning up an entirely separate organization independent of Bluesky to develop ATproto and applications using it. That includes, but is nowhere implied or explied to be limited to, the "second relay" they mention.
In any case, even the self-hosted relay described in that above-linked blog post (let alone some RPi under someone's bed) is in all likelihood a long ways off from one that's even remotely production-ready. There's no mention of redundancy, no mention of future-proofing, etc. It's reasonable to assume that the "second relay" would be multiple such relays, likely on machines with even beefier specs - in other words, at least as capable as the existing Bluesky-managed relay. I'd also be unsurprised if it expanded to a "third relay" and "fourth relay" and so on.
Further, there's more to running a relay than just the hardware; you need someone to maintain it. $4 million pays for 40 employee-years (assuming every employee is full-time with an annual salary of $100k). That could be one sysadmin for 40 years, or an 8 person team for 5 years, or a 40 person team for 1 year, or what have you. Free our feeds claims they'll need $30 million over 3 years, i.e. $10 million per year; if half that goes to salaries, we end up with a napkin-math-guesstimated team size of 50 - which is about the size I'd expect for an organization that wants to independently maintain a bunch of technical infrastructure, develop applications, prod whomever needs prodded to get ATproto formally standardized, etc.
That wasn't asserted anywhere. Quite the opposite: as I explained above, the expense is why few people have done it (and even fewer have done it in production). It's the PDSes which are (relatively) cheap and easy to self-host.
> why then "Free Our Feeds" initiative [0] looking to raise $4,000,000 [1] to establish a second relay [2]?
Per the section you cite, they're doing a lot more with that money than running a second relay: they're spinning up an entirely separate organization independent of Bluesky to develop ATproto and applications using it. That includes, but is nowhere implied or explied to be limited to, the "second relay" they mention.
In any case, even the self-hosted relay described in that above-linked blog post (let alone some RPi under someone's bed) is in all likelihood a long ways off from one that's even remotely production-ready. There's no mention of redundancy, no mention of future-proofing, etc. It's reasonable to assume that the "second relay" would be multiple such relays, likely on machines with even beefier specs - in other words, at least as capable as the existing Bluesky-managed relay. I'd also be unsurprised if it expanded to a "third relay" and "fourth relay" and so on.
Further, there's more to running a relay than just the hardware; you need someone to maintain it. $4 million pays for 40 employee-years (assuming every employee is full-time with an annual salary of $100k). That could be one sysadmin for 40 years, or an 8 person team for 5 years, or a 40 person team for 1 year, or what have you. Free our feeds claims they'll need $30 million over 3 years, i.e. $10 million per year; if half that goes to salaries, we end up with a napkin-math-guesstimated team size of 50 - which is about the size I'd expect for an organization that wants to independently maintain a bunch of technical infrastructure, develop applications, prod whomever needs prodded to get ATproto formally standardized, etc.
I at least am contrasting
> [running a relay being cheap and easy] wasn't asserted anywhere
With
> I recall posts of people running their own relays on RPi4s with NVMe drives
I would absolutely consider software I can host at home, on a RPi, cheap and easy to self-host. That's the assertion that's being called out here. Bluesky's relays do not scale down easily, and are difficult and expensive to host
> [running a relay being cheap and easy] wasn't asserted anywhere
With
> I recall posts of people running their own relays on RPi4s with NVMe drives
I would absolutely consider software I can host at home, on a RPi, cheap and easy to self-host. That's the assertion that's being called out here. Bluesky's relays do not scale down easily, and are difficult and expensive to host
> I would absolutely consider software I can host at home, on a RPi, cheap and easy to self-host.
That's expensive and difficult compared to running a PDS or appview (either of which can run with a tiny fraction of even an RPi's resources), which is exactly what I said. And to reiterate: an RPi4 with an NVMe SSD is very far off from something that's production-ready and suitable for public use. You can run your own relay, but it's probably not going to handle 30+ million users like Bluesky's relay does, or like Free Our Feeds' "second relay" presumably seeks to do.
That's expensive and difficult compared to running a PDS or appview (either of which can run with a tiny fraction of even an RPi's resources), which is exactly what I said. And to reiterate: an RPi4 with an NVMe SSD is very far off from something that's production-ready and suitable for public use. You can run your own relay, but it's probably not going to handle 30+ million users like Bluesky's relay does, or like Free Our Feeds' "second relay" presumably seeks to do.
- A raspberry pi with a nvme drive costs 200 dollars one time. Are you seriously going to assert that is expensive?
- You don’t understand what a relay does in atproto given the rest of your reply and should look.
- You don’t understand what a relay does in atproto given the rest of your reply and should look.
> A raspberry pi with a nvme drive costs 200 dollars one time. Are you seriously going to assert that is expensive?
Compared to the hardware required to run a PDS, yes, absolutely. "Expensive" is relative.
And like I've said above, a Raspberry Pi with an NVMe drive is surely a long ways off from Bluesky's own relay. It's good enough for personal needs, not for any sort of production use.
> You don’t understand what a relay does in atproto given the rest of your reply and should look.
I'd appreciate specific corrections, so that I (and anyone else reading these comments) can be better-informed.
(EDIT: my apologies for the previous version of this comment, which might've come across a bit hostile. That ain't my intention.)
Compared to the hardware required to run a PDS, yes, absolutely. "Expensive" is relative.
And like I've said above, a Raspberry Pi with an NVMe drive is surely a long ways off from Bluesky's own relay. It's good enough for personal needs, not for any sort of production use.
> You don’t understand what a relay does in atproto given the rest of your reply and should look.
I'd appreciate specific corrections, so that I (and anyone else reading these comments) can be better-informed.
(EDIT: my apologies for the previous version of this comment, which might've come across a bit hostile. That ain't my intention.)
Scale, its always question of scale. Whatever youncan easily and cheaply host yourself will likely be only good enough for you and your friends or family. Anything more will require more hardware and dedicated people for maintaining that hardware and software.
plainly, free our feeds are grifting.
the relay at this point is non-archival and can be spun up trivially. with a small sliding history window for subscriber catchup u can use like 32gb of scratch disk space and keep a few hours, the relay is literally just a subscribeRepos forwarder from PDSes.
the AppView is vastly more expensive to run since you need to handle the write volume of all bsky activity. if you build a non-bsky app on atproto this is a non-issue
the issue here really is that nobody writes about the state of things in long form outside the network so it's not really known how fast things move and change by those not engaged with the platform
the relay at this point is non-archival and can be spun up trivially. with a small sliding history window for subscriber catchup u can use like 32gb of scratch disk space and keep a few hours, the relay is literally just a subscribeRepos forwarder from PDSes.
the AppView is vastly more expensive to run since you need to handle the write volume of all bsky activity. if you build a non-bsky app on atproto this is a non-issue
the issue here really is that nobody writes about the state of things in long form outside the network so it's not really known how fast things move and change by those not engaged with the platform
Re: the relay, that depends on your needs. My impression from these sorts of "run a relay on an RPi" projects is that they're only dealing with a subset of the full firehose Bluesky's relay has to deal with - be it a shorter timeline (as you mention) or only concerning themselves with specific accounts (like the relay operator's own "following" feed, in the case of someone running a personal relay) or what have you. Pretty sure even Bluesky's relay doesn't try to drink from the whole firehose (or if it does, it's tolerant of "dribbling" so to speak; I recall a Bluesky dev blog post about how temporarily dropping posts from users' feeds is acceptable if the relay can't keep up).
Re: write traffic, my understanding is that the appviews shift most (if not all) of that burden to the PDSes, no?
Re: write traffic, my understanding is that the appviews shift most (if not all) of that burden to the PDSes, no?
- the relay storage volume scales only with the backfill window for consumers that drop briefly - the bluesky pbc operated relays let you reconnect up to 24h later and not miss any events but that requires around 200gb of scratch disk space -- live tailing an rpi relay without dropping a connection can give you events from the full network span (ie the complete set of the firehose) without requiring any backfill window, but it's nice to use a few tens of gigabytes anyway. -- the full firehose is like 20mbps at maximum so it's far from hard to serve a few live consumers
- bluesky's feed gen post-dropping is about internal operation of their appview and not anything to do with network sync semantics
- if you're running an AppView for the bsky data you are likely keeping a copy of all bsky posts in a database, since fetching from PDSes on-the-fly is network intensive over a relatively small pipe, which is what i mean by write volume requirements.
- bluesky's feed gen post-dropping is about internal operation of their appview and not anything to do with network sync semantics
- if you're running an AppView for the bsky data you are likely keeping a copy of all bsky posts in a database, since fetching from PDSes on-the-fly is network intensive over a relatively small pipe, which is what i mean by write volume requirements.
> the relay storage volume scales only with the backfill window for consumers that drop briefly […] bluesky's feed gen post-dropping is about internal operation of their appview and not anything to do with network sync semantics
Gotcha; thanks for the clarifications/corrections. Good to know that the firehose bandwidth is a lot less than I thought (though 20Mbps can certainly add up to some hefty pricetags depending on how you're billed for traffic).
> if you're running an AppView for the bsky data you are likely keeping a copy of all bsky posts in a database, since fetching from PDSes on-the-fly is network intensive over a relatively small pipe, which is what i mean by write volume requirements.
Right, but how much of that actually needs to hit the disk? I'd imagine most appviews can readily get away with just keeping posts in RAM, and even if disk storage is desired (e.g. to avoid needing to pull everything from the PDSes if an appview server reboots), it ain't like the writes need to be synchronous or low-latency. A full-blown ACID-compliant DBMS is probably overkill.
It'd also be overkill to cache all posts, rather than subsets (e.g. each users' "Discover" and "Following" feeds), so I reckon that'd also reduce the in-appview caching needs further.
Gotcha; thanks for the clarifications/corrections. Good to know that the firehose bandwidth is a lot less than I thought (though 20Mbps can certainly add up to some hefty pricetags depending on how you're billed for traffic).
> if you're running an AppView for the bsky data you are likely keeping a copy of all bsky posts in a database, since fetching from PDSes on-the-fly is network intensive over a relatively small pipe, which is what i mean by write volume requirements.
Right, but how much of that actually needs to hit the disk? I'd imagine most appviews can readily get away with just keeping posts in RAM, and even if disk storage is desired (e.g. to avoid needing to pull everything from the PDSes if an appview server reboots), it ain't like the writes need to be synchronous or low-latency. A full-blown ACID-compliant DBMS is probably overkill.
It'd also be overkill to cache all posts, rather than subsets (e.g. each users' "Discover" and "Following" feeds), so I reckon that'd also reduce the in-appview caching needs further.
the reference bluesky backend does just keep everything around but this idea has merit!! you're actually reinventing something like AppViewLite right now, which does throw away old data: https://github.com/alnkesq/AppViewLite
bluesky chooses to not refetch data from PDSes all the time so that the load for a PDS stays low (they like it to be possible to run on a home connection)
bluesky chooses to not refetch data from PDSes all the time so that the load for a PDS stays low (they like it to be possible to run on a home connection)
The genie is already out of the bottle for thirty years, and it's called the open web. Luckily I don't need half the world's population to agree upon and use something that already exists out there.
Keep posting on your website and keep linking. It's called the 'web' for a reason.
Keep posting on your website and keep linking. It's called the 'web' for a reason.
I absolutely love thew web. I wish I could get onboard with this sentiment. But, the web bitrots away, piece by piece year by year. Archive.org mercifully backstops against much of the loss. But its not enough.
And demanding everyone buy and own a domain name & figure out hosting seems unviable as requirements for the world's mass communication system.
The Bluesky PDS architecture is a lovely lovely modest & small iteration on the web, adds only a bit more. To make your vault of content still addressable, still a URL, but one tied to a cryptographic identifier. And to make the content all signed and secure.
And demanding everyone buy and own a domain name & figure out hosting seems unviable as requirements for the world's mass communication system.
The Bluesky PDS architecture is a lovely lovely modest & small iteration on the web, adds only a bit more. To make your vault of content still addressable, still a URL, but one tied to a cryptographic identifier. And to make the content all signed and secure.
> It's called the 'web' for a reason.
Yes, historic ones. It's not a prediction about the future shape of the internet.
Yes, historic ones. It's not a prediction about the future shape of the internet.
You know why it's called the web? Because all the websites and webpages linking to each other form the shape of a web. I don't know what shape the internet is going to be, but the web is shaped like a web. Also tomorrow.
I love the web, but it is also clear that much of the internet is now a hub and spoke model. Client/server, in other words. The difficulty was always matching clients with different servers, and the centralised services never had to solve it.
I'm on HN for years now and what I slowly came to realize is that people here (maybe more than somewhere else, maybe not) like to spend time and energy explaining how stuff is, like it's still some sort of school test that gives one points for all the correct answers. Thereby easily forgetting what has been and still is, what works, what one can do about stuff in a positive way, what choices people have (instead of constantly pretending there is no choice, or someone else already chose for us), etc.
Even pointing out that people have a choice - with some work, yes, but only a little work - elicits some counter response that basically comes down to: no, you don't have a choice. I don't like that, and it also signals creative laziness.
P.S. Zoom out a little. The open web is also client/server (although peer-to-peer would be nice), that's not the point. We all need some infra, some trust, and a clear rule of law for the internet and the web to work. We just don't need big tech's view of 'social media' (there's nothing social about that) or platforms that present themselves as being an alternative as long as we have the perfect alternative - called the open web - being around for thirty years, battle tested and actively maintained by millions.
P.P.S. This is in reply to you, but read it as a general opinion, it's not a personal attack on anything you said.
Even pointing out that people have a choice - with some work, yes, but only a little work - elicits some counter response that basically comes down to: no, you don't have a choice. I don't like that, and it also signals creative laziness.
P.S. Zoom out a little. The open web is also client/server (although peer-to-peer would be nice), that's not the point. We all need some infra, some trust, and a clear rule of law for the internet and the web to work. We just don't need big tech's view of 'social media' (there's nothing social about that) or platforms that present themselves as being an alternative as long as we have the perfect alternative - called the open web - being around for thirty years, battle tested and actively maintained by millions.
P.P.S. This is in reply to you, but read it as a general opinion, it's not a personal attack on anything you said.
Previously, related:
Wait, how did a decentralized service like Bluesky go down? - https://news.ycombinator.com/item?id=43789178 - April 2025 (2 comments)
Wait, how did a decentralized service like Bluesky go down? - https://news.ycombinator.com/item?id=43789178 - April 2025 (2 comments)
If you understand that Bluesky today is just Twitter for a different audience, then you realize nobody there cares about decentralization and all that mumbo jumbo.
Which is fine, I don’t care either, but this whole spiel that they are decentralized feels dishonest.
Which is fine, I don’t care either, but this whole spiel that they are decentralized feels dishonest.
i don’t actively contribute to Bluesky nor Fedi (i do consume content from both), but it’s pretty frustrating to see BlueSky being argued into a centralized service for the recent downtime.
- The downtime was not relay level, but it was a PDS level. So the point is moot already.
- Because it was decentralized at the PDS level, the outage did not affect anyone with personal PDSes, which contains the data that you care about.
- Even if it was the relay level, relays aren’t centralized, anyone can spin up another relay (because everything on the relay is derived from PDS data). It’s just that it’s going to be pretty expensive and consume much resources. Which is a fair point, and might be argued that the network currently has a single big point of failure, but that doesn’t mean it’s centralized.
And then people now start arguing that the fact that BlueSky-hosted PDSes went down at the same time is now another proof that BlueSky is centralized?
That’s like arguing that Gmail can go down and all @gmail.com mail addresses won’t work, so email is centralized. Or AWS can have an outage and all AWS-powered websites will break down, so the web is centralized.
One can say that there’s a single big point of failure (which the BlueSky LLC is, just like AWS, Gmail, or the mastodon.social instance in the case of Fedi), but that doesn’t make the whole service centralized.
- The downtime was not relay level, but it was a PDS level. So the point is moot already.
- Because it was decentralized at the PDS level, the outage did not affect anyone with personal PDSes, which contains the data that you care about.
- Even if it was the relay level, relays aren’t centralized, anyone can spin up another relay (because everything on the relay is derived from PDS data). It’s just that it’s going to be pretty expensive and consume much resources. Which is a fair point, and might be argued that the network currently has a single big point of failure, but that doesn’t mean it’s centralized.
And then people now start arguing that the fact that BlueSky-hosted PDSes went down at the same time is now another proof that BlueSky is centralized?
That’s like arguing that Gmail can go down and all @gmail.com mail addresses won’t work, so email is centralized. Or AWS can have an outage and all AWS-powered websites will break down, so the web is centralized.
One can say that there’s a single big point of failure (which the BlueSky LLC is, just like AWS, Gmail, or the mastodon.social instance in the case of Fedi), but that doesn’t make the whole service centralized.
Yeah lots of the commenting online seems in somewhat bad faith about this.
I’ve listened to interviews with the Bluesky CEO and the Mastodon founder (CEO?), and it was quite eye opening. They have very different views on their roles. The Bluesky CEO is thinking at the level of community building, incentives, longevity, and decentralisation at a fundamental level, whereas the Mastodon founder seems primarily motivated by building an open source project and supporting contributions from the community.
Neither approach is wrong, but they’re different and are clearly achieving different things.
Mastodon might be easier to self host on the surface, being just another Rails site, but the result isn’t a “world without caesars”, it’s a world with a lot more caesars, where anyone can be one. Migrating instances is roughly impossible if you have any presences you want to preserve, and ActivityPub compliance is essentially defined by Mastodon’s current behaviour.
Bluesky on the other hand seems to so far be far more successful at the community building and laying out the protocol foundations, at the cost of being harder to self host. But honestly, having self hosted Mastodon and switched (and forced to start again in the process), so what? Hosting a PDS is trivial and that’s the bit that really matters anyway.
Mastodon should be recognised for how much it shifted the conversation around social networking, especially in the first year after Twitter’s acquisition, but it seems clear that Bluesky has some fundamental advancements that Mastodon fans seem unable to recognise simply because it looks so different.
I’ve listened to interviews with the Bluesky CEO and the Mastodon founder (CEO?), and it was quite eye opening. They have very different views on their roles. The Bluesky CEO is thinking at the level of community building, incentives, longevity, and decentralisation at a fundamental level, whereas the Mastodon founder seems primarily motivated by building an open source project and supporting contributions from the community.
Neither approach is wrong, but they’re different and are clearly achieving different things.
Mastodon might be easier to self host on the surface, being just another Rails site, but the result isn’t a “world without caesars”, it’s a world with a lot more caesars, where anyone can be one. Migrating instances is roughly impossible if you have any presences you want to preserve, and ActivityPub compliance is essentially defined by Mastodon’s current behaviour.
Bluesky on the other hand seems to so far be far more successful at the community building and laying out the protocol foundations, at the cost of being harder to self host. But honestly, having self hosted Mastodon and switched (and forced to start again in the process), so what? Hosting a PDS is trivial and that’s the bit that really matters anyway.
Mastodon should be recognised for how much it shifted the conversation around social networking, especially in the first year after Twitter’s acquisition, but it seems clear that Bluesky has some fundamental advancements that Mastodon fans seem unable to recognise simply because it looks so different.
Most users don't care about that. Decentralized governance is more important that decentralized hosting. Wikipedia being the best example.
Didn't the Mastodon guys make their high-performance system a pay only system instead of open source?
Relay is not centralized service, plc.directory is.
Everything decentralized will become effectively centralized once it’s popular enough. Most people don’t care.
I think people are starting to care more and more. People are using Mastodon, many are switching to Bluesky believing it is decentralized.
If you teach people why they should care, I believe many actually do care. Many users just don't know enough to make a choice.
If you teach people why they should care, I believe many actually do care. Many users just don't know enough to make a choice.
Which people? The kind who browse HN?
Bluesky is only getting some attention now from people outside of tech because Musk is disliked and it’s basically the only alternative. I assure you most of them don’t know or care what “decentralized” means.
Bluesky is only getting some attention now from people outside of tech because Musk is disliked and it’s basically the only alternative. I assure you most of them don’t know or care what “decentralized” means.
I know plenty who know and care that are not in the HN crowd.
I used to not care about centralized or decentralized services, I just wanted to best product.
Now I see how each of the major social media networks has been involved in censorship of Israel's genocide in Palestine (including btw HN), it makes me realize how important decentralization is.
I think a lot of Americans (and other countries who have essentially handed over their media to US social media companies) are going to learn same lessons soon, as Trump starts to leverage the existing censorship tools more broadly.
Now I see how each of the major social media networks has been involved in censorship of Israel's genocide in Palestine (including btw HN), it makes me realize how important decentralization is.
I think a lot of Americans (and other countries who have essentially handed over their media to US social media companies) are going to learn same lessons soon, as Trump starts to leverage the existing censorship tools more broadly.
HN is primarily about tech. There is some non-tech political news, but not a lot.
Facebook is censoring pro-Palestine views? My friends must not have gotten the news.
Facebook is censoring pro-Palestine views? My friends must not have gotten the news.
Decentralization doesn't really help, it only creates more bubbles.
Reddit is extremely centralized, and you can find hundreds of subreddits that do cover Israel's ethnic cleansing. However, most popular subreddits are heavily censored, while the smaller communities become an echo chamber, and often get radicalized.
Decentralized bubbles are still bubbles—even worse, since the barrier to entry is higher.
Reddit is extremely centralized, and you can find hundreds of subreddits that do cover Israel's ethnic cleansing. However, most popular subreddits are heavily censored, while the smaller communities become an echo chamber, and often get radicalized.
Decentralized bubbles are still bubbles—even worse, since the barrier to entry is higher.
Giving users control over the bubble, rather than one company, is the important part.
Is it? How does it help with radicalization? If anything it makes it worse, since people on the extreme political spectrum will always try harder than the regulars, while companies care only about that sweet, sweet money that comes from being bland and attracting the masses.
Don't get me wrong, I care deeply about privacy and want a decentralized works, but it comes with its own cans of worms.
Don't get me wrong, I care deeply about privacy and want a decentralized works, but it comes with its own cans of worms.
The goal for distributed moderation is not to prevent radicalization, but to prevent one rich man from controlling the content people see. As a user, I can opt into whether I want to see terfs or developers or movie spoilers online or not. I think it's important for users to have more tools and agency over their online experiences.
There are other factors in play though that I think help create less polarising and radical social networks - ability to create custom homepage feeds gives to more variety and less control by the platform maker to sway audiences. Non-commercial networks (which doesn't count Bluesky) have less incentives to juice engagement. More distributed networks have levers audiences can pull if the main developers steer the network in the wrong direction.
> since people on the extreme political spectrum will always try harder than the regulars
Imagine if anyone could create an alternate feed algorithm that penalises these try-hards which people could subscribe to and use instead of the default.
There are other factors in play though that I think help create less polarising and radical social networks - ability to create custom homepage feeds gives to more variety and less control by the platform maker to sway audiences. Non-commercial networks (which doesn't count Bluesky) have less incentives to juice engagement. More distributed networks have levers audiences can pull if the main developers steer the network in the wrong direction.
> since people on the extreme political spectrum will always try harder than the regulars
Imagine if anyone could create an alternate feed algorithm that penalises these try-hards which people could subscribe to and use instead of the default.
It's a good and necessary thing to become "radicalized" in order to prevent genocide and break out from the pacifying programming of the state. the state puts pressure on centralized social media companies to enact clever methods of censorship to pacify the masses for them. such companies might even just willingly self-censor in anticipation out of obedience or just plain uncritical acceptance of state department propaganda.
So the state and the ruling class produced a radicalization of the masses themselves where the status quo is to pretend that genocide is normal - a "zone of interest" type of effect is supposed to be part of what's "normal". While I agree with you that decentralization is not a silver bullet, because it can create pockets of echo chambers, it's still a good tool for communities to be independent from certain influences like from the state or corporate culture.
So the state and the ruling class produced a radicalization of the masses themselves where the status quo is to pretend that genocide is normal - a "zone of interest" type of effect is supposed to be part of what's "normal". While I agree with you that decentralization is not a silver bullet, because it can create pockets of echo chambers, it's still a good tool for communities to be independent from certain influences like from the state or corporate culture.
I'm not singling out this kind of radicalization, but any kind, including alt-right, qanon, etc.
I'd say that inaccessible bubbles become echo chambers of people screaming into the void. The point is the be visible and heard, and federation does the opposite.
I'd say that inaccessible bubbles become echo chambers of people screaming into the void. The point is the be visible and heard, and federation does the opposite.
I wonder if maybe "radicalization" isn't as useful a term as you think.
LinkedIn's approach has been to segregate pro-Palestinian content to a bubble where it's not shown to people outside it. Now, that is believed to be because they are deliberately trying to limit pro-Palestinian liberation content due to the extremely pro-apartheid and pro-occupation beliefs of their CPO Tomer Cohen, but it's none the less pretty effective.
Use nostr. checkout primal.net
or activitypub ! Part of w3c, tons of servers and clients, even Meta runs an instance
I don't think it's as easy to run nostr relays as activitypub instances but maybe I am wrong. ActivityPub seems way more active than nostr also. Most of what I see on nostr is either cryptocurrency discussion or the kinds of alt-lite content you would find on Gab or Minds
I don't think it's as easy to run nostr relays as activitypub instances but maybe I am wrong. ActivityPub seems way more active than nostr also. Most of what I see on nostr is either cryptocurrency discussion or the kinds of alt-lite content you would find on Gab or Minds
Nostr relays are in essence extremely dumb websocket servers. The most popular relay software is nostr-rs-relay a single binary written in rust that automatically spins up an SQLite database.
Bitcoin discussion yes, but mostly because bitcoiners are avid self hosters. Cryptocurrency not much. The “crypto” people created their own thing if I’m not mistaken.
Nostr was spun out as a truly censorship resistant protocol so it’s a given people who distrusts both centralized services and island style moderation are there don’t you think?
dergigi once described Nostr with an analogy to a gay bar. The place is nice, the furniture is fancy, the drinks pleasant and the people who frequent it are mostly nice but if you are not gay you won’t stick for so long.
Nothing keeps you from creating a moderated relay or curate your feeds though.
One of the coolest things about Nostr IMO is how extendable the protocol is and how easy you can develop crazy stuff for it.
Ah and for some reason there are some Japanese users.
Bitcoin discussion yes, but mostly because bitcoiners are avid self hosters. Cryptocurrency not much. The “crypto” people created their own thing if I’m not mistaken.
Nostr was spun out as a truly censorship resistant protocol so it’s a given people who distrusts both centralized services and island style moderation are there don’t you think?
dergigi once described Nostr with an analogy to a gay bar. The place is nice, the furniture is fancy, the drinks pleasant and the people who frequent it are mostly nice but if you are not gay you won’t stick for so long.
Nothing keeps you from creating a moderated relay or curate your feeds though.
One of the coolest things about Nostr IMO is how extendable the protocol is and how easy you can develop crazy stuff for it.
Ah and for some reason there are some Japanese users.
Nostr is super flexible too since its a protocol. I see reddit-style and hacker news style sites, documentation sites, blog sites...and yes twitter/x style sites as well.
Hell there's even a streaming platform on nostr too. The common criticism is that its all bitcoiners, that's mainly because it uses Lightning Network for zaps and that's how people get paid in nostr. So naturally its a lot of bitcoiners in the nostr communities.
Hell there's even a streaming platform on nostr too. The common criticism is that its all bitcoiners, that's mainly because it uses Lightning Network for zaps and that's how people get paid in nostr. So naturally its a lot of bitcoiners in the nostr communities.
That sounds really interesting. I haven’t really kept up with nostr. Would you mind linking some examples of the types of sites you mentioned?
Thank you! What’s your favorite nostr thing you’ve seen?
I haven't looked at activitypub yet, didn't know it was a w3c rec.