Bluesky's AT Protocol - Federation Architecture Overview(blueskyweb.xyz)
blueskyweb.xyz
Bluesky's AT Protocol - Federation Architecture Overview
https://blueskyweb.xyz/blog/5-5-2023-federation-architecture
87 comments
> I still don't comprehend why Dorsey didn't just try to make the largest ActivityPub server on the web.
Have you seen the ecosystem study BlueSky published in 2020? https://gitlab.com/bluesky-community1/decentralized-ecosyste...
Bit more of a summary https://twitter.com/bluesky/status/1511811083954102273
BlueSky/AT Protocol seems to have pretty distinct design goals that are not possible with ActivityPub.
Have you seen the ecosystem study BlueSky published in 2020? https://gitlab.com/bluesky-community1/decentralized-ecosyste...
Bit more of a summary https://twitter.com/bluesky/status/1511811083954102273
BlueSky/AT Protocol seems to have pretty distinct design goals that are not possible with ActivityPub.
Yeah, I have read about the things they want to do which are mostly cryptography related and which have absolutely zero reasoning as to _why_ they can't be done with ActivityPub. Signing all your messages? Yeah, you can append Ed25519 signature chains to any ActivityPub message. You could publish your base public key when creating an account. You could make software that hides all this beneath an abstraction layer so the end-user never has to deal with it. Exporting all your data and uploading it to another server is something you can already do with Mastodon. E2EE message? Coming soon to Mastodon. It's all things you can do _very easily_ on top of a fault tolerant federated public messaging system.
ActivityPub of course uses the same cryptographic certification virtually all the web does, just TLS/HTTPS and relies on the host cert to be valid. But if you wanted to add further levels of verification like pubkeys and sigs, there's absolutely nothing stopping you.
ActivityPub of course uses the same cryptographic certification virtually all the web does, just TLS/HTTPS and relies on the host cert to be valid. But if you wanted to add further levels of verification like pubkeys and sigs, there's absolutely nothing stopping you.
> Yeah, I have read about the things they want to do which are mostly cryptography related and which have absolutely zero reasoning as to _why_ they can't be done with ActivityPub.
I guess the race is on, then. Surely ActivityPub, with its much larger user and developer base, will grab the best ideas from AT Protocol and squash this redundant newcomer.
I guess the race is on, then. Surely ActivityPub, with its much larger user and developer base, will grab the best ideas from AT Protocol and squash this redundant newcomer.
Differences in funding also matter.
Yeah, AT Protocol being "funded" by one company (Bluesky PBLLC) and ActivityPub being indirectly "funded" by multiple companies + organizations, some of them being Nextcloud GmbH, Framasoft (PeerTube), Tumblr (Yahoo).
Seems based on that variable, ActivityPub has the benefit of being more widespread, but that tends to also make changes go through slower. ActivityPub being a specification maintained by W3C also make changes extra slow, but stable at least, while AT Protocol probably won't have any stability guarantees for a long time.
Seems based on that variable, ActivityPub has the benefit of being more widespread, but that tends to also make changes go through slower. ActivityPub being a specification maintained by W3C also make changes extra slow, but stable at least, while AT Protocol probably won't have any stability guarantees for a long time.
Proponents of ActivityPub talk a lot about how flexible it is. I think that's the problem here.
Sure you could implement BlueSky-over-activitypub, but would it be usefully compatibile with mastodon-over-activitypub? If not, why add the complexity.
Sure you could implement BlueSky-over-activitypub, but would it be usefully compatibile with mastodon-over-activitypub? If not, why add the complexity.
Can't comment on the bluesky aspect, but you've touched on my frustration with ActivityPub. When people were migrating to Mastodon, I took a look at making a single-user instance that would suit my personal needs and gracefully interact with the wider ecosystem and found that in my case the capabilities of ActivityPub are irrelevant, you are tied to the Mastodon teams choices.
Felt very much like in practice it just gives you tools to make mostly-insular federated apps, rather than letting different apps interact.
Felt very much like in practice it just gives you tools to make mostly-insular federated apps, rather than letting different apps interact.
> Instead we get a haphazard rewrite of everything ActivityPub has already solved
As someone who doesn't seem to be a deep into the specifications (ActivityPub nor AT Protocol) as you are, how does ActivityPub handle people moving across servers without loosing old content and interactions? Like if I use server A but want to move to server B, how does that work in practice? How does server C who used to communicate with server A get notified that they now have to communicate with server B instead?
> The service feels too much like "Elon bought Twitter, now we need to make Twitter again"
Strange feeling considering bluesky was first announced in 2019 as a Twitter project, eventually spun out as a separate entity in 2021, and Elon didn't start the acquisition of Twitter until 2022 sometime.
As someone who doesn't seem to be a deep into the specifications (ActivityPub nor AT Protocol) as you are, how does ActivityPub handle people moving across servers without loosing old content and interactions? Like if I use server A but want to move to server B, how does that work in practice? How does server C who used to communicate with server A get notified that they now have to communicate with server B instead?
> The service feels too much like "Elon bought Twitter, now we need to make Twitter again"
Strange feeling considering bluesky was first announced in 2019 as a Twitter project, eventually spun out as a separate entity in 2021, and Elon didn't start the acquisition of Twitter until 2022 sometime.
Right now, changing accounts is implemented by the old server sending a Move activity which tells all your followers where your new account is. This doesn't handle moving any content.
But, there's nothing fundamental about ActivityPub that prevents this: https://shadowfacts.net/2023/activitypub-portable-identity/. You can absolutely have posts identified by your identity (read: domain) rather than where they're hosted. Complete and seamless moves are possible, Mastodon just doesn't implement them.
But, there's nothing fundamental about ActivityPub that prevents this: https://shadowfacts.net/2023/activitypub-portable-identity/. You can absolutely have posts identified by your identity (read: domain) rather than where they're hosted. Complete and seamless moves are possible, Mastodon just doesn't implement them.
Interesting, thanks for sharing that. This has been one of the things I felt that ATP got "right", so neat to see that ActivityPub can do it, even if Mastodon doesn't.
Interesting. So ActivityPub could support that use case, but currently doesn't. Combined with the fact that getting changes through to the specification itself (maintained by W3C), I could understand not wanting to spend the energy and effort on trying to go that route, and instead come up with something different.
Thanks for sharing your article, lots of good points in there.
Thanks for sharing your article, lots of good points in there.
The old server broadcasts a 'this person has moved'-message. Your followers will then automatically switch over to the new account.
This implies that:
- If the old server is offline or unwilling, you cannot move - Your old toots and interactions will still be tied to the old address, so you leave those behind.
There is some talk going on to find ways to mitigate this and it might be possible to fix the second issue.
The first issue is a bit trickier though. If you were to remove trust from the server, it becomes way harder to fight spam.
This implies that:
- If the old server is offline or unwilling, you cannot move - Your old toots and interactions will still be tied to the old address, so you leave those behind.
There is some talk going on to find ways to mitigate this and it might be possible to fix the second issue.
The first issue is a bit trickier though. If you were to remove trust from the server, it becomes way harder to fight spam.
From the AT Protocol FAQ:
> Our solution for portability requires both signed data repositories and DIDs, neither of which are easy to retrofit into ActivityPub. The migration tools for ActivityPub are comparatively limited; they require the original server to provide a redirect and cannot migrate the user's previous data.
So it seems that AT Protocol has a solution to that specific problem, while ActivityPub does not. So there is some differences between the protocols after all, and it might seem like they have at least one reason for creating a new protocol rather than trying to adjust ActivityPub to fit as a solution?
> Our solution for portability requires both signed data repositories and DIDs, neither of which are easy to retrofit into ActivityPub. The migration tools for ActivityPub are comparatively limited; they require the original server to provide a redirect and cannot migrate the user's previous data.
So it seems that AT Protocol has a solution to that specific problem, while ActivityPub does not. So there is some differences between the protocols after all, and it might seem like they have at least one reason for creating a new protocol rather than trying to adjust ActivityPub to fit as a solution?
We'll have to wait and see how many clients default to storing a complete data repository for AT in order to complete a migration from an offline server.
> If the old server is offline or unwilling, you cannot move - Your old toots and interactions will still be tied to the old address, so you leave those behind.
This is just like any other system that combine content delivery and identity on the same channel (e.g. web sites). Even if those 2 were separate, if your centralized identity provider is down, there is no way to update clients that depend on it, or to safely attest a new identity.
Allowing unverified servers to claim an identity or kick off a migration is a recipe for widespread hacking.
This is just like any other system that combine content delivery and identity on the same channel (e.g. web sites). Even if those 2 were separate, if your centralized identity provider is down, there is no way to update clients that depend on it, or to safely attest a new identity.
Allowing unverified servers to claim an identity or kick off a migration is a recipe for widespread hacking.
> I still don't comprehend why Dorsey didn't just try to make the largest ActivityPub server on the web.
Because that would just restart at square one? It's like we've learned nothing about centralization since the web1 era.
Because that would just restart at square one? It's like we've learned nothing about centralization since the web1 era.
Guess my post was too much of truth to handle, but I guess that’s just a proof of what I’ve said, at the end, I’m happy that social media is dying in its current form.
Who runs the BGS? It’s the most expensive part, a few “centralized” providers will run this? What if BGS injects its own filtering? Honestly this sounds like a more complicated way of implementing centralization
Effectively yes. It's roughly a giant follow bot, run by those with the resources to do so. If you don't like its filtering, I guess you'd switch to another more to your liking.
[deleted]
> The BGS handles "big-world" networking. It crawls the network, gathering as much data as it can, and outputs it in one big stream for other services to use. It’s analogous to a firehose provider or a super-powered relay node.
I think we should abandon whole idea of feed. Instead give me people/account/tag specific feed like Instagram or WhatsApp stories. With this I choose what I want to see. It would drastically reduce noise and my time on platform. The platform should be delight to use not something that is constantly trying to suck me into drama.
This would also eliminate need of Big Graph Services.
I think we should abandon whole idea of feed. Instead give me people/account/tag specific feed like Instagram or WhatsApp stories. With this I choose what I want to see. It would drastically reduce noise and my time on platform. The platform should be delight to use not something that is constantly trying to suck me into drama.
This would also eliminate need of Big Graph Services.
I think it is possible for a non-profit, non-commercial (and probably open source) social network platform to produce an 'algorithmic feed' that is some degree of 'good'. As you allude to, existing companies are incentivised to drive engagement by sucking you into some kind of drama to pump ad impressions.
I believe that a platform which doesn't have such financial incentives could produce a different spin on an algorithmic feed which isn't as unhealthy.
I also believe it should always be optional, and transparent. I probably wouldn't want to use it - I'm only interested in what the people i explicitly follow have to say.
I believe that a platform which doesn't have such financial incentives could produce a different spin on an algorithmic feed which isn't as unhealthy.
I also believe it should always be optional, and transparent. I probably wouldn't want to use it - I'm only interested in what the people i explicitly follow have to say.
That's how it works without the BGS. You don't have to use it. Follow whoever you want, see their posts. Done.
Can you also prevent the BGS from scraping your server? What if I don't want to pay the networking bill they would cause? How is the BGS going to pay for sending out the firehose to all the listeners?
It's still unclear to me how the federated economics will work out
It's still unclear to me how the federated economics will work out
Sure, why not? Block things based on the user-agent, ala robots.txt
Robots.txt isnt a blocklist; more of polite suggestions you have to hope are respected.
Google plus circles?
Feed is usually for accommodating promoted content.
Has anyone done an in-depth comparison between Locutus and Bluesky? https://github.com/freenet/locutus
Examples of what can be built on Locutus include:
- Decentralized email (with a gateway to legacy email via the @freenet.org domain)
- Decentralized microblogging (think Twitter or Facebook)
- Instant Messaging (Whatsapp, Signal)
- Online Store (Amazon)
- Discussion (Reddit, HN)
- Video discovery (Youtube, TikTok)
- Search (Google, Bing)
Bluesky is just a decentralised social media.
- Decentralized email (with a gateway to legacy email via the @freenet.org domain)
- Decentralized microblogging (think Twitter or Facebook)
- Instant Messaging (Whatsapp, Signal)
- Online Store (Amazon)
- Discussion (Reddit, HN)
- Video discovery (Youtube, TikTok)
- Search (Google, Bing)
Bluesky is just a decentralised social media.
It is my understanding that the at protocol has the building blocks for alot of these use cases as well. So while yes bluesky the app is just a micro blogging social media, the protocol should be able to do the rest.
...email is already decentralized, though.
Email is federated no? Can't really email you if gmail bans you or goes under, or your domain is taken down somehow. But maybe with a decentralized email it's possible to email pubkeys directly.
gmail email is centralized. hosting your own email server would make it decentralized? I am not sure what makes things "federated".
Because if you host your own email server, the email address has a domain or possibly IP address to point at your server. But getting domains, or getting IP addresses are both centralized.
If you gave your email address to someone, you cannot change it on their devices. Your domain registrar with DNS for the domain case, or your ISP for the IP case, can stop email deliveries to you or redirect them somewhere else.
With a decentralized email, people email "you". Not "your username" on Google's service, or your username on a domain you rent from others, and not your username on an IP you rent from a centralized authority.
If you gave your email address to someone, you cannot change it on their devices. Your domain registrar with DNS for the domain case, or your ISP for the IP case, can stop email deliveries to you or redirect them somewhere else.
With a decentralized email, people email "you". Not "your username" on Google's service, or your username on a domain you rent from others, and not your username on an IP you rent from a centralized authority.
How do you prove your you-ness without some sort of centralized authority in this case? Or, alternatively, how do I know how to send an email to only you?
I think they mean from a user perspective. As a user I can create an account with a provider(be it google or myself), but I can't migrate that account without leaving a forwarding address at the original address.
The alternative would be storing the user data on a distributed data structure(like a pki network or distributed hash table) so that it's not tied to a single provider.
The alternative would be storing the user data on a distributed data structure(like a pki network or distributed hash table) so that it's not tied to a single provider.
[deleted]
I frankly don’t care, and hope all “social media” are just like HN style, a thread about specific thing, and people discuss it, none of the crap of “content creators” or similar grifters, I also think personally youtube was ruined when it has all of those creators after it monetized the platform.
There would be nothing to post to HN without "content creators"
Same style not an identical one, where your contents are in the platform itself, not just a link, similar to old forums style where you write whatever you want and people discuss it back.
Okay? I think it’s pretty clear that a lot of people don’t want all their social media to be like Hacker News though. And that’s fine.
Disappointing that it’s not a true Hays-style AT protocol. That would have made writing clients much easier,
I think the federation part of this post is kinda ill-written. BGS is more of a proxy for messages in PDS, and not critical for federation itself. It's something additional to the federation network, but still necessary to build a user-friendly application. In other words, it belongs to application layer, which is above the federation layer. I'm pretty sure this is the intent of the design.
Fun to see this pop up here: https://staging.bsky.app/profile/newsycombinator.bsky.social...
Can't wait for the day bluesky discovers it's possible to render content for visitors who aren't logged in.
In case others are curious, it just looks like this:
https://imgur.com/a/ILAK4Ms
Not much to it really.
And yeah, for sure the only reason people outside of bluesky cannot see the content is because they're still figuring out how to actually serve image bytes over HTTP, which turned out to be a technically difficult problem that there is no prior research about.
https://imgur.com/a/ILAK4Ms
Not much to it really.
And yeah, for sure the only reason people outside of bluesky cannot see the content is because they're still figuring out how to actually serve image bytes over HTTP, which turned out to be a technically difficult problem that there is no prior research about.
It is an invite only beta on a staging domain.
Shhh we are supposed to be mad and cynical in this thread.
If you dont mind using the console apps, it seems like you are able to query from gosky and slurp all(?) of the data with the bigsky app. Im not sure if that is intended though. I dont even have an account or invite code yet -- just on my local server.
It's hard to see how this protocol would resist centralization, similar to Mastodon.
Why do you think that? Of course some actors will be larger and more popular than others, but what about the protocol inherently leads to centralization?
All the strongest network effects accrue to components outside the protocol's control or within the direct control of the Blue Sky team.
What is their plan to monetize?
Isn't the the saying: "if it's free, you're the product"?
Isn't the the saying: "if it's free, you're the product"?
Bluesky has said ads and services. Every aggregator / gateway is going to have to figure that out independently.
ads, vc, data harvesting and brokerage, managed services (hosting, infra) on a subscription model offered to orgs?
the ethical model is obviously ActivityPub tho
adding more than snark to my reply: I want to know if the PBLLC intends to submit this protocol to the IETF or W3C. We do the RFC thing for a reason folks.
the ethical model is obviously ActivityPub tho
adding more than snark to my reply: I want to know if the PBLLC intends to submit this protocol to the IETF or W3C. We do the RFC thing for a reason folks.
How do ActivityPub/Mastodon servers monetize (or stay online)? Same way?
they typically don’t. this seems to be why many of the users prefer it. i think i generally agree with them—it leads to a different style of interaction. i haven’t figured out yet how to elequently describe it, but the interactions and experience is entirely different from a for-profit site. it feels “more pure” or something (though i do recognize why description is inadequate.)
when “how do we monetize this?” becomes the driving force of a social thing, the experience changes in very real ways.
when “how do we monetize this?” becomes the driving force of a social thing, the experience changes in very real ways.
I felt that same feeling in ~2009 when I first joined Facebook, before it had ads. I think new communities in general tend to feel more "pure" in the beginning. Of course keeping that intimacy long term is the challenge.
I think that is pretty fair, but maybe I am more interested in the "keeping it running" component. Labor of love and donations can only go so far. When a env/population outpaces love and donations, what keeps the servers online? Does the decentralized nature of Mastodon minimize cost? Or has it just not reached a critical mass?
Im more interested in the Mastodon/Bluesky approach to re-decentralizing (we had thriving forums long ago). But eventually cost does add up and monetization (or a benevolent benefactor) has to be considered.
Im more interested in the Mastodon/Bluesky approach to re-decentralizing (we had thriving forums long ago). But eventually cost does add up and monetization (or a benevolent benefactor) has to be considered.
in general they're either labors of love or they run on a donation basis. it's a really good question.
They don't.
+++ ATH
They are trying to build a decentralized and open social protocol and one of the first things they have done is removing scrollbar from their pages for no reason at all. They are definitely on the right track.
BGS seems like a very costly and unnecessary system designed to keep functional centralization and central info control. Whatever interest I had in this is gone.
Does blue-sky even implement federation yet?
The article answers that question in the third sentence:
> Soon, we’re launching a sandbox environment to begin the testing phase of federation for the AT Protocol with allow-listed servers.
> Soon, we’re launching a sandbox environment to begin the testing phase of federation for the AT Protocol with allow-listed servers.
So it's vapourware
The 60,000+ of us in the Bluesky beta would disagree but sure, be flippant.
It's a work-in-progress. As I'm sure you know, you can't build and launch everything at once, so the rather small dev team has a roadmap, and is building and testing features in sequence.
When they get close to launching a new major feature, like federation they make announcements and posts about it, to encourage discussion and debate about it, and look for early adopters who are willing to test it out.
It's a work-in-progress. As I'm sure you know, you can't build and launch everything at once, so the rather small dev team has a roadmap, and is building and testing features in sequence.
When they get close to launching a new major feature, like federation they make announcements and posts about it, to encourage discussion and debate about it, and look for early adopters who are willing to test it out.
They are so insistent that most people want the entertainment use case for social media, they informed their protocol architecture design by that. No, no one wants that. Everyone is already sick of centralized social media services forcefully shoving recommended content from unknown people down their throats. They just want a chronological timeline of posts made by people they explicitly followed. No ads, no recommendations (Instagram's "mute recommended content for 30 days" is straight mockery btw), no prompts, no nothing, just updates from people you followed. And that's already solved by ActivityPub. Everything becomes much, much simpler when you don't need some kind of pool of "all posts in the universe" to choose from.
> forcefully shoving recommended content from unknown people down their throats...
Unpopular opinion for HN: I like recommend content. Especially with TikTok where the vast majority of content I consume (and I think most others) is generated by non-friends. Crafting the perfect list of who to follow is tough work so when the algorithms can do it I'm happy. Of course, the recommend content needs to be good and interesting to me. I suspect most people's frustration with it is the result of getting bad recommend content. Though for some people recommend content might be particular bad for you if your interests are more niche.
But I also agree that I wish you could turn recommend content off. If it's recommendations are bad, I agree it shouldn't be shoved down your throat, but also please don't force a recommend content free world on the rest of us.
Unpopular opinion for HN: I like recommend content. Especially with TikTok where the vast majority of content I consume (and I think most others) is generated by non-friends. Crafting the perfect list of who to follow is tough work so when the algorithms can do it I'm happy. Of course, the recommend content needs to be good and interesting to me. I suspect most people's frustration with it is the result of getting bad recommend content. Though for some people recommend content might be particular bad for you if your interests are more niche.
But I also agree that I wish you could turn recommend content off. If it's recommendations are bad, I agree it shouldn't be shoved down your throat, but also please don't force a recommend content free world on the rest of us.
You're mixing two distinct use cases here: entertainment and staying up to date on your friends' lives. I think these should be served by different services because they're so fundamentally different.
Bluesky has both: a following feed in chronological order and an algorithmic feed that shows you stuff new to you. they're also releasing custom algorithms soon so that you can plug in whatever algorithm you want instead of being limited to just theirs.
"everyone" doesn't buy ads or make political donations. After WW3, there will be only cockroaches and recommendation algos left.
> forcefully shoving recommended content from unknown people down their throats. They just want a chronological timeline of posts made by people they explicitly followed
So... You're saying you like AT Protocol/Bluesky then? Because if that's what you're trying to say, your post kind of reads like you don't like it, but then what you want is exactly what they are trying to provide...
So... You're saying you like AT Protocol/Bluesky then? Because if that's what you're trying to say, your post kind of reads like you don't like it, but then what you want is exactly what they are trying to provide...
My understanding from this and earlier blog posts is that they want an algorithmic timeline where people can choose their algorithm. Those "feed generators" are the pluggable algorithms they promised and they do need global indexes (BGS's) to be able to inject out-of-network content to your feed. It all checks out. Yes you can get a chronological feed of accounts you follow, but that seemingly isn't what they see as the most important use case.
The beauty of this being decentralized and open source is that, in theory, there would be a large number of possible algorithms (including "no algorithm") for users to choose from.
There's an earlier blog post that talks about the "algorithm marketplace" that is worth looking up.
I don't think this team "sees" any use cases for sorting feeds right now. In the current beta there are two feeds, your followers which is chronological (and the default) and "What's Hot" which is a haphazard very much placeholder timeline of the "most interacted with" posts, and I think that just means "13 likes or more, then chronological" at the moment.
It's super basic and not at all the priority for them. From all I've seen in my time in the beta, the most important use case they are working towards is "let everyone pick their own way of sorting/filtering/moderating their timeline".
There's an earlier blog post that talks about the "algorithm marketplace" that is worth looking up.
I don't think this team "sees" any use cases for sorting feeds right now. In the current beta there are two feeds, your followers which is chronological (and the default) and "What's Hot" which is a haphazard very much placeholder timeline of the "most interacted with" posts, and I think that just means "13 likes or more, then chronological" at the moment.
It's super basic and not at all the priority for them. From all I've seen in my time in the beta, the most important use case they are working towards is "let everyone pick their own way of sorting/filtering/moderating their timeline".
One of those will be chronological. That's still an algorithm...
The only market it seems to be gaining traction in from my perspective is millennials who have nostalgia for old Twitter -- and as someone who grew up with old Twitter, I can say I don't really miss it.