How should group chats work in decentralized systems?(marindedic.com)
marindedic.com
How should group chats work in decentralized systems?
https://marindedic.com/groups/
34 comments
One option that you sort-of mentioned but missed: go with the static groups, but don’t let the users feel that.
In other words, show the kick/invite options to users when it does happen, but destroy and create a new group behind the scenes.
In other words, show the kick/invite options to users when it does happen, but destroy and create a new group behind the scenes.
I understand the vision, but I would still have to rotate keys through different groups. It doesn't solve anything, it just gives the illusion of a clean group delete-then-rebuild
This is a nice little write up and I kinda feel like the author (sensibly) chose centralization just on a smaller scale. I also think that the algorithm is pretty similar to the og textsecure2[1] protocol signal used (and still uses?) in terms of key generation. It's different in that messages are in a distributed hash table instead of sent through a server and also that there's less cross-verification by chat members, but I'm not sure the author would lose any of their goals by using the signal approach (with distributed storage).
[1] https://signal.org/blog/private-groups/
[1] https://signal.org/blog/private-groups/
The issue with that is that when there is no "leader", there is also no way to guarantee kicking someone out. Signal didn't have the kick option for years, and they only added it once they moved the group state management to the server. Now, is "kicking" a good enough justification to go with the leadership route? That is up for debate...
> The issue with that is that when there is no "leader", there is also no way to guarantee kicking someone out.
Why is that an issue? It's a fundamental fact about the world that your software will never address. No matter what options you purport to provide, you can't stop people from telling other people what messages they received.
In a decentralized system, messages are sent to a list of recipients. If you don't want someone to receive your message, you can take them off the list of recipients that you send to. But if you send a message to party B, and they recommunicate it to party C, there's nothing you can do about that. The only solutions are (1) to stop communicating with people you don't trust; or (2) to have the guy you want to kick out of the chat group kicked out of the world.
Why is that an issue? It's a fundamental fact about the world that your software will never address. No matter what options you purport to provide, you can't stop people from telling other people what messages they received.
In a decentralized system, messages are sent to a list of recipients. If you don't want someone to receive your message, you can take them off the list of recipients that you send to. But if you send a message to party B, and they recommunicate it to party C, there's nothing you can do about that. The only solutions are (1) to stop communicating with people you don't trust; or (2) to have the guy you want to kick out of the chat group kicked out of the world.
But notice your own option: "stop communicating with people you don't trust." For a
group, kick is an option: everyone stops including X, at once.
And that's where the leader matters -> to make that option executable as a group. Without an agreed authority, it's N separate choices that have to stay consistent forever: one member with a stale roster keeps X in the loop by accident, or X kicks Y while Y kicks X and there are two rosters claiming to be the group. One signed kick says "we stopped talking to X" for everyone.
But then again, you're right. If 4 members want to kick out member#6, but member#5 doesn't want to, there is nothing we can do to stop member#5 from sending everything to member#6. That's not a software-solvable problem.
And that's where the leader matters -> to make that option executable as a group. Without an agreed authority, it's N separate choices that have to stay consistent forever: one member with a stale roster keeps X in the loop by accident, or X kicks Y while Y kicks X and there are two rosters claiming to be the group. One signed kick says "we stopped talking to X" for everyone.
But then again, you're right. If 4 members want to kick out member#6, but member#5 doesn't want to, there is nothing we can do to stop member#5 from sending everything to member#6. That's not a software-solvable problem.
I'm curious what the motivation is for the protocol being decentralized?
Being serverless is cool and all, but what makes it worth all the extra complexity and effort?
To me the future of decentralized group chats is focused on what's missing from platforms like WhatsApp, Telegram and WeChat: - Verifiable Sender and Conversation Privacy - Censorship Resistance / Permission-less operation - User ownership of their own messaging (no application lock-in) - Harvest Now, Decrypt Later (HNDL) protections
While maintaining a similar feature set: - Efficient scalable groups - Recovery from compromised/stolen devices - Accessible UX for non-technical users
All design work is managing trade offs, however this approach appears to make some serious concessions:
Sender & Social Graph Privacy: Queries in a DHT (like Kademlia) bind the requested key to the requester's IP/Node ID. This would allow the network to be able to reconstruct the social graph, and unmask group membership by associating who is querying for who.
Compromise Recovery: Users are unable to rotate their own keys without the creator being online. In the event a device/account is compromised, a user has no mechanism to stop attackers from posting messages on their behalf.
Additionally with the group size limits, lack of forward secrecy/post compromise security -
My question is, what makes this particular path worth it?
Being serverless is cool and all, but what makes it worth all the extra complexity and effort?
To me the future of decentralized group chats is focused on what's missing from platforms like WhatsApp, Telegram and WeChat: - Verifiable Sender and Conversation Privacy - Censorship Resistance / Permission-less operation - User ownership of their own messaging (no application lock-in) - Harvest Now, Decrypt Later (HNDL) protections
While maintaining a similar feature set: - Efficient scalable groups - Recovery from compromised/stolen devices - Accessible UX for non-technical users
All design work is managing trade offs, however this approach appears to make some serious concessions:
Sender & Social Graph Privacy: Queries in a DHT (like Kademlia) bind the requested key to the requester's IP/Node ID. This would allow the network to be able to reconstruct the social graph, and unmask group membership by associating who is querying for who.
Compromise Recovery: Users are unable to rotate their own keys without the creator being online. In the event a device/account is compromised, a user has no mechanism to stop attackers from posting messages on their behalf.
Additionally with the group size limits, lack of forward secrecy/post compromise security -
My question is, what makes this particular path worth it?
Not the author, but to me the main benefit of decentralisation will always be the stability of independence from organisations. If any aspect is centralised, you are dependent on the provider of that service. What if they go out of business, what if the EU make encrypted chat illegal, what if they get coerced by the state to hand over information? An entire class of security issues disappear when you are not dependent on a third party, particularly one that with be subject to state regulation and interference.
Everything on your list except one is possible with servers + crypto.
The exception is permissionless operation. Any server-based design has an "operator" who can be compromised. The network should outlive the operator, function without it. Infrastructure servers that I set up (in order to reduce user onboarding friction) shut down in about 2 weeks. The app will keep working without me.
Regarding the extra complexity - the design was "no privileged party at all", the price is the complexity.
You're right about the DHT metadata -> queries expose requester IP to nodes on the path, so designing a social graph is possible. However, that is exactly why there are two modes:
- fast mode, which basically trades metadata privacy for lower latency and calls - anonymous routes everything (DHT queries also) over Tor
Regarding compromise recovery, you're also correct. That is on top of the v2 list.
> Additionally with the group size limits, lack of forward secrecy/post compromise security Direct (1:1) chats rotate keys every 15 messages - I thought about a similar approach for group chats, but it turned out to be very noisy, also a v2 feature to address.
You mentioned some valid flaws, but none of them seem fundamental/unsolvable. Of course, there is going to be a certain kind of trade-off when going fully decentralized, but these trade-offs are becoming smaller and smaller each day. In return, we are getting our privacy back. There is still a long way to go regarding the things that you mentioned, but also some basic UX: - Mobile app - Anonymous mode Tor alternative (thought about I2P, but it's very slow) - Calls in anonymous mode ...
Regarding the extra complexity - the design was "no privileged party at all", the price is the complexity.
You're right about the DHT metadata -> queries expose requester IP to nodes on the path, so designing a social graph is possible. However, that is exactly why there are two modes:
- fast mode, which basically trades metadata privacy for lower latency and calls - anonymous routes everything (DHT queries also) over Tor
Regarding compromise recovery, you're also correct. That is on top of the v2 list.
> Additionally with the group size limits, lack of forward secrecy/post compromise security Direct (1:1) chats rotate keys every 15 messages - I thought about a similar approach for group chats, but it turned out to be very noisy, also a v2 feature to address.
You mentioned some valid flaws, but none of them seem fundamental/unsolvable. Of course, there is going to be a certain kind of trade-off when going fully decentralized, but these trade-offs are becoming smaller and smaller each day. In return, we are getting our privacy back. There is still a long way to go regarding the things that you mentioned, but also some basic UX: - Mobile app - Anonymous mode Tor alternative (thought about I2P, but it's very slow) - Calls in anonymous mode ...
Appreciate the clarity. It helps me focus without going off on a wild tangent. The durability/independence of the protocol being a primary driver makes sense, and aligns with the design.
> "no privileged party at all" > "The network should outlive the operator, function without it." I also believe these are absolute requirements for the future of groupchats. I'd argue for stronger requirements, but I'm definitely onboard here.
This protocol has strengths: 1) Per-sender buckets. This is a clever workaround for the downsides of client-side fanout, and provides reasonable sender identity decoupling. 2) Realistic membership limits. I think the number is quite low, but in general I love the willingness to draw a reasonable Schelling fence here. My criticism of MLS is that focusing on transport-level security in a 10k-person group chat seems rather moot — as the membership count grows, the largest risk comes from "inside" the group. Drawing a reasonable limit here, and basing scalability around it, makes sense. 3) It doesn't accept the status quo. Most protocols attempt to solve the whole world. Re-evaluating what's actually required from a feature set is a great approach to reducing complexity. 4) Static epoch key avoids the issues with state desynchronization due to dropped messages. Assuming the pairwise channel is reliable, this removes a whole host of problems. It's a security tradeoff, but I see the value.
> but none of them seem fundamental/unsolvable
I think this really depends on your requirements for privacy, throughput, and acceptable threat level. If the privacy and security models are targeted at financially motivated corporations selling data, then privacy may be adequate. Under an active attacker, I think some blockers start to emerge. It's hard to tell without a full spec, though, so I'll admit I could be fully off-base here.
You are correct to point out that holding a high security/privacy standard results in some complexity/performance trade off. In general I believe there is a baseline of privacy/security requirements which are "table-stakes" for modern messaging landscapes.
A) My team evaluated a similar approach for our latest messaging protocol; however, reaching group consensus through a single actor was a blocker to users recovering from compromise in a reasonable timeframe. With a group creator potentially offline for extended periods, there's no means to have a rapid response to a compromise. It may work depending on the model, but we were forced to pursue a model with multiple "creator" roles to address this.
B) Over the lifecycle of a group, the lack of forward secrecy would be a blocker to most definitions of privacy. Long-term confidentiality would be at risk given the inevitable compromise of keys. The real risk here is HNDL attacks, especially since messages are being broadcast over libp2p's DHT. One benefit of the construction cleverly alleviates the immediate need for post-quantum encryption, but without FS, harvested messages are still at risk from leaked keys.
C) Sender privacy is more important than recipient privacy. Posting outbound messages to an append log leaks significant metadata: users' online frequency and activity. In the face of timing or correlation attacks, this data would be catastrophic. From a design POV, it's hard to imagine workarounds (but very interested in potential designs).
D) Poll-based querying of messages `groups x members x not_closed_epochs` is hard to manage from a bandwidth and privacy perspective. If you assume 200 group chats (modest by current standards) with 10 members, that's 2000 queries every time a connection is dropped. Let's assume the requester identity is sufficiently masked; the query fingerprint will still be unique for each participant, leaking metadata/identity. While perhaps insignificant on its own, in aggregate this leads to complete unmasking and social graph reconstruction over time.
As you mentioned, there is "not free lunch" however ABCD are what I would consider baseline requirements for the future of groupchats, even if that means adding technical complexity.
> In return, we are getting our privacy back.
I'd suggest that this approach actually makes privacy worse. The query-based approach to message discovery will always leak some metadata through action. The cover traffic required to maintain k-anonymity pushes bandwidth utilization into unrealistic territory.
There's a school of thought that Moxie touches on in his [The Ecosystem is Moving](https://www.youtube.com/watch?v=DdM-XTRyC9c) talk (hotly debated in these circles). Given the same metadata, a user is better off trusting one centralized entity(X) rather than broadcasting it to the entire network(X+Everyone). Decentralization is a requirement for permissionlessness, censorship resistance, and data ownership; but if the only goal is "Privacy" it's categorically harmful. Minimizing metadata should be a priority for all privacy-focused protocols, centralized ones included.
> "no privileged party at all" > "The network should outlive the operator, function without it." I also believe these are absolute requirements for the future of groupchats. I'd argue for stronger requirements, but I'm definitely onboard here.
This protocol has strengths: 1) Per-sender buckets. This is a clever workaround for the downsides of client-side fanout, and provides reasonable sender identity decoupling. 2) Realistic membership limits. I think the number is quite low, but in general I love the willingness to draw a reasonable Schelling fence here. My criticism of MLS is that focusing on transport-level security in a 10k-person group chat seems rather moot — as the membership count grows, the largest risk comes from "inside" the group. Drawing a reasonable limit here, and basing scalability around it, makes sense. 3) It doesn't accept the status quo. Most protocols attempt to solve the whole world. Re-evaluating what's actually required from a feature set is a great approach to reducing complexity. 4) Static epoch key avoids the issues with state desynchronization due to dropped messages. Assuming the pairwise channel is reliable, this removes a whole host of problems. It's a security tradeoff, but I see the value.
> but none of them seem fundamental/unsolvable
I think this really depends on your requirements for privacy, throughput, and acceptable threat level. If the privacy and security models are targeted at financially motivated corporations selling data, then privacy may be adequate. Under an active attacker, I think some blockers start to emerge. It's hard to tell without a full spec, though, so I'll admit I could be fully off-base here.
You are correct to point out that holding a high security/privacy standard results in some complexity/performance trade off. In general I believe there is a baseline of privacy/security requirements which are "table-stakes" for modern messaging landscapes.
A) My team evaluated a similar approach for our latest messaging protocol; however, reaching group consensus through a single actor was a blocker to users recovering from compromise in a reasonable timeframe. With a group creator potentially offline for extended periods, there's no means to have a rapid response to a compromise. It may work depending on the model, but we were forced to pursue a model with multiple "creator" roles to address this.
B) Over the lifecycle of a group, the lack of forward secrecy would be a blocker to most definitions of privacy. Long-term confidentiality would be at risk given the inevitable compromise of keys. The real risk here is HNDL attacks, especially since messages are being broadcast over libp2p's DHT. One benefit of the construction cleverly alleviates the immediate need for post-quantum encryption, but without FS, harvested messages are still at risk from leaked keys.
C) Sender privacy is more important than recipient privacy. Posting outbound messages to an append log leaks significant metadata: users' online frequency and activity. In the face of timing or correlation attacks, this data would be catastrophic. From a design POV, it's hard to imagine workarounds (but very interested in potential designs).
D) Poll-based querying of messages `groups x members x not_closed_epochs` is hard to manage from a bandwidth and privacy perspective. If you assume 200 group chats (modest by current standards) with 10 members, that's 2000 queries every time a connection is dropped. Let's assume the requester identity is sufficiently masked; the query fingerprint will still be unique for each participant, leaking metadata/identity. While perhaps insignificant on its own, in aggregate this leads to complete unmasking and social graph reconstruction over time.
As you mentioned, there is "not free lunch" however ABCD are what I would consider baseline requirements for the future of groupchats, even if that means adding technical complexity.
> In return, we are getting our privacy back.
I'd suggest that this approach actually makes privacy worse. The query-based approach to message discovery will always leak some metadata through action. The cover traffic required to maintain k-anonymity pushes bandwidth utilization into unrealistic territory.
There's a school of thought that Moxie touches on in his [The Ecosystem is Moving](https://www.youtube.com/watch?v=DdM-XTRyC9c) talk (hotly debated in these circles). Given the same metadata, a user is better off trusting one centralized entity(X) rather than broadcasting it to the entire network(X+Everyone). Decentralization is a requirement for permissionlessness, censorship resistance, and data ownership; but if the only goal is "Privacy" it's categorically harmful. Minimizing metadata should be a priority for all privacy-focused protocols, centralized ones included.
Based on what I understand about your use case, I'd suggest considering the following if looking to improve the security/privacy posture.
- A Hash style ratchet still works in your decentralized construction; Users ratchet a lamport timestamp, hash ratchet the next key. It still allows multiple messages to be sent with the same key if messages are sent simultaneously, but it's better than the approach where every message uses the same key. Ref: B
- There is real work on making Decentralized MLS work without requiring classic servers: (Disclosure: Not my work, but affiliated) https://lip.logos.co/anoncomms/raw/decentralized-mls-offchai.... It's far from ready, and It definitely adds more complexity, but provides the same server-less architecture without sacrificing on security. Ref: A+B+C
- Push not Pull: Query based interfaces are hard to protect from unmasking attacks. PubSub based peer to peer networks decouple the intended recipient from a message - Messages are broadcast which provides the necessary ambiguity about who the intended recipient is. Combined with P2P syncing for message durability. Ref: D
As you said, its all tradeoffs ¯\_(ツ)_/¯
- A Hash style ratchet still works in your decentralized construction; Users ratchet a lamport timestamp, hash ratchet the next key. It still allows multiple messages to be sent with the same key if messages are sent simultaneously, but it's better than the approach where every message uses the same key. Ref: B
- There is real work on making Decentralized MLS work without requiring classic servers: (Disclosure: Not my work, but affiliated) https://lip.logos.co/anoncomms/raw/decentralized-mls-offchai.... It's far from ready, and It definitely adds more complexity, but provides the same server-less architecture without sacrificing on security. Ref: A+B+C
- Push not Pull: Query based interfaces are hard to protect from unmasking attacks. PubSub based peer to peer networks decouple the intended recipient from a message - Messages are broadcast which provides the necessary ambiguity about who the intended recipient is. Combined with P2P syncing for message durability. Ref: D
As you said, its all tradeoffs ¯\_(ツ)_/¯
@Realman78 Have you looked into how SimpleX does it?
https://github.com/simplex-chat/simplex-chat/blob/stable/doc...
https://github.com/simplex-chat/simplex-chat/blob/stable/doc...
I've looked now... We have a similar design regarding the leadership roles, with the difference that they support multiple "leaders". However, they use a full-mesh which means sending a message to each user separately. Kiyeovo has "anonymous" mode which routes traffic through Tor. That means that even offline messages that get saved to DHT get routed through Tor. They are already slow as it is when sending 1, now imagine sending 9... Thanks for the suggestion though!
There's a truly decentralized alternative to MLS:
Key Agreement for Decentralized Secure Group Messaging with Strong Security Guarantees [0]
[0] https://eprint.iacr.org/2020/1281
Key Agreement for Decentralized Secure Group Messaging with Strong Security Guarantees [0]
[0] https://eprint.iacr.org/2020/1281
Decentralization is not really a feasible option when you have more than one actors. Either you embed the centralization from beginning with some good and verifiable contracts or a certain majority is going to hijack the platform and act as centralized controllers.
Sure, but what is there to hijack in a messenger platform? The groups basically act as their own separate islands, and everything is signed for their buckets. Worst thing that an attacker can do is hurt availability
People are generally really smart and resourceful. Specially the ones on the hacking side, boils down to whats available to take advantage of.
Yes, and that's what I'm saying - the attach vector is small. There isn't much on the network to take advantage of. It's basically one giant value-encrypted lookup table
Like Bitchat https://news.ycombinator.com/item?id=48852405
I am not sorry for commenting that a public blockchain is not a server and it would satisfy this use case.
The hardest problem is social. Who is going to use this?
Probably not a ton of people in the largely-peaceful, largely-comfortable world that many people here have only ever experienced.
But history (and world-awareness) shows that those periods don't last forever, so having mature decentralized technology ready and warm for periods of crisis or devolotion is hugely valuable in the long term. It can be hard to maintain commitment to maturing and seeding that kind of technology when there's not yet a pressing need, exactly because it's hard to gain enough traction to overcome the relative inconveniences. It's admirable and important work regardless.
But history (and world-awareness) shows that those periods don't last forever, so having mature decentralized technology ready and warm for periods of crisis or devolotion is hugely valuable in the long term. It can be hard to maintain commitment to maturing and seeding that kind of technology when there's not yet a pressing need, exactly because it's hard to gain enough traction to overcome the relative inconveniences. It's admirable and important work regardless.
Thank you!
I agree, and since there is no mobile version, this won't replace your whatsapp, and it was never designed for that. The actual people I see using this:
- People who want anonymous messaging (I realize that there are already Tor messengers, so the idea was to make this one much more feature rich)
- Friend groups that want private group chats without any central dependencies or accounts
- security, self-hosting, decentralization and open-source enthusiasts
Like...email? Usenet?
Possibly very much like Usenet. In the 80s there was a distributed "Slack-like" (for lack of a better description) distributed BBS network called Citadel (still some sites around now Internet based).
I wrote a UUCP clone and UseNet gateway for CitNet and made it available through a Citadel BBS "newsreader" to callers.
"uccico unidel ken bitsko macleod" still brings up some results[0]
[0] https://groups.google.com/g/alt.bbs/c/QFvfoxxONGU
I wrote a UUCP clone and UseNet gateway for CitNet and made it available through a Citadel BBS "newsreader" to callers.
"uccico unidel ken bitsko macleod" still brings up some results[0]
[0] https://groups.google.com/g/alt.bbs/c/QFvfoxxONGU
Sure, the offline delivery part can be thought of as e-mail-ish. Is that what you're going for?
Not sure why you're downvoted but email came to mind too: can't group chat be compared to an email thread with the same set of recipients throughout?
I did want to point out that Matrix does do distributed eventually-consistent authorization, which is their key invention IMHO. (Rooms are distributed among the homeservers, none of which are privileged over the others. You could (and their long-term plan from back in the day) was to run a tiny little single-device homeserver on every device to achieve P2P.)
It's tricky, but a very cool algorithm! Several entities (including myself as a hobby project) are working in combining the Matrix eventually-consistent CRDT with MLS for encryption for a no-compromise distributed E2EE system. It's possible, but very hard, as you might imagine.
Edit: Here's one academic paper writing up the abstract algorithm behind Matrix https://dl.acm.org/doi/10.1145/3381991.3395399