How Prosody developers spent 2020(blog.prosody.im)
blog.prosody.im
How Prosody developers spent 2020
https://blog.prosody.im/2020-retrospective/
41 comments
That statement doesn't reflect the sad state of XMPP clients available nowadays.
I used to run my own XMPP server for years before moving on to Matrix and the reality was that the only good client was Conversations for Android. There wasn't a single one for Linux desktop that felt modern and up to speed with the latest XEPs.
And let's not even mention trying to get multiple clients supporting the same history with encryption enabled, that was super chaotic.
I'm really glad I abandoned my XMPP server in favor of Matrix. Matrix is modern and it just works and setting up a Synpase server is way easier than setting up a Prosody server.
I used to run my own XMPP server for years before moving on to Matrix and the reality was that the only good client was Conversations for Android. There wasn't a single one for Linux desktop that felt modern and up to speed with the latest XEPs.
And let's not even mention trying to get multiple clients supporting the same history with encryption enabled, that was super chaotic.
I'm really glad I abandoned my XMPP server in favor of Matrix. Matrix is modern and it just works and setting up a Synpase server is way easier than setting up a Prosody server.
When was "nowadays"? Gajim and Dino are definitely as decent as any native apps available for Matrix.
Sorry you feel that way about Prosody. If you have specific feedback we're always looking to improve the setup experience.
Snikket (briefly mentioned in the post) is an attempt to bundle everything you need for a modern communication server into a single package (e.g. automatic certs and stuff that you need for audio /video) out of the box. Maybe it sounds like it would suit you more.
Sorry you feel that way about Prosody. If you have specific feedback we're always looking to improve the setup experience.
Snikket (briefly mentioned in the post) is an attempt to bundle everything you need for a modern communication server into a single package (e.g. automatic certs and stuff that you need for audio /video) out of the box. Maybe it sounds like it would suit you more.
I don't feel negative towards Prosody, Prosody is an awesome project.
My criticism is about XMPP in general and its ecosystem.
I stopped my XMPP server a few years ago, Dino was still in its infancy. It's good to see that the client is reaching some maturity but then, looking at the supported XEPs I can see that MAM is not supported for group chats:
- https://github.com/dino/dino/wiki/Supported-XEPs
Fortunately Matrix and the respective clients have been evolving quite well, so I don't see the need of going back to XMPP.
My criticism is about XMPP in general and its ecosystem.
I stopped my XMPP server a few years ago, Dino was still in its infancy. It's good to see that the client is reaching some maturity but then, looking at the supported XEPs I can see that MAM is not supported for group chats:
- https://github.com/dino/dino/wiki/Supported-XEPs
Fortunately Matrix and the respective clients have been evolving quite well, so I don't see the need of going back to XMPP.
I stopped my XMPP server some 7..9 years ago after seeing loads of spam in MUCs.
The fact that Gajim is the best desktop XMPP client says more about how bad desktop XMPP clients are than how good gajim is. Dino is just very recently getting good enough to use, but it seems like it will be too little, too late.
And that's without talking about the elephant in the room of iOS. 5 years ago my workplace moved from XMPP to mattermost. There was a list of features desired by people; 7/8 could have been solved by updating ejabberd and telling people to switch clients, but iOS support meant switching away.
And that's without talking about the elephant in the room of iOS. 5 years ago my workplace moved from XMPP to mattermost. There was a list of features desired by people; 7/8 could have been solved by updating ejabberd and telling people to switch clients, but iOS support meant switching away.
How come there hasn’t been a single really good implementation of it (besides defunct google chat), with presence, synchronized read receipts etc in the past decade?
There are several Matrix projects that are quite full-featured and were built by a ~single person. Leveraging an appropriate SDK makes it feasible.
If you use an SDK, you're not implementing the protocol or any XEP, and therefore it's not a testament to how easy that is to do.
That's true, but then again, if everyone uses a small set of SDKs that are robust and known to be compliant, isn't that a strictly better situation? Why is it important to reimplement the standard itself from scratch?
Most of them in alpha state, with some functionality - read not all XEP implemented ;)
Sure, but that is not unlike the XMPP/XEP situation. That's pretty much unavoidable and comes down to the responsibility of the client. It's natural that there will only be a few full-featured clients since those require a lot of effort with either Matrix or XMPP.
That's exactly what I'm saying. XMPP got attacked a lot because of how XEP support was lacking in certain clients, but was implemented in others, yet nearly every federated system out there has the exact same, inevitable problem - email, Matrix, ActivityPub are no exceptions.
I think XMPP got attacked not because of the fragmented ecosystem of clients, but because the standard itself (in the sense of the complete set of XEPs in existence) got fragmented. As far as I know, multiple competing/overlapping XEPs were not an uncommon situation.
Compared to this, the Matrix standard(s) have thus far been able to maintain a certain kind of coherence (IMO), allowing for at least a vision of what a complete implementation of a server or client should look like. The standard is versioned so that you can aim for support of a particular version instead of a pick-and-choose subset of functionality. Also, part of the Matrix's coherence story probably lies in the fact that there is an "official" set of clients in the form of the Element clients.
Ultimately, you cannot prevent others from writing incomplete servers or clients. But you can show what the goal looks like and have other implementations approach it asymptotically.
Compared to this, the Matrix standard(s) have thus far been able to maintain a certain kind of coherence (IMO), allowing for at least a vision of what a complete implementation of a server or client should look like. The standard is versioned so that you can aim for support of a particular version instead of a pick-and-choose subset of functionality. Also, part of the Matrix's coherence story probably lies in the fact that there is an "official" set of clients in the form of the Element clients.
Ultimately, you cannot prevent others from writing incomplete servers or clients. But you can show what the goal looks like and have other implementations approach it asymptotically.
There have been significant efforts to reduce this kind of fragmentation and to outline which XEPs a client or server developer should focus on, given a certain target application.
We are reviewing the most important specifications on a yearly base and provide recommendations in the form of Compliance Suites with different profiles like Instant Messaging, Mobile etc.:
https://xmpp.org/extensions/xep-0443.html
We are reviewing the most important specifications on a yearly base and provide recommendations in the form of Compliance Suites with different profiles like Instant Messaging, Mobile etc.:
https://xmpp.org/extensions/xep-0443.html
That's great info, I wasn't aware of this. Thanks!
> It doesn't look like an insurmountable effort from one person to support the XEPs required to create a competent chat application, unlike my impression of Matrix.
Could you explain how you got this particular impression?
Could you explain how you got this particular impression?
I'm one of them, and I love it. Desktop clients are lagging in support compared to Conversations[^1] on Android, but even Pidgin can be flogged into an acceptable state[^2] - no video or audio though, I couldn't get that working, unlike with Prosody and Conversations[^3].
The thing with XMPP vs Matrix is weight: and XMPP server is featherweight compared to Matrix due to their different nature, so I honestly believe that XMPP still has a future.
(And maybe one day, Pidgin 3 will see the light[^4])
[^1]: https://conversations.im/
[^2]: https://github.com/petermolnar/awesome-pidgin-plugins
[^3]: https://gist.github.com/petermolnar/10d815eae0b7cbda2b1e5948...
[^4]: https://pidgin.im/development/building/3.0.0/
The thing with XMPP vs Matrix is weight: and XMPP server is featherweight compared to Matrix due to their different nature, so I honestly believe that XMPP still has a future.
(And maybe one day, Pidgin 3 will see the light[^4])
[^1]: https://conversations.im/
[^2]: https://github.com/petermolnar/awesome-pidgin-plugins
[^3]: https://gist.github.com/petermolnar/10d815eae0b7cbda2b1e5948...
[^4]: https://pidgin.im/development/building/3.0.0/
XMPP is featherweight compared to Matrix because it does less things than Matrix.
Having a gigantic group chat with hundreads of people, with encryption, history and media synchronized between everyone without relying on a centralized server is something that XMPP doesn't even try to solve. Matrix does.
Prosody is a great project, I used to run my own Prosody server before moving on to Matrix. But people shouldn't lose perspective of the state of the protocol.
XMPP is okay if you plan to chat with a few people individually that are willing to spend time choising the right client with the right plugins and if you don't care about modern features but anything beyond than that and it becomes a huge headache.
Having a gigantic group chat with hundreads of people, with encryption, history and media synchronized between everyone without relying on a centralized server is something that XMPP doesn't even try to solve. Matrix does.
Prosody is a great project, I used to run my own Prosody server before moving on to Matrix. But people shouldn't lose perspective of the state of the protocol.
XMPP is okay if you plan to chat with a few people individually that are willing to spend time choising the right client with the right plugins and if you don't care about modern features but anything beyond than that and it becomes a huge headache.
We care about modern features, and are adding new ones continuously.
You're right that Matrix focuses on a distributed design where no single server is responsible for a room. XMPP doesn't focus on this (though there are XEPs for it, I don't know of any use of them outside of military deployments).
The "everything everywhere" eventual consistency approach Matrix takes means it is resilient for sure, but it's not always a desirable feature. The IETF recently trialled Matrix and were caught out by this behaviour: https://mailarchive.ietf.org/arch/msg/tools-discuss/bdGVrXm7...
It can raise questions about data ownership and retention.
I am happy both protocols exist, and bridges between them exist. For my use cases I am sticking with XMPP.
You're right that Matrix focuses on a distributed design where no single server is responsible for a room. XMPP doesn't focus on this (though there are XEPs for it, I don't know of any use of them outside of military deployments).
The "everything everywhere" eventual consistency approach Matrix takes means it is resilient for sure, but it's not always a desirable feature. The IETF recently trialled Matrix and were caught out by this behaviour: https://mailarchive.ietf.org/arch/msg/tools-discuss/bdGVrXm7...
It can raise questions about data ownership and retention.
I am happy both protocols exist, and bridges between them exist. For my use cases I am sticking with XMPP.
If you are going to post that specific email, then you should also post the mail in which Matthew Hodgson from Matrix addresses and clarifies some of the concerns there:
- https://mailarchive.ietf.org/arch/msg/tools-discuss/G4-c-2P9...
- https://mailarchive.ietf.org/arch/msg/tools-discuss/G4-c-2P9...
Yeah, for desktop I'd generally recommend Gajim (recent versions) or Dino (younger project, but progressing well). I personally use a console client (poezio).
XMPP has evolved a lot over recent years, but many folk still judge it by the state of Pidgin (where development is focusing on their big 3.0 rather than catching up with modern messaging features) or with the features XMPP had in 2006.
XMPP has evolved a lot over recent years, but many folk still judge it by the state of Pidgin (where development is focusing on their big 3.0 rather than catching up with modern messaging features) or with the features XMPP had in 2006.
> and XMPP server is featherweight compared to Matrix due to their different nature
I agree that Synapse, the only currently available Matrix server is resource heavy, but I would not extrapolate this to the future of Matrix servers.
There is still quite some optimization possible.
I agree that Synapse, the only currently available Matrix server is resource heavy, but I would not extrapolate this to the future of Matrix servers.
There is still quite some optimization possible.
Dendrite is the next big thing right? Written in Rust? I'd be curious to see how the performance is there with it being in Rust and a rewrite/new implementation that can learn from experience.
If popular rooms still can't be joined without consuming gigabytes of RAM then, I will probably give up on Matrix altogether.
Also Dendrite is Go, not Rust.
Also Dendrite is Go, not Rust.
> Also Dendrite is Go, not Rust.
Ah, my bad. Thanks.
Ah, my bad. Thanks.
I've been trying Dendrite for a few days: a single user in the system - me -, being in two, federated, medium (<1k) sized rooms on matrix.org, the memory usage runs between 250MB to 1.5GB.
I'm going to give Prosody a serious look then, thanks for the feedback.
If you have any questions, we're friendly, feel free to reach out! :)
https://prosody.im/discuss
https://prosody.im/discuss
Conduit is the Rust implementation: https://conduit.rs/
> Prosody does not “phone home” in any way, which means we do not have a lot of insight into how many people are using Prosody. But there are some indicators that we can use to see at least the growth of the project.
Great kudos for the authors for writing their software this way; this is analytics done right.
Great kudos for the authors for writing their software this way; this is analytics done right.
Absolutely love what you're doing, and I may come to you later to discuss possible commercial support and collaboration. Right now I'w working on a Prometheus to XMPP gateway with advanced filtering capabilities ( my idea is monitoring and alerting loop has to be tight , so you need means to deliver alerts using only things under your control), and XMPP is the right choice for this task, and I’m using Prosody as an alert delivery server component, with Conversation as a mobile client, and Dino on desktop.
Sounds interesting :)
Long before Prometheus was a thing I was working on a project to tunnel metrics in realtime over XMPP. I used it for some personal monitoring but never got around to polishing and publishing it.
These days it totally makes sense to bridge with Prometheus or equivalents.
Long before Prometheus was a thing I was working on a project to tunnel metrics in realtime over XMPP. I used it for some personal monitoring but never got around to polishing and publishing it.
These days it totally makes sense to bridge with Prometheus or equivalents.
I'm running ejabberd now, it's been fine for me. Years ago I found it much easier to get all the XEPs conversations wanted working with it. Everything I needed was built into ejabbered and not all the prosody modules I needed were packaged for debian.
What is the landscape like now? I want to be convinced to switch, what does prosody do better?
What is the landscape like now? I want to be convinced to switch, what does prosody do better?
I'm a Prosody developer. If you're happy with ejabberd, that's fine by me! It used to be a real pain to set up and maintain (one reason we started Prosody), but both ejabberd and Erlang have made progress in that area recently.
Prosody has a strong community and one of our priorities is keeping stable, lightweight, and easy to extend and experiment with.
We are aware that some useful/popular modules are still in the community repository. We polish and merge new ones with every major release. As detailed in the blog post we are adding a simple way to fetch and install community modules to a Prosody installation.
Contrary to popular belief, XMPP is always evolving and adding new features, so this is our way of ensuring you can always keep up with new stuff even if it's not in a release yet.
We have a chat, feel free to hang out and ask questions: https://prosody.im/discuss
Prosody has a strong community and one of our priorities is keeping stable, lightweight, and easy to extend and experiment with.
We are aware that some useful/popular modules are still in the community repository. We polish and merge new ones with every major release. As detailed in the blog post we are adding a simple way to fetch and install community modules to a Prosody installation.
Contrary to popular belief, XMPP is always evolving and adding new features, so this is our way of ensuring you can always keep up with new stuff even if it's not in a release yet.
We have a chat, feel free to hang out and ask questions: https://prosody.im/discuss
Huh, cool!
Are all Prosody modules pure-lua, or can/do some have a C/binary component to them? I note that the API documentation describes only lua bits, at least.
Are all Prosody modules pure-lua, or can/do some have a C/binary component to them? I note that the API documentation describes only lua bits, at least.
They are generally pure Lua, but they can load binary libraries. Some existing such libraries can be found in most distro repositories or on luarocks.org. Lua also has a well designed API in case you need to write your own bindings to something.
If you have questions about how to do something specific we're very approachable and happy to share our knowledge :)
If you have questions about how to do something specific we're very approachable and happy to share our knowledge :)
Prosody is also used by the Jitsi videoconferencing solution.
I think a nice, lean-but-glossy desktop client (we already have Conversations on mobile) would be really good for the platform. It's a lot easier to fight network effects if the end-user-experience is tangibly better (and you can do a LOT better than the current incumbents that are Slack and Discord.)