Running an IRC Network in 2019: Challenges and Opportunities(darenet.org)
darenet.org
Running an IRC Network in 2019: Challenges and Opportunities
https://www.darenet.org/blog/running-an-irc-network-in-2019-challenges-and-opportunities
35 comments
I've also run one, and nowadays it is more like a bridge between Slack and Discord channels (there is even a dedicated API for such use cases, much like "bot" accounts in those proprietary services). Given that those services generally refuse to cooperate with each other, this might be an interesting opportunity for IRC.
Interesting to see how this compares to how we run things for EsperNet.
- We don't use much cloud technology at present. That means we run our own DNS nameservers (using PowerDNS and a single MySQL node - AXFR and slave nameservers give us redundancy [1]). We implement Let's Encrypt renewals on top of this using the DNS challenge and have some custom software to securely push certificates to the IRC servers. This keeps costs down.
- We've replaced our old Iris webchat with The Lounge in public mode, which is fully containerised.
- We have some additional infrastructure for staff: our LDAP directory, an IdP which runs on top and provides SAML & OIDC auth, Git for code and some internal web services for pooling/depooling servers and monitoring.
- Some of the staff have written services and IRCd modules to help control spam and support connection limit exemptions (for e.g. bouncers).
[1] This allowed us to continue operating with no user-visible impact when we had a hardware failure on the primary DNS box.
- We don't use much cloud technology at present. That means we run our own DNS nameservers (using PowerDNS and a single MySQL node - AXFR and slave nameservers give us redundancy [1]). We implement Let's Encrypt renewals on top of this using the DNS challenge and have some custom software to securely push certificates to the IRC servers. This keeps costs down.
- We've replaced our old Iris webchat with The Lounge in public mode, which is fully containerised.
- We have some additional infrastructure for staff: our LDAP directory, an IdP which runs on top and provides SAML & OIDC auth, Git for code and some internal web services for pooling/depooling servers and monitoring.
- Some of the staff have written services and IRCd modules to help control spam and support connection limit exemptions (for e.g. bouncers).
[1] This allowed us to continue operating with no user-visible impact when we had a hardware failure on the primary DNS box.
> Unfortunately, no workable solution for mobile connections has been found yet.
I've come to the conclusion that trying to get the irc protocol to handle this is a dead end. A better solution would be to make the server handle multiple protocols. I'm using quassel for over a year now, and it makes irc usable from mobile. It does so by being split into a core and a GUI, so basically like a bouncer, but the protocol between them is designed from scratch, to handle connection drops and even connecting with multiple quassel clients to a single core while syncing up everything properly.
Now the big question would be if it were feasible to support the quassel protocol directly on the irc server, in parallel to irc. This way the hardcore irc fans can keep using it the way they always did, while at the same time making it easy for the kids these days to get started, by not having to set up a quassel core somewhere first. Just grab the app and go.
I've come to the conclusion that trying to get the irc protocol to handle this is a dead end. A better solution would be to make the server handle multiple protocols. I'm using quassel for over a year now, and it makes irc usable from mobile. It does so by being split into a core and a GUI, so basically like a bouncer, but the protocol between them is designed from scratch, to handle connection drops and even connecting with multiple quassel clients to a single core while syncing up everything properly.
Now the big question would be if it were feasible to support the quassel protocol directly on the irc server, in parallel to irc. This way the hardcore irc fans can keep using it the way they always did, while at the same time making it easy for the kids these days to get started, by not having to set up a quassel core somewhere first. Just grab the app and go.
I've been extremely happy with The Lounge (https://thelounge.chat/) as a modern, web-based IRC client. It's successfully replaced a decade of ZNC and WeeChat acting as bouncers for irssi, Textual, and X-Chat.
- It's a PWA, so I can add it to my homescreen on Android just like any other native app (and I find the UX better than most native Android IRC clients).
- It supports Web Push notifications, so I find out when I've been mentioned, even without the client running locally.
- It remembers and restores state across restarts.
...and it turns out that's basically all I wanted. I wonder if the network hosting its own instance of such a capable client would be enough for new users to get going?
- It's a PWA, so I can add it to my homescreen on Android just like any other native app (and I find the UX better than most native Android IRC clients).
- It supports Web Push notifications, so I find out when I've been mentioned, even without the client running locally.
- It remembers and restores state across restarts.
...and it turns out that's basically all I wanted. I wonder if the network hosting its own instance of such a capable client would be enough for new users to get going?
>...and it turns out that's basically all I wanted. I wonder if the network hosting its own instance of such a capable client would be enough for new users to get going?
This is what we do on EsperNet (https://webchat.esper.net/), for what it's worth - using a forked version of The Lounge with some minor changes. It works reasonably well, but we do have to run it in public mode which limits some of the advantages (e.g. the bouncer-like behaviour).
With that said, it's a well made application and is a big improvement upon CGI:IRC and Iris. xPaw and astorije are great, too.
This is what we do on EsperNet (https://webchat.esper.net/), for what it's worth - using a forked version of The Lounge with some minor changes. It works reasonably well, but we do have to run it in public mode which limits some of the advantages (e.g. the bouncer-like behaviour).
With that said, it's a well made application and is a big improvement upon CGI:IRC and Iris. xPaw and astorije are great, too.
I've been running irssi in a screen for 15 years or so, and ircII before that. I won't say it's great UX on a phone through something like JuiceSSH or Termius, but I've been accepting it as I always want/need access to my terminal on the go regardless. I might give quassel a try though, thanks!
I personally use weechat and weechat-android (https://github.com/ubergeek42/weechat-android) and it's the best Android client I've tried. It does require some technical set-up, though.
You can plug Glowing Bear into it too, if you want a web interface.
You can plug Glowing Bear into it too, if you want a web interface.
[deleted]
Containerising the IRCD would probably broaden the pool of people willing to host the network, due to not needing root on their boxes anymore.
Shameless plug for anyone who just wants their own IRCD: https://github.com/LukeB42/Psyrcd
Shameless plug for anyone who just wants their own IRCD: https://github.com/LukeB42/Psyrcd
For me, a major pain point about IRC is that I, as a user, need to use a bouncer in order to not miss messages. And even then, you still risk missing some messages if the bouncer needs to disconnect from the IRC server (e.g. due to updates to the bouncer software or to the hosting OS).
I personally see that as a feature because you can then specify what triggers you want recorded. eg just your nick? What about other phrases like an unusual hobby you might have? etc
Slack and other services have this without the bouncer
Slack doesn't support triggers in nearly as sophisticated way as IRC bounces can (believe me, I've tried).
I wasn't intended to compare IRC to Slack but since you've started us down this rabbit hole: I'd take IRC over Slack any day. Sure Slack is more "modern" and it can - in theory - do anything IRC can do. However IRC is more easily extendible and in a more versatile way. To be honest there's nothing Slack does that you couldn't also do in IRC - albeit some features might require a more modern IRC client than the terminal ones us geeks often harp on about (disclaimer: I've written tools for both IRC and Slack).
The problem with IRC faces isn't a technological one - it's just a mixture of NIH (not invented here) syndrome and capitalists wanting to make money from reinventing public domain tools as proprietary solutions with expensive subscriptions. Not that I have a problem with people wanting to make money - quite the opposite in fact - but I do take issue with people who say "this can't be done in X which is why Y exists" when the real reason is financially motivated rather than any technological shortcoming.
I wasn't intended to compare IRC to Slack but since you've started us down this rabbit hole: I'd take IRC over Slack any day. Sure Slack is more "modern" and it can - in theory - do anything IRC can do. However IRC is more easily extendible and in a more versatile way. To be honest there's nothing Slack does that you couldn't also do in IRC - albeit some features might require a more modern IRC client than the terminal ones us geeks often harp on about (disclaimer: I've written tools for both IRC and Slack).
The problem with IRC faces isn't a technological one - it's just a mixture of NIH (not invented here) syndrome and capitalists wanting to make money from reinventing public domain tools as proprietary solutions with expensive subscriptions. Not that I have a problem with people wanting to make money - quite the opposite in fact - but I do take issue with people who say "this can't be done in X which is why Y exists" when the real reason is financially motivated rather than any technological shortcoming.
Fellow IRC:ers, I salute you guys!
As a client I've been happily using https://www.irccloud.com/ for the last few years. Does what I need: decent mobile and web client.
If the problem is that Slack/Discord are proprietary, isn't the solution to build an FLOSS alternative to Slack/Discord that anyone can run a server for? Do such projects exist?
While I loved using IRC 10-15 years ago, the user experience is simply terrible by our standards today (and adding emoji reactions isn't going to solve that imo).
While I loved using IRC 10-15 years ago, the user experience is simply terrible by our standards today (and adding emoji reactions isn't going to solve that imo).
> While I loved using IRC 10-15 years ago, the user experience is simply terrible by our standards today (and adding emoji reactions isn't going to solve that imo).
The terrible user experience of slack and discord is why I started using IRC even more. emoji, images, link previews, and the like all lead to annoying/awful spam. Instead of a conversation you get a stream on memes and image macros along with link and twitter spam. I find it derails the conversation and clutters the screen. Nothing less fun than trying to have a conversation and some idiot is posting "related" image macros flooding the chat with garbage.
The bottom line is I want to talk to someone using text. IRC does exactly that.
The terrible user experience of slack and discord is why I started using IRC even more. emoji, images, link previews, and the like all lead to annoying/awful spam. Instead of a conversation you get a stream on memes and image macros along with link and twitter spam. I find it derails the conversation and clutters the screen. Nothing less fun than trying to have a conversation and some idiot is posting "related" image macros flooding the chat with garbage.
The bottom line is I want to talk to someone using text. IRC does exactly that.
> The terrible user experience of slack and discord is why I started using IRC even more. emoji, images, link previews, and the like all lead to annoying/awful spam. Instead of a conversation you get a stream on memes and image macros along with link and twitter spam.
I have experienced that to a certain extent on Matrix, but only in certain rooms that permit it. They are not usually #some_project but rather general channels like #linux
What I have found good is code sharing, (nothing a pastebin can't solve, except for the fact if it's a publicly logged channel the link may not work in the future). I only tend to hang out in technical related rooms.
> I find it derails the conversation and clutters the screen. Nothing less fun than trying to have a conversation and some idiot is posting "related" image macros flooding the chat with garbage.
> The bottom line is I want to talk to someone using text. IRC does exactly that.
I find the same with mailing lists vs bulletin boards. I think the barrier to a certain extent means people that have something worth adding will be bothered. A bit like this place. Imagine how HN would be if it was like 90% of Reddit these days shivers.
I have experienced that to a certain extent on Matrix, but only in certain rooms that permit it. They are not usually #some_project but rather general channels like #linux
What I have found good is code sharing, (nothing a pastebin can't solve, except for the fact if it's a publicly logged channel the link may not work in the future). I only tend to hang out in technical related rooms.
> I find it derails the conversation and clutters the screen. Nothing less fun than trying to have a conversation and some idiot is posting "related" image macros flooding the chat with garbage.
> The bottom line is I want to talk to someone using text. IRC does exactly that.
I find the same with mailing lists vs bulletin boards. I think the barrier to a certain extent means people that have something worth adding will be bothered. A bit like this place. Imagine how HN would be if it was like 90% of Reddit these days shivers.
Hmm, logger bot could potentially detect pastebin links and fetch the content and store it along with the log in some link->content map.
This is honestly the problem of not having subchats. Slack and Discord's previews and stuff are fine, but at their own time and place. I heavily use IRC and I do miss rich previews a lot like Discord and similar have, I really don't want to open a browser to quickly check out a video or a song, IRC forces me to do so really often - especially annoying in a phone.
That is dependent on your IRC client not on IRC itself, just use a more robust client. I use Textual and it provides embedded previews of pictures and video directly in the chat window/feed...
> https://riot.im
aka Matrix, (Riot is the main client) is a good one because it is decentralized and federated. This means there can be many servers and they can all be linked up. Additionally IRC can be be bridged among other things
• https://en.wikipedia.org/wiki/Matrix_(protocol)
• https://matrix.org/docs/guides/types-of-bridging.html
• https://matrix.org/docs/projects/bridges
• https://github.com/matrix-org/matrix-appservice-irc/wiki/Bri...
aka Matrix, (Riot is the main client) is a good one because it is decentralized and federated. This means there can be many servers and they can all be linked up. Additionally IRC can be be bridged among other things
• https://en.wikipedia.org/wiki/Matrix_(protocol)
• https://matrix.org/docs/guides/types-of-bridging.html
• https://matrix.org/docs/projects/bridges
• https://github.com/matrix-org/matrix-appservice-irc/wiki/Bri...
zulip is open source (despite being developed by a company). They don't discourage you to run your own server, it's fully documented (and we do it in our company) It's not intended to be a Slack clone, but better by introducing email-like discussion topics. I like it, but seems that some less geeky people are more difficult to educate how to use it reasonably.
There’s also Mattermost.
Mumble
"Until about a year ago, we instead used a self-hosted PowerDNS on the back-end. PowerDNS was backed by a MySQL (later MariaDB) database. MySQL/MariaDB had been chosen for its support for streaming replication."
This is completely unnecessary (to say nothing of how MySQL silently corrupts data): PowerDNS can be configured for regular AXFR's between servers and that works exactly as one would expect, irrespective of what the back-end storage engine is.
The most effective, simplest, robust and therefore cheapest way to run PowerDNS is with the SQLite back-end as the storage engine and AXFR's between master and the slaves configured (PowerDNS BV run it with SQLite themselves); this inter-database replication is nonsense, just asking for trouble due to fragility and complexity. When I see such crazy unnecessarily complex solutions, I often wonder: what the hell went through those people's minds? Aren't they able to reason it out logically? What the hell were they thinking and why the hell were they thinking it??? This solution is so irrational, that it has to be ignorance.
This is completely unnecessary (to say nothing of how MySQL silently corrupts data): PowerDNS can be configured for regular AXFR's between servers and that works exactly as one would expect, irrespective of what the back-end storage engine is.
The most effective, simplest, robust and therefore cheapest way to run PowerDNS is with the SQLite back-end as the storage engine and AXFR's between master and the slaves configured (PowerDNS BV run it with SQLite themselves); this inter-database replication is nonsense, just asking for trouble due to fragility and complexity. When I see such crazy unnecessarily complex solutions, I often wonder: what the hell went through those people's minds? Aren't they able to reason it out logically? What the hell were they thinking and why the hell were they thinking it??? This solution is so irrational, that it has to be ignorance.
We actually chose PowerDNS with MySQL for a chatops style bot which automatically yanks servers in and out of the pool as needed (on splits, when limits are hit, etc), this was for the purpose of load balancing. We then decided to go with Route53 as it is less maintenance and more accurate geo-wise.
And why wouldn't normal AXFR's between servers work in this scenario?
The bot would directly update the MySQL database this was also chosen due to the complexity of geodns.
This answer makes no sense. Why does an IRC bot have to update the database? What does geolocation have to do with choosing database replication over AXFR?
With every answer you raise more questions than you answer.
With every answer you raise more questions than you answer.
[deleted]
It's strange, and there isn't much I would say I could do personally. But I'll keep my server going as long as I physically can, out of stubbornness if not anything else.