What I'm wondering: Are the salaries high? Not just because you've been employed at the job for a long time with regular raises, but because it's hard to find developers.
Using tls client auth certificates for submission would make slightly more sense indeed. But who would request LE certificates to install them on desktops/phones of individuals? That sounds more complicated than managing your own client certificates.
In my mail server, accounts can use TLS client authentication with submission and imap. But only the public key in the certificate is used, to identify the account. No verification of name or before/after time. You just save the public key with an account, or remove it when you no longer want to allow it. No external CA to trust, or internal CA to run, and no automatic expiration of your connectivity.
As for LE: I think they want to keep things simple. AFAIK they are a relatively small organization. Every task they take on weighs on them. They also don't sign certificates for use with S/MIME.
Most SMTP traffic is encrypted nowadays, at least "opportunistically", without verification.
Only with MTA-STS enabled for the server will an SMTP client (that's delivering to an SMTP server) verify the TLS certificate against PKI (with DANE, it's verified against "self-signed" or CA certs in DNS). (I'm the developer of a mail server that sets up TLS for SMTP with MTA-STS and DANE using Let's Encrypt certificates by default).
I have never heard of any SMTP server doing TLS client certificate authentication. I'm pretty sure there's no standard for that, so it can't be a requirement for all incoming email. It could be a requirement between parties that have made agreements about that explicitly. And theoretically, some mail servers could use it as a signal of authenticity of the sender. But email has other, standardized mechanisms for that. And I suspect you might see delivery failures if you start requesting TLS client cert authentication from all SMTP clients.
> I started writing a guide to IMAP back when I was working on an email client
I would be very interested in hearing from developers about how they write email clients that need to work with all the servers out there, with the varying levels of IMAP4 (original, rev1, rev2, combinations of at least a dozen extensions) and various levels of buggy behaviour.
I'm assuming a client developer would implement various "profiles". With an "advanced" profile for servers that implement things like CONDSTORE/QRESYNC, and a "very basic" profile that only do the absolute minimum. And probably a profile or two in between. When you encounter unexpected behaviour (eg bad syntax in protocol), you would downgrade the profile for that server for a while (it may get fixed)? If it works like this, I'm curious to the profiles developer choose. If it's not like this, I wonder how client developers work around compatibility issues (can't just keep reconnecting and trying the same if it results in an error!).
> Any sane client will instead use UIDs for everything
Yes! Since last year there's the experimental UIDONLY extension that allows clients & servers to operate entirely without message sequence numbers. Saves quite a bit of accounting and possibly memory (for large mailboxes). It's a bit surprising the RFC so recent.
> [... thick vs thin clients ...]
For some programmatic email access, IMAP seems quite easy to use. Just write a programmer-readable line of text as command (eg fetch/search/store/etc) and parse the response lines (that's a bit trickier in many cases, probably needing a library, but you may be able to get away without it some of the time) and you're done.
About thick clients: Storage capacity is growing much harder than my email, also on mobile. I would be fine with thick clients that sync all messages. Allows for simpler protocols and simpler implementations. An email access protocol is then not much more than a two-way sync of email message files and associated "dynamic data" like flags/keywords.
But IMAP and JMAP are doing many things at once: synchronization for thick clients, and online access for thin clients. JMAP is specifically targeting the web (thin clients), so what was old is new again (although base IMAP isn't too helpful to (web-based) thin clients either).
> You can send lots of requests to a server and you have to tag them so you can match them up with responses later
Yes, you can match the final OK/NO/BAD responses with the original command based on the tag. The annoying thing is that each command typically sends most of its data in "untagged responses". And IMAP servers can send unsolicited untagged responses at any time too. It's quite tricky to handle all those responses correctly in email clients. Indeed, normally you just want to send a command, and get all the data for that command back until the finishing response. IMAP clients typically open multiple connections to isolate commands.
Btw, I don't think it's better to have separate _components_ as in separate software packages. Integrating this functionality into one software package prevents all kinds of complexity that would otherwise arise in the integration points. Integrated software also allows for new/user-friendlier functionality.
Yes, you can add domains, add accounts (which contain the mailboxes and messages), and configure addresses for the domains with accounts. So one account can have many addresses, at one or more domains, including catchalls.
> because the updates and security fixes are managed by Debian. Once things are configured, I don't need to do anything using unattended upgrades
This is a good point. It would be great to have mox packaged in more
distributions. I spoke with a package maintainer about this. They
understandably need to be able to upgrade unattended from old versions to a
new version. In the past year, admins have had to run an upgrade command here
and there (e.g. to reparse all the messages after the parsing code changed). I
hope to make all this more automatic this year. That should make it more
appealing for packagers (and for all non-distro-using admins too!).
I think a new debian LTS release will be coming up soonish, we probably won't
make that.
> (which may already exist) - I'd love to be able to use my external spamassassin with it instead of (or even in addition to) the built in one
This isn't possible yet. For me, the builtin filtering has been enough. But
it's worth investigating what it takes to ask spamassassin for a
classification. Could you open an issue at github for this?
> the documentation/instructions could be clearer for non-typical use cases, for instance catch-all emails
Agreed, documentation is in need of improvement. So far I'm often pointing
people at https://www.xmox.nl/config/. Searching there typically pops up a
config option. But it's not the easiest to find functionality that way. The
admin web interface also needs to be made less spartan.
The catchall is possible, by configuring an address "@$yourdomain" with an
account.
> because it just seems to work
This is certainly the goal. And I think we'll only get better over time!
> incoming emails didn't work from Microsoft but they fixed that in 0.13
Yeah, this one was interesting. It looks like microsoft updated their TLS
stack to TLS 1.3, but incorrectly, breaking TLS connections to Go TLS
servers. I don't know how to contact Microsoft about it, but others have
raised issues with Microsoft. Mox got a workaround (disabling session tickets
for SMTP) so Microsofts TLS stack wouldn't abort the connection anymore. This
is a downside of being a small guy: You have to work around the bugs of the
big guys.
> this allows those spam checks which will still work to benefit from the normal delivery path.
It's not exactly the same. When a backup MX has accepted the message, it takes responsibility of the message, and will have to send a DSN when it is rejected for being spam. Mox never "delivers" messages to a spam mailbox (it's that behaviour from the bigmail providers I don't like and undermines trust in email!). Mox either accepts a message, or rejects it at the SMTP level. When the backup sends to the primary, and the primary wants to reject, the backup would have to send a DSN to the potential spammer. Not great, and not something we have to do now.
But still, if it's only needed for emergencies, when the primary is down, it probably isn't too problematic. And the backup mx (with primary offline) can always be more strict, requiring dmarc-like alignment before accepting (to prevent backscatter if the primary rejects later on).
> Also, with a version number starting with 0.0. I'm left wondering if Mox is already stable enough to be entrusted with my precious email.
It's been suggested to just increase the version number since it's more stable than a 0.0.X might suggest. I'm currently considering mox at release number 14. I'm still on the fence about it. Ideally people make the decision on the merits of stability, not based on the looks of the version number. But I understand it's used as a signal for how stable software is (but mileage will vary!).
At least I'm trying hard not to break anything, so upgrades will work for all installations.
From a security perspective, it is better to have separate & isolated processes (but written in modern languages that provide safety).
Mox is currently a single process handling all connections, including deliveries over smtp, imap connections, and webmail and other http requests, which isn't great. User connections should probably be in a separate process. I'm not too afraid of the mox process being taken over (by a bug being abused, I don't think that's easy/common in software written in Go), but of course it will be a good line of defense against that. Resource limit enforcement of separate processes would perhaps be even nicer to get.
I haven't gotten around to really designing privilege separation, but I'm forseeing some complications around handling http requests (of the webmail, pass each request on to the user process? Have to figure out how to do that with the http library), and message database access (the database files can only be open by a single process, need to do quite some back and forth to the user process in various places).
For performance, I imagine it only helps to have an integrated server. Performance isn't really top of mind, I don't think mail servers are commonly highly loaded, at least not for the smallish scale servers. Btw, mox does not require a lot of resource (eg RAM) to run.
Hi Bron! Thanks, JMAP will come at some point too! (:
Someone has already been working on JMAP support in mox. I'm currently in a refactor of the storage layer, keeping history of (deleted) mailboxes too. Should address storage requirements for JMAP.
I'm focusing on functionality/protocol support now. User/admin-friendliness and making it more attractive will come later. Mox will become irresistible to the masses then!
Seems reasonable. I have been thinking about schemes for backup mx'es. When the primary is online, the backup would just forward the smtp connections to the primary directly. The backup mx only has to do their own work when the primary is down. I was/am concerned about the backup mx accepting spam that you don't really want to bounce later on. But that should be an exception.
I am also thinking about synchronizing all the data to another machine. It would allow a manual failover procedure. And it's nice to have another machine (IP) for outgoing email in case the primary IP gets on a block list. But this is all future work.