The qmail security guarantee(cr.yp.to)
cr.yp.to
The qmail security guarantee
https://cr.yp.to/qmail/guarantee.html
71 comments
I'm surprise how long qmail lasted but it definitely has issues (some of them with patches, the patches definitely have much more flimsy security and realistically you can't run it without many feature patches these days)
I actually wrote a replacement for vpopmail called 'rmail' that stored the virtual user databases in cdb. I never liked requiring a central DB like MySQL for your mail receipt to work in real time in an otherwise relatively decentralised server protocol (despite literally working at MySQL for 9 years) and liked the idea of a file better. Powered 1000+ users for many many years. Now replaced with postfix for about 5 years :)
Still, it definitely has a surprisingly solid place in my historic heart.
I actually wrote a replacement for vpopmail called 'rmail' that stored the virtual user databases in cdb. I never liked requiring a central DB like MySQL for your mail receipt to work in real time in an otherwise relatively decentralised server protocol (despite literally working at MySQL for 9 years) and liked the idea of a file better. Powered 1000+ users for many many years. Now replaced with postfix for about 5 years :)
Still, it definitely has a surprisingly solid place in my historic heart.
Postfix has always (or, at least, as long as I can remember) used something very similar to that idea for all of its various user, alias, virtual, etc. tables. It's mostly transparent to the user, but when you run postmap, you're creating a key/value database (for those who don't know, cdb was djb's efficient key/value database implementation way back when) of the things in the map file.
It can even use cdb as a backend, but realistically for the kinds of map tables one works with in a mail server, any of them are Good Enough(tm) because the entire data set will always fit in memory (even if you have an absurdly large mail server with 100,000 mail users on a single system, it'll still fit in RAM).
We used to get a lot of people wanting MySQL support in our mail stack, which I actively have always discouraged. It's (orders of magnitude) slower, harder to maintain, and provides no benefit for the vast majority of deployments. For whatever reason (maybe just that fewer people want to host their own mail beyond sending mail from apps and notifications from the system itself), we get a lot fewer requests for it lately. Maybe I finally convinced everyone it was a bad idea.
It can even use cdb as a backend, but realistically for the kinds of map tables one works with in a mail server, any of them are Good Enough(tm) because the entire data set will always fit in memory (even if you have an absurdly large mail server with 100,000 mail users on a single system, it'll still fit in RAM).
We used to get a lot of people wanting MySQL support in our mail stack, which I actively have always discouraged. It's (orders of magnitude) slower, harder to maintain, and provides no benefit for the vast majority of deployments. For whatever reason (maybe just that fewer people want to host their own mail beyond sending mail from apps and notifications from the system itself), we get a lot fewer requests for it lately. Maybe I finally convinced everyone it was a bad idea.
I'm responsible for several mail servers and all are them are running postfix with the user data stored in SQL (including my own personal mail server). The primary reason is to simplify administration (management of user accounts and such, using various scripts and webapps).
There is an argument to be made for some sort of central backend store when you have multiple mail servers for the same domain(s) and users. SQL isn't terrible in that context (I would likely prefer LDAP for that, but SQL works, too).
Most of our users have one server, and lots of tutorials and stuff are about one mail server. It just makes no sense to add that huge amount of extra complexity for that simple use case.
Most of our users have one server, and lots of tutorials and stuff are about one mail server. It just makes no sense to add that huge amount of extra complexity for that simple use case.
The way we do it is the primary store is in SQL but I sync that data out to a file on demand
Yep, qmail was awesome back in the day -- especially all the virtual hosting stuff. It was used extensively at Yahoo! for quite a while with great success. I imagine they've probably long since moved on, though.
WRT OpenSMTPD: I have used it and do use it on several machines, although just for locally generated mail (e.g. system e-mails to root) that I want delivered elsewhere. It's lightweight, easy as hell to configure, and just works(TM). Unless you're running a large mail environment or need lots of integrations with other software (e.g. SpamAssassin, OpenDKIM, etc.) it'll probably serve your needs quite well.
WRT OpenSMTPD: I have used it and do use it on several machines, although just for locally generated mail (e.g. system e-mails to root) that I want delivered elsewhere. It's lightweight, easy as hell to configure, and just works(TM). Unless you're running a large mail environment or need lots of integrations with other software (e.g. SpamAssassin, OpenDKIM, etc.) it'll probably serve your needs quite well.
As a hardcore qmail (and qmail-ldap) user back in the days, I can confirm that this is all accurate and correct.
... and yet writes netqmail entirely out of history.
* http://netqmail.org.
* http://netqmail.org.
From the page you've linked: "Last modified: Wed Feb 2 23:37:31 EST 2005"
What aspect of the existence of netqmail disputes what I wrote?
What aspect of the existence of netqmail disputes what I wrote?
The onus is upon you to explain where you have accounted for the existence of netqmail in your narrative, from which it is otherwise conspicuous by its entire absence.
You clearly went to its WWW site, but didn't even look at its changelog. Instead, you took the last modification date of the opening page of the WWW site itself and bizarrely tried to base some sort of argument, it is unclear what, upon that.
When it comes to discussion of qmail and patches, netqmail is a fairly important part of the history. Hint: One of its co-authors literally wrote the book on qmail. (And indeed updated that book to describe netqmail.)
You clearly went to its WWW site, but didn't even look at its changelog. Instead, you took the last modification date of the opening page of the WWW site itself and bizarrely tried to base some sort of argument, it is unclear what, upon that.
When it comes to discussion of qmail and patches, netqmail is a fairly important part of the history. Hint: One of its co-authors literally wrote the book on qmail. (And indeed updated that book to describe netqmail.)
20071130 version: netqmail 1.06
2007.
2007.
FWIW, I still use qmail. I wouldn't recommend it to someone who is setting up a new mail server -- then again, I generally wouldn't recommend that people set up mail servers -- but I've copied my setup from one system to another over the years and it's still working just fine.
I really hate that the general wisdom is, "don't run your own mail". Email is such a beautiful thing; anybody can talk directly to (almost) anybody else anywhere in the world, and it's been consistent and (mostly) compatible practically forever (much of what we call email was defined before it was even called "the internet").
And, that negative general wisdom is all because of abuse. Even if you figure out how to solve incoming spam, you might still have trouble sending mail to others because of their abuse prevention practices.
But, it's really not as bad as the general wisdom would indicate. Even with the particularly severe challenges we deal with on behalf of our customers (we mostly interact with them while they're initially setting up a new server on an IP of indeterminate history), we almost always get things to a state where delivery is reliable. DKIM and SPF, while a little complicated, seem to be helping a lot...the big providers are seemingly relying less and less on IP address in determining spamminess and more on history of the sender.
Anyway, I really hate that email is heavily consolidated into the hands of a very few very large corporations who have little respect for privacy.
We still have some old-timers using QMail, and we keep supporting it in a sort of "we won't break it" way because we like our old-timers and they mostly take care of themselves and sometimes even submit patches. There's nothing really wrong with QMail. It's just quite dated in a number of areas. So, I don't mind supporting it. It's not bad software, it's just old software.
And, that negative general wisdom is all because of abuse. Even if you figure out how to solve incoming spam, you might still have trouble sending mail to others because of their abuse prevention practices.
But, it's really not as bad as the general wisdom would indicate. Even with the particularly severe challenges we deal with on behalf of our customers (we mostly interact with them while they're initially setting up a new server on an IP of indeterminate history), we almost always get things to a state where delivery is reliable. DKIM and SPF, while a little complicated, seem to be helping a lot...the big providers are seemingly relying less and less on IP address in determining spamminess and more on history of the sender.
Anyway, I really hate that email is heavily consolidated into the hands of a very few very large corporations who have little respect for privacy.
We still have some old-timers using QMail, and we keep supporting it in a sort of "we won't break it" way because we like our old-timers and they mostly take care of themselves and sometimes even submit patches. There's nothing really wrong with QMail. It's just quite dated in a number of areas. So, I don't mind supporting it. It's not bad software, it's just old software.
> the big providers are seemingly relying less and less on IP address in determining spamminess...
It's not the big providers that you need to worry about. Setup SPF and most of them are fine. Some will look at your DKIM record too. The problem is every mid-level smtp service that has been configured by a varying number of people with varying levels of policies and ideas around how to mitigate spam. If your system triggers their spam detection then sometimes you get an email saying it was returned to sender and marked as spam. Sometimes, it will quietly go into their spam folder. I only ever got DMARC reports from Google.
One needs an IP registered to their own AS number owned by a real company to reliably deliver email to the entire web. You can't run an MTA on a VPS or most home internet connections without spending a lot of time monitoring and troubleshooting delivery. Using a 3rd party service for sending while hosting the MX at home is still the most secure and reliable setup. What would be really useful is a "bounce_forwarding_mta" where bounced emails would always attempt to redeliver through the 2nd MTA. Then you could send everything through your own MTA on your home internet connection and if one bounces then the system automatically attempts to send it through whatever smtp SaaS the user configures.
Things on the internet usually suck because of corporate policy, not usually because of the technology.
It's not the big providers that you need to worry about. Setup SPF and most of them are fine. Some will look at your DKIM record too. The problem is every mid-level smtp service that has been configured by a varying number of people with varying levels of policies and ideas around how to mitigate spam. If your system triggers their spam detection then sometimes you get an email saying it was returned to sender and marked as spam. Sometimes, it will quietly go into their spam folder. I only ever got DMARC reports from Google.
One needs an IP registered to their own AS number owned by a real company to reliably deliver email to the entire web. You can't run an MTA on a VPS or most home internet connections without spending a lot of time monitoring and troubleshooting delivery. Using a 3rd party service for sending while hosting the MX at home is still the most secure and reliable setup. What would be really useful is a "bounce_forwarding_mta" where bounced emails would always attempt to redeliver through the 2nd MTA. Then you could send everything through your own MTA on your home internet connection and if one bounces then the system automatically attempts to send it through whatever smtp SaaS the user configures.
Things on the internet usually suck because of corporate policy, not usually because of the technology.
Yeah, I agree with most of what you say, though I disagree about needing your own AS; we've never had one and almost none of our clients have one.
Nonetheless, there's a lot of pressure toward using the big providers, and that's dangerous, IMHO. Also, the mail client landscape isn't great (lots of people are working on it, including my company, but a great Open Source webmail client comparable to GMail does not yet exist...and, the mobile client situation is probably even worse than the state of webmail).
Nonetheless, we have never outsourced our mail, and we deliver several thousand emails a day (forum and tracker notifications, mostly, but also expiration notices and billing failure notices) to thousands of different domains from our own MTAs. We are on static IPs in a colocation provider on our own physical server. We lived on the same IP for about five years and then moved to a new one a couple of years ago (same colocation). We only had one deliverability problem on the fresh IP, which was with Hotmail/Outlook, who seem to block every IP by default and you have to ask to be unblocked. But, once we jumped through their hoops we've never had deliverability problems since.
We may be dealing with less troublesome recipients. Our users are mostly Linux/UNIX users at small-ish companies. They probably care about people being able to send them mail reliably more than maybe bigger companies with layers of security theater to deal with.
I like the idea of a bounce forwarding MTA. That's actually a pretty brilliant approach to the problem; having something like Sendgrid or Mailjet or whatever as the backup would be a pretty neat thing (and for such a low volume of mail it'd be cheap or free for the small companies we deal with). Briefly googling didn't really turn up anything about that approach to bounces (googling anything related to bounces is tough because it can be about backscatter, my mail bouncing, my mail server bouncing incoming mail, delivery of bounces, etc.).
It's already possible to detect bounces in Postfix, but I dunno how one would turn the bounce message (which are infinitely varied in their contents) into a forward for the original email. Maybe just instant rejections would be handleable in such a case. That'd be a reasonable start on the problem.
And, in the case of email, corporate policy sucks because abuse sucks (and people are dumb about security practices). Because somebody is gonna click on that virus attachment if it finds its way through, IT is expected to act like the TSA and engage in all manner of security theater to prevent it, at any cost.
Nonetheless, there's a lot of pressure toward using the big providers, and that's dangerous, IMHO. Also, the mail client landscape isn't great (lots of people are working on it, including my company, but a great Open Source webmail client comparable to GMail does not yet exist...and, the mobile client situation is probably even worse than the state of webmail).
Nonetheless, we have never outsourced our mail, and we deliver several thousand emails a day (forum and tracker notifications, mostly, but also expiration notices and billing failure notices) to thousands of different domains from our own MTAs. We are on static IPs in a colocation provider on our own physical server. We lived on the same IP for about five years and then moved to a new one a couple of years ago (same colocation). We only had one deliverability problem on the fresh IP, which was with Hotmail/Outlook, who seem to block every IP by default and you have to ask to be unblocked. But, once we jumped through their hoops we've never had deliverability problems since.
We may be dealing with less troublesome recipients. Our users are mostly Linux/UNIX users at small-ish companies. They probably care about people being able to send them mail reliably more than maybe bigger companies with layers of security theater to deal with.
I like the idea of a bounce forwarding MTA. That's actually a pretty brilliant approach to the problem; having something like Sendgrid or Mailjet or whatever as the backup would be a pretty neat thing (and for such a low volume of mail it'd be cheap or free for the small companies we deal with). Briefly googling didn't really turn up anything about that approach to bounces (googling anything related to bounces is tough because it can be about backscatter, my mail bouncing, my mail server bouncing incoming mail, delivery of bounces, etc.).
It's already possible to detect bounces in Postfix, but I dunno how one would turn the bounce message (which are infinitely varied in their contents) into a forward for the original email. Maybe just instant rejections would be handleable in such a case. That'd be a reasonable start on the problem.
And, in the case of email, corporate policy sucks because abuse sucks (and people are dumb about security practices). Because somebody is gonna click on that virus attachment if it finds its way through, IT is expected to act like the TSA and engage in all manner of security theater to prevent it, at any cost.
> there's a lot of pressure toward using the big providers, and that's dangerous
This is true and you mention this in other posts. The resounding answer to "how do I run my own mail server" is "don't" and that creates a dangerous environment where we abandon a previously working and useful decentralized part of the core internet because corporations do not accept our individual participation.
This is a form of censorship and I really don't know the best way to approach it.
If a message is signed with DKIM there is a statistically insignificant probability that it is not from the asserted sender. Every MX should block based on content and behavior after initially accepting based on sender validation (SPF/DKIM). It seems like we need an email advocacy group to fight the long fight and tighten the screws on individual companies with overly protective policies like comcast and att. Perhaps even working at a federal/eu policy level would make sense.
This is true and you mention this in other posts. The resounding answer to "how do I run my own mail server" is "don't" and that creates a dangerous environment where we abandon a previously working and useful decentralized part of the core internet because corporations do not accept our individual participation.
This is a form of censorship and I really don't know the best way to approach it.
If a message is signed with DKIM there is a statistically insignificant probability that it is not from the asserted sender. Every MX should block based on content and behavior after initially accepting based on sender validation (SPF/DKIM). It seems like we need an email advocacy group to fight the long fight and tighten the screws on individual companies with overly protective policies like comcast and att. Perhaps even working at a federal/eu policy level would make sense.
How do you deal with the spam issue?
SpamAssassin, greylisting with postgrey, DKIM/SPF checking in SpamAssassin (though DKIM is still configured to be somewhat forgiving).
My email address has not changed in 12+ years and is widely available on the internet, and I get maybe a half dozen spam messages that make it to my inbox each day. That's more than I'd like, and slightly more than my GMail address receives. But, it's tolerable. I could probably tweak it a bit more to prevent more spam, but I don't really have time to micromanage it...I just let the filters do their thing. If I enforced DKIM I'd probably successfully block nearly everything, but I'd get a lot of false positives, too.
A surprisingly large amount (like 5%) of the spam I get that successfully makes it through is not mass-email. It's people who have acquired a list of Y Combinator company founders and are sending out semi-personalized mail. A surprising amount of my spam mail name drops Y Combinator (and tries to imply they're already working with other YC companies). Which is really the weirdest damned thing, but it happens a couple times a week. But, I don't know that any spam filter could stop that. I get the same kind of thing (not YC related, usually, but things like recruiter emails) on my GMail account which is what I use in some developer communities.
My email address has not changed in 12+ years and is widely available on the internet, and I get maybe a half dozen spam messages that make it to my inbox each day. That's more than I'd like, and slightly more than my GMail address receives. But, it's tolerable. I could probably tweak it a bit more to prevent more spam, but I don't really have time to micromanage it...I just let the filters do their thing. If I enforced DKIM I'd probably successfully block nearly everything, but I'd get a lot of false positives, too.
A surprisingly large amount (like 5%) of the spam I get that successfully makes it through is not mass-email. It's people who have acquired a list of Y Combinator company founders and are sending out semi-personalized mail. A surprising amount of my spam mail name drops Y Combinator (and tries to imply they're already working with other YC companies). Which is really the weirdest damned thing, but it happens a couple times a week. But, I don't know that any spam filter could stop that. I get the same kind of thing (not YC related, usually, but things like recruiter emails) on my GMail account which is what I use in some developer communities.
For sending or receiving?
How do you avoid receiving spam messages?
I can't claim to be an expert in this field because I gave up years ago. Back when I was actively trying to prevent spam from entering my mail servers I would use:
* Fail if no SPF record
* Fail if DKIM record present in NS & not-present in message
* blacklists (DNSBL)
* heuristics (SpamAssassin)
Back then, not everyone published both SPF and DKIM. I would only fail a message if it did have a DKIM record and the message did not have the matching DKIM header. If I recall correctly, I failed everything that didn't publish an SPF record.
Blacklists are tricky because they are outside of your control. You are relying on a community of folks to tell you what is best. They are still extremely useful but use them with caution.
Heuristics need to plug in to your mail clients' "Mark as Spam" feature. There has to be a feedback mechanism.
There are a number of other configuration settings that control what can and cannot get through like requiring reverse DNS to accurately resolve sender ip = domain name.
It is a wicked problem and a game of whack-a-mole. I found it better to pay dedicated experts to handle it.
* Fail if no SPF record
* Fail if DKIM record present in NS & not-present in message
* blacklists (DNSBL)
* heuristics (SpamAssassin)
Back then, not everyone published both SPF and DKIM. I would only fail a message if it did have a DKIM record and the message did not have the matching DKIM header. If I recall correctly, I failed everything that didn't publish an SPF record.
Blacklists are tricky because they are outside of your control. You are relying on a community of folks to tell you what is best. They are still extremely useful but use them with caution.
Heuristics need to plug in to your mail clients' "Mark as Spam" feature. There has to be a feedback mechanism.
There are a number of other configuration settings that control what can and cannot get through like requiring reverse DNS to accurately resolve sender ip = domain name.
It is a wicked problem and a game of whack-a-mole. I found it better to pay dedicated experts to handle it.
I recently received via email an update for a patch I wrote for qmail over a decade ago. I'd forgotten I'd even written the patch and was shocked anyone was even still using qmail.
+1 for OpenSMTPD.
I started using qmail in 1996-ish, and was a pretty big fan for a while.
I wouldn't use it at this point though, it really has languished and is, IMHO, totally unusable without a lot of dicking around to get the right set of patches and the like.
He kind of lost me when AOL increased their MX record to be larger than UDP could handle in a single packet. He stood firm that AOL was doing it wrong, meanwhile my users didn't care about what the RFCs said, they just wanted to e-mail their grandma like all their friends could.
The qmail community collected a lot of people with similar personalities and I got tired of it.
These days, I'm pretty happy with Postfix, and it has things a modern e-mail servers needs, supported in the core rather than requiring a ton of patches that haven't been eyeballed by the author.
I wouldn't use it at this point though, it really has languished and is, IMHO, totally unusable without a lot of dicking around to get the right set of patches and the like.
He kind of lost me when AOL increased their MX record to be larger than UDP could handle in a single packet. He stood firm that AOL was doing it wrong, meanwhile my users didn't care about what the RFCs said, they just wanted to e-mail their grandma like all their friends could.
The qmail community collected a lot of people with similar personalities and I got tired of it.
These days, I'm pretty happy with Postfix, and it has things a modern e-mail servers needs, supported in the core rather than requiring a ton of patches that haven't been eyeballed by the author.
According to this: https://en.wikipedia.org/wiki/Comparison_of_mail_servers
qmail doesn't support SMTP over TLS or SSL. How is that "secure"?
qmail doesn't support SMTP over TLS or SSL. How is that "secure"?
Security of an email server has nothing to do with secure transport of emails. If you run qmail, nobody will ever root your server. Sure they'll be able to intercept and read your email, but they won't root your server!
I did run qmail in 1998ish and it was quirky but rock solid. But that was coming from sendmail, which I'd never recommend. Now I'd say postfix is the way to go: quite secure and fully TLS capable.
I did run qmail in 1998ish and it was quirky but rock solid. But that was coming from sendmail, which I'd never recommend. Now I'd say postfix is the way to go: quite secure and fully TLS capable.
Wikipedia is misleading again. Notice that there are no sources for any of the claims made about several of the softwares, to check the article against.
qmail does not, strictly speaking, even support SMTP over TCP. This is because it relies upon UCSPI tools to do the actual transport layer setup. UCSPI-TCP does the TCP part for qmail. And one can equally well layer it over UCSPI-SSL.
People did. Erwin Hoffmann, one of the UCSPI-SSL authors, even wrote a lengthy article on the subject of SMTP over TSL/SSL with qmail.
* http://jdebp.eu./FGA/UCSPI.html
* http://www.fehcom.de/qmail/smtptls.html
qmail does not, strictly speaking, even support SMTP over TCP. This is because it relies upon UCSPI tools to do the actual transport layer setup. UCSPI-TCP does the TCP part for qmail. And one can equally well layer it over UCSPI-SSL.
People did. Erwin Hoffmann, one of the UCSPI-SSL authors, even wrote a lengthy article on the subject of SMTP over TSL/SSL with qmail.
* http://jdebp.eu./FGA/UCSPI.html
* http://www.fehcom.de/qmail/smtptls.html
Well, if you sent an email to `|program` it didn't execute program. Seriously[1]
[1]: https://www.tenable.com/plugins/index.php?view=single&id=102...
Indeed, many sysadmins of the time lauded qmail's lack of supporting "standard features" in the name of "security" which sadly was still a new concept on the Internet.
Something to consider: SMTP over TLS offers some privacy and confidentiality between two mail servers that have established a trust relationship, but it offers no protection against an upstream network (who can simply fake some DNS records and get a letsencrypt certificate) or a state actor (who simply threatens the CA). I think referring to "SMTP over TLS" as "secure" is dangerous because it leads us to equate "more code" as providing security.
[1]: https://www.tenable.com/plugins/index.php?view=single&id=102...
Indeed, many sysadmins of the time lauded qmail's lack of supporting "standard features" in the name of "security" which sadly was still a new concept on the Internet.
Something to consider: SMTP over TLS offers some privacy and confidentiality between two mail servers that have established a trust relationship, but it offers no protection against an upstream network (who can simply fake some DNS records and get a letsencrypt certificate) or a state actor (who simply threatens the CA). I think referring to "SMTP over TLS" as "secure" is dangerous because it leads us to equate "more code" as providing security.
Truly, the only way to have security is to encrypt the message with the receiver's public key.
I find it discouraging that it's almost 2018 and this function of E-mail has not been made standard in all clients yet.
I understand the concerns about trust, but why not make trust the extra step for now, and make encryption the standard. And in time we can standardize trust as well. (I know it's pretty standard already but I'm thinking about 'the average user')
"Well I can't trust the source so why bother with encryption" is what we have presently, and that's just ridiculous.
I understand the concerns about trust, but why not make trust the extra step for now, and make encryption the standard. And in time we can standardize trust as well. (I know it's pretty standard already but I'm thinking about 'the average user')
"Well I can't trust the source so why bother with encryption" is what we have presently, and that's just ridiculous.
"What's your email address" is about 90-120 bits of information -- a long way from the 2000-5000 bits that are in a public key. I figure if we solve this problem then we can make encrypting email the norm.
"It's too bad DJB is so abrasive, because he has some good ideas and nobody will listen to him because of it." This came from someone fairly famous, but I didn't ask to quote him so I won't. His name is on a definitive book in his field.
Personality is important.
Personality is important.
More important than aesthetics, competence and the code itself?
What if the user does not care "who" the author is, other than to identify what he has written?
For example, I have no idea "who" Arthur Whitney is, but I will not hesitate to spend time learning any software he writes. His work speaks for itself.
Same for djb.
In the context of how I choose software, personality is irrelevant.
I am unlikely to ever interact with these authors.
If people in forums and committees put these authors down or criticize their work then that tells me something about the people in forums and committees.
I am not sure I understand what bothers them about the few people who have rare ability to "push the envelope" in software and share their work over the internet, but I am aware that it bothers some people.
What if the user does not care "who" the author is, other than to identify what he has written?
For example, I have no idea "who" Arthur Whitney is, but I will not hesitate to spend time learning any software he writes. His work speaks for itself.
Same for djb.
In the context of how I choose software, personality is irrelevant.
I am unlikely to ever interact with these authors.
If people in forums and committees put these authors down or criticize their work then that tells me something about the people in forums and committees.
I am not sure I understand what bothers them about the few people who have rare ability to "push the envelope" in software and share their work over the internet, but I am aware that it bothers some people.
> More important than aesthetics, competence and the code itself?
Yes, if that personality results in "sexual harassment, bullying, blackmail, and physical harm" (https://medium.com/@hdevalence/when-hell-kept-on-payroll-som...), then that personality is more important then "aesthetics".
Previous HN discussion: https://news.ycombinator.com/item?id=13891513 I wonder if this has gone anywhere in the courts.
Yes, if that personality results in "sexual harassment, bullying, blackmail, and physical harm" (https://medium.com/@hdevalence/when-hell-kept-on-payroll-som...), then that personality is more important then "aesthetics".
Previous HN discussion: https://news.ycombinator.com/item?id=13891513 I wonder if this has gone anywhere in the courts.
Many people use software everyday without being aware of the identity of its author(s) and certainly not the personal life of its author(s). Because I prefer open source software, I may know who are the authors of the software I use, but I have no need to know the personal lives of these authors.
At this time there is another item on the HN front page about a Linux distribution called "Void Linux". Several of the comments indicate it uses "runit". Runit is a copy of software called "daemontools", written by djb. I use daemontools. I would like to keep using it.
Do I need to cease using it because people in blogs and forums are discussing personal matters involving the author? Is it acceptable to use Void Linux?
To list all the popular software/websites that may include/use code/software from this author would be an exhausting exercise. It would probably include many well-known entries, such as major email providers, DuckDuckGo, OpenDNS and WhatsApp.
I can easily avoid those popular choices, but I still need to use daemontools, tinydns, clockadd, and other programs by this author. IMHO they have no equal.
For clarification, by "aesthetics" I mean software aesthetics. Namely, a preference for small program size, terse syntax, low resource requirements, and numerous other "aesthetic" qualities. These qualities are evident from the code itself and I need not know anything about the life of the author.
It may be better that I do not know. Consider an avid reader who grows to love the work of a particular author. Then one day she decides to meet the author. Unfortunately she is severely let down when she learns the author's personality is not what she expected. Does this reduce the quality of his art that she previously enjoyed? One can apply this idea to any form of art. The art itself vs the life of the artist. To me, software is a form of art.
At this time there is another item on the HN front page about a Linux distribution called "Void Linux". Several of the comments indicate it uses "runit". Runit is a copy of software called "daemontools", written by djb. I use daemontools. I would like to keep using it.
Do I need to cease using it because people in blogs and forums are discussing personal matters involving the author? Is it acceptable to use Void Linux?
To list all the popular software/websites that may include/use code/software from this author would be an exhausting exercise. It would probably include many well-known entries, such as major email providers, DuckDuckGo, OpenDNS and WhatsApp.
I can easily avoid those popular choices, but I still need to use daemontools, tinydns, clockadd, and other programs by this author. IMHO they have no equal.
For clarification, by "aesthetics" I mean software aesthetics. Namely, a preference for small program size, terse syntax, low resource requirements, and numerous other "aesthetic" qualities. These qualities are evident from the code itself and I need not know anything about the life of the author.
It may be better that I do not know. Consider an avid reader who grows to love the work of a particular author. Then one day she decides to meet the author. Unfortunately she is severely let down when she learns the author's personality is not what she expected. Does this reduce the quality of his art that she previously enjoyed? One can apply this idea to any form of art. The art itself vs the life of the artist. To me, software is a form of art.
> Is it acceptable to use Void Linux?
Yes, and it's disingenious to suggest that I suggested otherwise.
> These qualities are evident from the code itself
But you don't read the code of every single package on your system. Similarly, you probably don't have to research the personal background of every single developer.
> IMHO they have no equal.
Well, there you go then. That's a good reason to use them. I said personality can be more important than aesthetics, I did not say personality is more important than any other criterion you might think of.
> It may be better that I do not know.
It may, although now you do know. Actively seeking out information to act on is different from acting on information that was communicated to you. But again, if you feel that I am saying you are a bad person for using tinydns, you feel wrong.
Yes, and it's disingenious to suggest that I suggested otherwise.
> These qualities are evident from the code itself
But you don't read the code of every single package on your system. Similarly, you probably don't have to research the personal background of every single developer.
> IMHO they have no equal.
Well, there you go then. That's a good reason to use them. I said personality can be more important than aesthetics, I did not say personality is more important than any other criterion you might think of.
> It may be better that I do not know.
It may, although now you do know. Actively seeking out information to act on is different from acting on information that was communicated to you. But again, if you feel that I am saying you are a bad person for using tinydns, you feel wrong.
It is "disingeuous" to suggest I use binary packages or that I use third party software whose source that I have not edited or read.
As it happens, I am constantly reading the sources of the software I use: kernel, userland and third party. And I do pay attention to attributed authorship on that software.
But I am not evaulating any software based on "personalities" of its authors, as alleged by random people, often anonymous, in internet mailing lists, forums, blogs, etc.
I am going to continue to use the "best software available", as determined by me, for better or worse. I think I am not alone in that approach and I think it is reasonable. I will not be distracted by petty criticisms of what I know to be good software or "dirty laundry" about the authors on the internet that I may encounter in the process.
As it happens, I am constantly reading the sources of the software I use: kernel, userland and third party. And I do pay attention to attributed authorship on that software.
But I am not evaulating any software based on "personalities" of its authors, as alleged by random people, often anonymous, in internet mailing lists, forums, blogs, etc.
I am going to continue to use the "best software available", as determined by me, for better or worse. I think I am not alone in that approach and I think it is reasonable. I will not be distracted by petty criticisms of what I know to be good software or "dirty laundry" about the authors on the internet that I may encounter in the process.
> or that I use third party software whose source that I have not edited or read
I'm calling bullshit on the statement that you have read every single line of code for the software running on your system.
> I am going to continue to use the "best software available", as determined by me
Good for you.
I'm calling bullshit on the statement that you have read every single line of code for the software running on your system.
> I am going to continue to use the "best software available", as determined by me
Good for you.
I'm calling bullshit on injecting an absurd dichotomy between choosing software based on the author's personality versus "reading every single line of code of the software running on your system". It is possible to choose software based on qualities inherent in the software and not know or care about the personality or personal matters of the author. And it is possible to do this without "reading every single line of code of the software running on your system".
Those qualities of the software and its design, what I call "software aesthetics" might include, among other things, the size of the program, its resource requirements, dependencies, configuration, and even, yes, the source code itself. Nowhere did I suggest that I read "every line" of every source code file comprising the operating system I use.
Where possible I do selectively read and sometimes edit some of these files. With respect to third party software, I often do read every file. I prefer software that is small enough where I can do this. But what does the idea of reading "every line of code" in an (operating) system have to do with my original comment? Nothing.
In any event, since you have shifted the discussion to (operating) systems, I can confirm I did not choose the (operating) system I use based on the "personality" of its authors. I chose it because of the "software aesthetics" reflected in the software itself. As I see it, this might include an appreciation for the command line and small program size, manual configuration by the user and having all options off by default, documentation, portability and "clean code", among other things. People making comments in email or on the www on whether they like or dislike the authors of this software did not affect my decision to use it.
The point of my original comment was simple: someone may choose software based on the software itself, not the author's personality, whatever that may be. I thought this is worth considering in response to the parent comment that "Personality is important". But others may disagree.
I would like to end this exchange now. I appreciate your input.
Those qualities of the software and its design, what I call "software aesthetics" might include, among other things, the size of the program, its resource requirements, dependencies, configuration, and even, yes, the source code itself. Nowhere did I suggest that I read "every line" of every source code file comprising the operating system I use.
Where possible I do selectively read and sometimes edit some of these files. With respect to third party software, I often do read every file. I prefer software that is small enough where I can do this. But what does the idea of reading "every line of code" in an (operating) system have to do with my original comment? Nothing.
In any event, since you have shifted the discussion to (operating) systems, I can confirm I did not choose the (operating) system I use based on the "personality" of its authors. I chose it because of the "software aesthetics" reflected in the software itself. As I see it, this might include an appreciation for the command line and small program size, manual configuration by the user and having all options off by default, documentation, portability and "clean code", among other things. People making comments in email or on the www on whether they like or dislike the authors of this software did not affect my decision to use it.
The point of my original comment was simple: someone may choose software based on the software itself, not the author's personality, whatever that may be. I thought this is worth considering in response to the parent comment that "Personality is important". But others may disagree.
I would like to end this exchange now. I appreciate your input.
It's too bad so many people look at personality instead of focusing on the ideas. I try to watch myself, and if I catch myself focusing on personality, try to switch back to focusing on the ideas.
> Personality is important.
I agree, but I wonder how it factors in. Compare DJB to Linus Torvalds or Theo de Raadt or Steve Jobs or Steve Balmer or Travis Kalanick, as examples with a variety of outcomes (and I'm not trying hard to come up with those names; there are many more out there). Maybe the determining factor is something other than abrasiveness, maybe it's just balanced against demand for your project or other factors, maybe it depends on the market segment you're operating in - or maybe it's circumstance and luck that really matter.
I agree, but I wonder how it factors in. Compare DJB to Linus Torvalds or Theo de Raadt or Steve Jobs or Steve Balmer or Travis Kalanick, as examples with a variety of outcomes (and I'm not trying hard to come up with those names; there are many more out there). Maybe the determining factor is something other than abrasiveness, maybe it's just balanced against demand for your project or other factors, maybe it depends on the market segment you're operating in - or maybe it's circumstance and luck that really matter.
I don't think that's the right way of looking at this.
It's hard to deny that Linus would not lose influence by having a modicum of self control and not saying someone should have been aborted because of a bad patch. Sure, maybe DJB won't become uber-famous by being nicer, but at the very least he wouldn't generate these sorts of conversations every time he writes something.
There's so many documented cases of people saying they don't want to work with Torvalds or Kalanick. In exchange for what? Being able to be some massive edgelord?
If even a single good core Linux developer would otherwise be contributing, Torvald's abrassiveness would be a net negative I think.
It's hard to deny that Linus would not lose influence by having a modicum of self control and not saying someone should have been aborted because of a bad patch. Sure, maybe DJB won't become uber-famous by being nicer, but at the very least he wouldn't generate these sorts of conversations every time he writes something.
There's so many documented cases of people saying they don't want to work with Torvalds or Kalanick. In exchange for what? Being able to be some massive edgelord?
If even a single good core Linux developer would otherwise be contributing, Torvald's abrassiveness would be a net negative I think.
But no one have mentioned licensing issues. Linux or *BSD do have quite liberal licenses comparing to QMail license or at least last time I read it, it was quite years ago. I have maintained my own set of patches with some of my own additions but it was a hurdle and it didn't make exchanging of those easy. My point it, maybe it is not only abrassiveness issue?
QMail is public domain.
Oh, I see now that it went public domain in 2007. I had used it prior to the change (migrated to Exim and the Postfix later). Wonder if it would get a better traction and community behind it if it was OSI-compilant from the start?
Except that he does not generate these sorts of conversations, given then he has written plenty since qmail and no such conversations have happened.
Kids, I speak as one of the few people, possibly the only person, in this discussion who has actually been subject to this. Daniel J. Bernstein is not in the same class of people as the people who call for retroactive abortions of their correspondents or who call people "slimy hypocritical asshole". It is a calumny to put him in that class.
And if I, who actually has been subject to this, say this, all of you, who have not, do not have a leg to stand upon.
Kids, I speak as one of the few people, possibly the only person, in this discussion who has actually been subject to this. Daniel J. Bernstein is not in the same class of people as the people who call for retroactive abortions of their correspondents or who call people "slimy hypocritical asshole". It is a calumny to put him in that class.
And if I, who actually has been subject to this, say this, all of you, who have not, do not have a leg to stand upon.
Sorry, I might have been overly generalizing here. I'm not aware of djb's behavior in particular, and was working off of the comparison that OP had made.
I had cited the abortion example as it's perhaps the most well-known example of Torvald's behavior. I should have made that more clear.
I had cited the abortion example as it's perhaps the most well-known example of Torvald's behavior. I should have made that more clear.
FYI: "Some thoughts on security after ten years of qmail 1.0" (by DJB, of course).
Abstract:
The qmail software package is a widely used Internet-mail transfer agent that has been covered by a security guarantee since 1997. In this paper, the qmail author reviews the his- tory and security-relevant architecture of qmail; articulates partitioning standards that qmail fails to meet; analyzes the engineering that has allowed qmail to survive this failure; and draws various conclusions regarding the future of secure programming.
Abstract:
The qmail software package is a widely used Internet-mail transfer agent that has been covered by a security guarantee since 1997. In this paper, the qmail author reviews the his- tory and security-relevant architecture of qmail; articulates partitioning standards that qmail fails to meet; analyzes the engineering that has allowed qmail to survive this failure; and draws various conclusions regarding the future of secure programming.
The qmail security guarantee was challenging. I made it a try and actually I found a bug
http://adivo.se/qmail/qmail-1.03.qmail_local_c.patch
but it was no security bug. Unfortunately I can't find any mailing list archive of qmail where the bug was discussed.
http://adivo.se/qmail/qmail-1.03.qmail_local_c.patch
but it was no security bug. Unfortunately I can't find any mailing list archive of qmail where the bug was discussed.
You've been on http://jdebp.eu./FGA/qmail-problems.html for about 14 years, and your patch was one of the ones that went into netqmail.
I'm glad that the patch came into use.
Back in 2001 when I was investigating the dependencies I wanted to take on for a webmail service, qmail was on my list of possible candidates, also because of this security guarantee.
However, I quickly learned that while that security guarantee probably was a good thing and valid, practically nobody was actually running vanilla qmail because it also had a very limited feature set.
What people were running was a mixture of qmail with various patches of their own or downloaded from somewhere on the internet, none of them properly maintained and all of them likely full of security issues nobody was caring about.
However, if you asked them what they were running, they were still saying qmail and they were still quoting the security guarantee which at that point was completely worthless.
In my case I went with a "batteries included" solution that didn't provide a security guarantee but also came with all the features I needed built-in and which also got regular updates by its maintainer.
Not needing to add third-party patches was more important than the design and guarantees of the base-product because the moment you patch the thing yourself, you own it and all the possibly given guarantees and positive aspects of the underlying architecture were null and void.
The solution I went with in the end was exim and it still is the MTA on our central relay. It had more security issues than qmail or postfix over its history, but I also do not need to patch it for my solution with users-in-SQL. Meaning that whenever a security issues is found, I can rely on my distro to provide updates which will fix the issue.
qmail and postfix provided a better security picture out-of-the-box but both would have needed manual patching for my needs which I deemed too risky.
There is no question that the less functionality offered by a product, the more secure it is. If you don't need any features, then by all means go with a product that has no features and you'll automatically enjoy the best security: Features breed complexity. Complexity breeds security issues.
But if you absolutely need a list of features that is offered by a tool, be careful in weighing up the risks of manually adding those additional, things compared to going with the full solution where somebody else is taking care of the security picture.
However, I quickly learned that while that security guarantee probably was a good thing and valid, practically nobody was actually running vanilla qmail because it also had a very limited feature set.
What people were running was a mixture of qmail with various patches of their own or downloaded from somewhere on the internet, none of them properly maintained and all of them likely full of security issues nobody was caring about.
However, if you asked them what they were running, they were still saying qmail and they were still quoting the security guarantee which at that point was completely worthless.
In my case I went with a "batteries included" solution that didn't provide a security guarantee but also came with all the features I needed built-in and which also got regular updates by its maintainer.
Not needing to add third-party patches was more important than the design and guarantees of the base-product because the moment you patch the thing yourself, you own it and all the possibly given guarantees and positive aspects of the underlying architecture were null and void.
The solution I went with in the end was exim and it still is the MTA on our central relay. It had more security issues than qmail or postfix over its history, but I also do not need to patch it for my solution with users-in-SQL. Meaning that whenever a security issues is found, I can rely on my distro to provide updates which will fix the issue.
qmail and postfix provided a better security picture out-of-the-box but both would have needed manual patching for my needs which I deemed too risky.
There is no question that the less functionality offered by a product, the more secure it is. If you don't need any features, then by all means go with a product that has no features and you'll automatically enjoy the best security: Features breed complexity. Complexity breeds security issues.
But if you absolutely need a list of features that is offered by a tool, be careful in weighing up the risks of manually adding those additional, things compared to going with the full solution where somebody else is taking care of the security picture.
> which at that point was completely worthless.
I disagree. qmail's design decisions meant that a lot of the patches were insulated even from themselves.
It also made me a better programmer and a better sysadmin.
I disagree. qmail's design decisions meant that a lot of the patches were insulated even from themselves.
It also made me a better programmer and a better sysadmin.
I still run qmail (in addition to postfix and opensmtpd) for one thing: ezmlm-idx.
Is there something I can replace ezmlm-idx with, that would seamlessly import all the lists, subscribers and archives?
Is there something I can replace ezmlm-idx with, that would seamlessly import all the lists, subscribers and archives?
http://mlmmj.org/ claims to be heavily inspired by ezmlm. I haven't used either so I don't know how well it works as a replacement, but it might be worth a look.
Ah yes, the one that djb denied to somebody who found a resource exhaustion
hole, from what I remember.
> In May 2005, Georgi Guninski claimed that some potential 64-bit portability problems allowed a ``remote exploit in qmail-smtpd.'' This claim is denied. Nobody gives gigabytes of memory to each qmail-smtpd process, so there is no problem with qmail's assumption that allocated array lengths fit comfortably into 32 bits.
That's adorably quaint. I wonder if djb still believes that's the case?
qmail-smtpd is a very small binary that is run per connection received through tcpserver.
There is no good reason to give it even a gigabyte of ram, let alone many multiple gigs.
I would say the notion that small binaries working together (the unix way) would still allow him to believe to be the case.
There is no good reason to give it even a gigabyte of ram, let alone many multiple gigs.
I would say the notion that small binaries working together (the unix way) would still allow him to believe to be the case.
Nobody "gives" set amounts of memory to their processes at all, so this thinking is backwards to begin with. The vulnerability is legitimate.
I seem to remember that the installation directions included something to limit memory size, and trying to install without reading the directions would fail well before opening a listener port.
systemd by default limits programs to a certain amount of memory (this bit me in the ass while working on getting OpenStack running on Ubuntu 16.04, and MySQL would get killed because of memory usage).
Various other systems also limit with things like /etc/login.conf on FreeBSD. There is also /etc/security/limits.conf on Linux for non-systemd stuff.
Limits for memory usage/open files/things of that nature are natural. You don't want one runaway process to cause the rest of your system to OOM.
Various other systems also limit with things like /etc/login.conf on FreeBSD. There is also /etc/security/limits.conf on Linux for non-systemd stuff.
Limits for memory usage/open files/things of that nature are natural. You don't want one runaway process to cause the rest of your system to OOM.
Sounds like the no true scottsman fallacy to me.
There is actually a section on the web site that addresses that one.
"write bug-free code"
There's no such a thing, specially when you decide it's a good idea to rewrite libc.
But yeah, the "stdio seem designed to encourage bugs" phrase seems pretty correct.
There's no such a thing, specially when you decide it's a good idea to rewrite libc.
But yeah, the "stdio seem designed to encourage bugs" phrase seems pretty correct.
That's missing the point.
You can either learn from his techniques and improve your skill, or you can argue about definitions and not improve your skill.
There is no doubt that DJB has good coding skills.
You can either learn from his techniques and improve your skill, or you can argue about definitions and not improve your skill.
There is no doubt that DJB has good coding skills.
Because we need to have a little hero-worship here: http://www.aaronsw.com/weblog/001502 (see section on writing bug free code)
I personally find it hopeful that such people, and such techniques, exist.
I personally find it hopeful that such people, and such techniques, exist.
When QMail was initially conceived, Sendmail was king, and Sendmail was awful (from a security perspective). Today there are multiple very good alternatives to Sendmail (I've used Postfix for many years with no complaints), and QMail has seen very sparse development. While Postfix has a moderately less stellar security record, it's still pretty darned good, and still under active maintenance.
Many modern mail security features are either hacked onto QMail via patches from third parties (none of whom are as famous for their security chops as djb) or completely non-existent in QMail. The number of people running it today is extremely small, so you're going to find relatively few resources about it when you run into problems (and what resources you find will be very, very, very old, in most cases).
If you're wanting a similarly limited SMTP server that is under active maintenance, opensmtpd from the OpenBSD folks might be an option (though I've never used it). It, at least, has people looking at it regularly and supports basic features lacking in QMail, like encryption, without patches.
QMail has a tiny but rabid following (kinda like djb himself) who settled into their beliefs about the MTA landscape in 1998 and have ignored everything that's happened since. (Funny, or maybe just odd, story: When we removed qmail-ldap, an unmaintained patched version of QMail that adds LDAP authentication, from Virtualmin, I literally had someone bringing up companies that have been out of business for more than a decade as proof that it was still in active use and we should keep maintaining support for it, even though no one has asked a question about it in our forums in eight years. It was the most bizarre technical conversation I've had in a while. I was like, "But, umm...they have no website and every mention I can find of that company is from 2006. What makes you think they're still using qmail-ldap or would care that we're removing support for it?" Then again, they also wanted to rant for a couple dozen paragraphs about gay people oppressing them in our issue tracker and for an hour or so in our IRC channel, so he may not have been entirely connected to our reality.)