Ask HN: The middle ground for email self-hosting?
104 comments
I have been selfhosting my email stack on a cheap VPS (~ $10) for many years now, probably since 2009 or so. I used to set up everything manually, but that was quite painful and there were some rough edges.
Nowadays, I just use mailcow (https://github.com/mailcow/mailcow-dockerized) for the setup part and have a much more polished experience. Email deliverability is not a problem. Generally, you just have to make sure to correctly setup DKIM/SPF (and DMARC) and check if your IP is on some blacklist. You can get it removed easily. (Edit: Also required is forward-confirmed reverse DNS, see below).
There was one provider that denied incoming mails from me, even though I got the IP removed from every blacklist I could find. I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP. Since being whitelisted there, everything works like a charm, couldn't be happier.
Nowadays, I just use mailcow (https://github.com/mailcow/mailcow-dockerized) for the setup part and have a much more polished experience. Email deliverability is not a problem. Generally, you just have to make sure to correctly setup DKIM/SPF (and DMARC) and check if your IP is on some blacklist. You can get it removed easily. (Edit: Also required is forward-confirmed reverse DNS, see below).
There was one provider that denied incoming mails from me, even though I got the IP removed from every blacklist I could find. I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP. Since being whitelisted there, everything works like a charm, couldn't be happier.
> I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP.
Interesting. Was that just to prove to that particular provider that you, the emailer, own the domain? Or is it some more widely used (beyond that provider) practice?
Interesting. Was that just to prove to that particular provider that you, the emailer, own the domain? Or is it some more widely used (beyond that provider) practice?
Afaik, it is specific to this provider. I'm not entirely sure why exactly tbh, probably to require some kind of associated identity (the policy is likely targeted to larger providers). It was a manual process and I never had to do anything similar for a different one.
They require that
a) the sending IP address has a PTR record b) from a domain that you own c) that resolves to the same IP address.
This is also very important for general deliverability (https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS).
They furthermore recommend d) that your host name should clearly mark it as a mail server and e) to make sure the domain leads to a web page that contains provider details and contact information.
They require that
a) the sending IP address has a PTR record b) from a domain that you own c) that resolves to the same IP address.
This is also very important for general deliverability (https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS).
They furthermore recommend d) that your host name should clearly mark it as a mail server and e) to make sure the domain leads to a web page that contains provider details and contact information.
When I hosted my own mail I did all of that except (e). I had trouble, as seems to be typical, with deliverability to Microsoft-hosted addresses. I wonder if (e) would have helped (doubt it).
Lemme guess, Well-known Security?
As in `http://mydomain.example/.well-known/security.txt ?
https://securitytxt.org/
As in `http://mydomain.example/.well-known/security.txt ?
https://securitytxt.org/
Nope, it really was a static HTML page containing a name and contact information, like you would write on traditional mail. ^^
> I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP.
T-Online?
T-Online?
Many VPS providers have an SMTP gateway that you can use to ensure that your email does not get marked as spam. I use transip.eu and never have problems. Prior to setting up my mail server to use their gateway, I ONLY had problems with Microsoft-based receivers such as @outlook.com. So please, don't use Microsoft for anything email related, as they are currently one of the worst offenders in making it hard for people to run their own mail server.
[deleted]
I do outbound email for my domain from a generic Gmail account, where my domain address is added as an alternate "Send mail as". This gets sent through Mailgun's free SMTP.
Incoming email to my domain is forwarded through Cloudflare's free service to the generic Gmail account.
This seems to pass all quality checks to avoid being sent to spam.
Incoming email to my domain is forwarded through Cloudflare's free service to the generic Gmail account.
This seems to pass all quality checks to avoid being sent to spam.
I do this but with Namecheap + Gmail. It has worked flawlessly for years.
The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some.
The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some.
> The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some.
I have a somewhat similar setup, slightly different. Most emails arrive within 10-15 seconds, but sometimes gets stuck somewhere along the line (as it happens with emails), which is normally not a problem.
But some platforms force you to use the "send link to login via email" option for login, which again, normally is not a problem. But when they have a timeout of 10 minutes + it takes 15 minutes for it to arrive, you end up not being able to login.
Only happened to me a few times during the years of this setup, but when it does happen, it really sucks.
I have a somewhat similar setup, slightly different. Most emails arrive within 10-15 seconds, but sometimes gets stuck somewhere along the line (as it happens with emails), which is normally not a problem.
But some platforms force you to use the "send link to login via email" option for login, which again, normally is not a problem. But when they have a timeout of 10 minutes + it takes 15 minutes for it to arrive, you end up not being able to login.
Only happened to me a few times during the years of this setup, but when it does happen, it really sucks.
This is called "greylisting" and it's fairly normal if you don't run your own server. Someone triggers a spam block and gets put on a partial time-out. Then the more times they try to re-send an email to you, the longer their emails get kept in limbo.
If you don't control the server, you don't have independent email.
If you don't control the server, you don't have independent email.
Well, could be that, could be other things as well. Could be the sender who have implemented their email sending via a queue, and currently they are overloaded. Could be their email sending server/service who is behind and having delivery problems. Could be numerous things thanks to the nature of email.
In the end, the UX of having to wait for an email sucks, sometimes.
In the end, the UX of having to wait for an email sucks, sometimes.
True.
The outbound queuing thing is going away, though. A lot of banks and service companies used to wait until a regular 2, 3 or 5 minute mark (00:00, 00:02) to send those verification emails in batch. In the past year or so there's been a move away from that it seems. The initial reason for batching them was to prevent spammers from triggering hordes of emails somehow off their system by gaming the "submit" button for a form, but now there's other security in place and the emails are being sent out faster. Generally, once an email is sent, it's almost instantaneously received if the recipient isn't delaying or blocking it. 99% of the time when I hear from clients who are waiting for an email, it either came in right away and bounced because our spam filters were too tight, or our own greylisting kicked in and slowed it down.
A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings.
Seems like it shouldn't be hard to check and collect reference statistics with a survey, though I'm failing to find surveys of that kind, and getting accounts on public services would be the tricky part for me personally (since I don't like to provide my phone number), so not doing that myself either. Only occasionally tried to check it with others, and messages were delivered fine in those cases -- but that's just a few samples.
Seems like it shouldn't be hard to check and collect reference statistics with a survey, though I'm failing to find surveys of that kind, and getting accounts on public services would be the tricky part for me personally (since I don't like to provide my phone number), so not doing that myself either. Only occasionally tried to check it with others, and messages were delivered fine in those cases -- but that's just a few samples.
The problem with deliverability issues is the impossibility of proving a negative.
If I send an e-mail to a company's customer support, or to my senator, or I reply to a potential client, or I contact an open source mailing list and I don't receive a reply - do I know if my message made it to them or not?
I mean, it's plausible that [email protected] just didn't deign to reply to my e-mail. But it's equally plausible there's some subtle misconfiguration - like an e-mail forwarder that breaks the SPF signature. It's not like I can sign up for a senate.gov e-mail address to test with.
Meanwhile, to paraphrase an old joke, when your senator rejects your e-mails you have a problem. When your senator rejects @gmail.com they have a problem.
If I send an e-mail to a company's customer support, or to my senator, or I reply to a potential client, or I contact an open source mailing list and I don't receive a reply - do I know if my message made it to them or not?
I mean, it's plausible that [email protected] just didn't deign to reply to my e-mail. But it's equally plausible there's some subtle misconfiguration - like an e-mail forwarder that breaks the SPF signature. It's not like I can sign up for a senate.gov e-mail address to test with.
Meanwhile, to paraphrase an old joke, when your senator rejects your e-mails you have a problem. When your senator rejects @gmail.com they have a problem.
Sure, strictly speaking it's impossible to ensure that a message was actually read by a user even with automated end-to-end delivery acknowledgements and/or in centralized systems: UIs manage to gobble/hide messages, users fail to find how to open attached documents (and declare that those are missing), etc. But I imagine that a survey/statistics would still help to estimate how bad deliverability in general (in a variety of common cases) is: without that there are differing and even more vague ideas of its state.
I can prove a different negative with my own mailserver - when I've sent things to @gov, they've always been responded to. I think that just proves the government reads ALL their spam.
I can only contribute my own experience. I have a dedicated server with an IP address in a datacentre, with approx 6 users using my email server for their primary email. DKIM/DMARC/SPF all configured correctly. I also have policies that suspend logins for accounts if they send too much in a certain timeframe, because this is a pretty good indicator of account compromise. The limits would never be hit by humans.
I've had three issues. The first was delivering to outlook.com, but this was temporary and resolved relatively quickly: I simply contacted their support. At the time, they didn't bother to validate DKIM or DMARC according to their headers.
The second was a sender sending to us with a misconfigured SPF policy. I had quite strict rules that spf failure => user's junk folder that I had to relax, but I also had a discussion with the admins at the sending company to explain the issue.
The third was yahoo. For reasons known only to them, they decided that IPs they've never seen before will be blocked by returning an smtp deferral that is permanent, which is bad for legitimate mail servers because the email remains stuck in the mail queue forever. I ended up discussing this with their support also and after some discussion that block too was removed.
That's pretty much it. I receive dmarc reports now from many providers so I've an idea what percentage of our email is quarantined or rejected (none). I've been running email since 2011, for my own main email and a few others. I don't think deliverability is that much of an issue and I was able to resolve all the problems I've had in 10+ years of doing this by emailing support, explaining myself and asking to be unblocked. Usually this simply resulted in "OK but if you do bad things we will block you no guarantee of inbox delivery etc etc etc". That's fine. It seems that there is a large degree of per-account spam filtering as well at the big providers mapping to individual users' preferences.
Of course, if you don't set up SPF/DKIM/DMARC, or you have an IP with poor reputation (you can check the DNSBL) or worse a residential address, you will have trouble. I would generally look for a provider that has a relatively strict acceptable use policy, and in particular doesn't allow VPN endpoints to be run from their infra for your email, to reduce the chances your IP has a terrible reputation with the big providers. Also, join all the sender programmes, set reverse dns, don't let your users do things like send bulk email and that will reduce many of the problems.
I've had three issues. The first was delivering to outlook.com, but this was temporary and resolved relatively quickly: I simply contacted their support. At the time, they didn't bother to validate DKIM or DMARC according to their headers.
The second was a sender sending to us with a misconfigured SPF policy. I had quite strict rules that spf failure => user's junk folder that I had to relax, but I also had a discussion with the admins at the sending company to explain the issue.
The third was yahoo. For reasons known only to them, they decided that IPs they've never seen before will be blocked by returning an smtp deferral that is permanent, which is bad for legitimate mail servers because the email remains stuck in the mail queue forever. I ended up discussing this with their support also and after some discussion that block too was removed.
That's pretty much it. I receive dmarc reports now from many providers so I've an idea what percentage of our email is quarantined or rejected (none). I've been running email since 2011, for my own main email and a few others. I don't think deliverability is that much of an issue and I was able to resolve all the problems I've had in 10+ years of doing this by emailing support, explaining myself and asking to be unblocked. Usually this simply resulted in "OK but if you do bad things we will block you no guarantee of inbox delivery etc etc etc". That's fine. It seems that there is a large degree of per-account spam filtering as well at the big providers mapping to individual users' preferences.
Of course, if you don't set up SPF/DKIM/DMARC, or you have an IP with poor reputation (you can check the DNSBL) or worse a residential address, you will have trouble. I would generally look for a provider that has a relatively strict acceptable use policy, and in particular doesn't allow VPN endpoints to be run from their infra for your email, to reduce the chances your IP has a terrible reputation with the big providers. Also, join all the sender programmes, set reverse dns, don't let your users do things like send bulk email and that will reduce many of the problems.
The third was yahoo. For reasons known only to them
I had to work around them for some datacenter mail relays. The only solution I found was to sum up the number of mail relays behind a SNAT and then apply rate limits for their domain to not exceed 6 concurrent connections total per SNAT. To your point and AFAIK they do not document this anywhere.
I had to work around them for some datacenter mail relays. The only solution I found was to sum up the number of mail relays behind a SNAT and then apply rate limits for their domain to not exceed 6 concurrent connections total per SNAT. To your point and AFAIK they do not document this anywhere.
I'm fairly sure they were abusing the standard with this particular technique. Deferred messages are OK if they can be retried later and that's what the MTA will try to do. Permanent deferral I suspect is really supposed to mean "we can't deliver right now and we don't know when but keep retrying".
What this does not do is trigger "undeliverable mail returned to sender" messages, so the end user has no idea their message is stuck in their own MTA's mailqueue until the MTA decides it has tried enough and gives up, and MTAs will usually persist for quite some time.
Spammers won't care what error code you send them or even worry about deferral messages, which is why the temporarily deferred spam trick works (first send is deferred for 1 hour, if you are a genuine MTA and you try again respecting this, it works). But permanent deferral, as I say, is very user hostile. The user thinks they've sent an email, but it isn't in the spam folder of the recipient. The sysadmin then has to go and dig to find out what exactly happened, and remove the mail from the mailqueue.
Luckily so far as I know we are only emailing a single yahoo address.
tl;dr the technique they are using is designed to handle the case where the receiving MTA is offline temporarily. There is a spam defence trick you can use and I don't object too much to that, but they used it to implement their block list rather than outright rejecting, and set the timeout to deferred indefinitely, which is just bad.
What this does not do is trigger "undeliverable mail returned to sender" messages, so the end user has no idea their message is stuck in their own MTA's mailqueue until the MTA decides it has tried enough and gives up, and MTAs will usually persist for quite some time.
Spammers won't care what error code you send them or even worry about deferral messages, which is why the temporarily deferred spam trick works (first send is deferred for 1 hour, if you are a genuine MTA and you try again respecting this, it works). But permanent deferral, as I say, is very user hostile. The user thinks they've sent an email, but it isn't in the spam folder of the recipient. The sysadmin then has to go and dig to find out what exactly happened, and remove the mail from the mailqueue.
Luckily so far as I know we are only emailing a single yahoo address.
tl;dr the technique they are using is designed to handle the case where the receiving MTA is offline temporarily. There is a spam defence trick you can use and I don't object too much to that, but they used it to implement their block list rather than outright rejecting, and set the timeout to deferred indefinitely, which is just bad.
For me, thinking about potential deliverability issues is just too stressful. Even if it works 99% of the time, who knows how important the remaining 1% of emails will be. Personally, having control over my own domain is a good-enough middle ground.
Been doing this long enough that I can sense right away if people aren't getting my emails. It's not great to call people up and ask if they got an email, and admit that my mail might not be getting through today, but it's not terrible. Gives me a chance to touch base with them for a minute, since I was emailing them anyway.
Having people whitelist you on google / yahoo / msn because you explicitly ask them to does have a wider effect, as far as I can tell, of keeping your emails in the clear for everyone else.
Having people whitelist you on google / yahoo / msn because you explicitly ask them to does have a wider effect, as far as I can tell, of keeping your emails in the clear for everyone else.
But there's no deliverability guarantee with such a setup either. AIUI the premise of this thread is that the chances of delivery are higher with it, which may be the case (especially taking into account possibilities of misconfiguration on the sender's side), but that's precisely what made me to wonder about a survey/statistics once again: I wonder whether there's actually an observable difference.
> and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine
That is pretty much it. One factor is that once you are on a blacklist it can spread like wildfire and be much faf to get off them all again, so the risk is small but the hassle if it happens is high. Also if you send mail for numerous people there is going to be a much higher risk: every extra user/account/address is an extra hack target (do all your users have good, non-shared, passwords?) or just extra volume that might be accidentally classified as junk (and once something from your server gets classed that way, future content may get more aggressively analysed and more mistakes may happen).
I've run my own mail server, including sending mail directly, for many years and to my knowledge not had a significant delivery problem. But I have a few mitigating factors: the IPv4 address is essentially on a commercial ISP range, not one that looks like a residential account or a VPS service provider, and the ISP is one that takes junk mail seriously, so there is less “splash damage” potential, and the same range has been used this way for several years (the main sender has moved around that small range, when testing upgrades on a copy VM for instance, but never away from it entirely) so it never looks like a brand new mail server these days, I only serve myself and a very small number of other users, our outgoing mail volume is pretty low.
It is a bigger problem for hosting services (much bigger user-base and little control over what they might send) or if you are sending from one of their ranges, if sending from a residential ISP address range, if your volume is high (perhaps you have apps that send mail as well as your personal mail?), etc., but it can be a problem for everyone.
I'm rebuilding my mail service soon (moving off Zimbra to just configuring the parts myself, as we don't need the extra features these days, it is too chunky for just a mail server, and at the end of next year they stop releasing easy install packages for the non-paid users (they already have for v9., next year v8. hits EOL)) at which point I might reconsider where it is hosted and if I should be sending via a paid SMTP relay to let them worry about deliverability, though as far as I know I've not had a problem.
That is pretty much it. One factor is that once you are on a blacklist it can spread like wildfire and be much faf to get off them all again, so the risk is small but the hassle if it happens is high. Also if you send mail for numerous people there is going to be a much higher risk: every extra user/account/address is an extra hack target (do all your users have good, non-shared, passwords?) or just extra volume that might be accidentally classified as junk (and once something from your server gets classed that way, future content may get more aggressively analysed and more mistakes may happen).
I've run my own mail server, including sending mail directly, for many years and to my knowledge not had a significant delivery problem. But I have a few mitigating factors: the IPv4 address is essentially on a commercial ISP range, not one that looks like a residential account or a VPS service provider, and the ISP is one that takes junk mail seriously, so there is less “splash damage” potential, and the same range has been used this way for several years (the main sender has moved around that small range, when testing upgrades on a copy VM for instance, but never away from it entirely) so it never looks like a brand new mail server these days, I only serve myself and a very small number of other users, our outgoing mail volume is pretty low.
It is a bigger problem for hosting services (much bigger user-base and little control over what they might send) or if you are sending from one of their ranges, if sending from a residential ISP address range, if your volume is high (perhaps you have apps that send mail as well as your personal mail?), etc., but it can be a problem for everyone.
I'm rebuilding my mail service soon (moving off Zimbra to just configuring the parts myself, as we don't need the extra features these days, it is too chunky for just a mail server, and at the end of next year they stop releasing easy install packages for the non-paid users (they already have for v9., next year v8. hits EOL)) at which point I might reconsider where it is hosted and if I should be sending via a paid SMTP relay to let them worry about deliverability, though as far as I know I've not had a problem.
>, and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings.
It's not just misconfigured email server settings like DKIM, SPF, DMARC etc. One can correctly set all of those and still have the outgoing emails rejected or spamholed. Why? Because the big email players like GMail, Microsoft Outlook.com, etc use black-box heuristics of reputation datapoints that exist outside the boundaries of email settings such as... "amount of email volume", "# of spam abuse reports from ip block", etc.
Because "sender reputation" cannot be encoded into an email configuration (DKIM/SPF/DMARC/etc), that's why nobody can provide a convenient Docker container with a perfectly working self-hosted email server that can reliably send email. If such a thing existed, the spammers would use it as well!
A datapoint such as "volume of email from this ip" is an unstated behavior/activity number and not an identity setting like DKIM.
And the invisible heuristics keep changing which causes previous email setups that worked -- to later stop working for no obvious reason. Why? Because there's a constant arms race between spammers and email filter algorithms. This means others' email spam heuristics that keep evolving and that you don't control -- blocks your self-hosted outbound emails without warning.
That's why you have example of skilled admins who know what they're doing and had a working self-hosted setup for years suddenly getting their emails rejected: https://www.tablix.org/~avian/blog/archives/2019/04/google_i...
As to the contradicting anecdotes about the difficulties of self-hosting email, the issue is that the conversation shares the same unstated environments in comments about Uber or umbrellas that affects how the writer perceives the truth or relevance of their anecdote.
- "The problems of self-hosted email getting blocked is overstated. I've been doing it and it's working fine."
- "I'm not sure what value Uber provides. Taxi services have smartphone apps."
- "I'm not sure why people use umbrellas. Every time I walk outside, it's not raining."
As an example of evangelists and advice-givers not noticing their unstated environments... Back in October 2017, a commenter (lucb1e) argued[1] that I was exaggerating the difficulties of reliably sending email but a year later in 2019, he eventually confirmed the same difficulties! [2]
[1] https://news.ycombinator.com/item?id=15525505
[2] https://news.ycombinator.com/item?id=19757607
It's not just misconfigured email server settings like DKIM, SPF, DMARC etc. One can correctly set all of those and still have the outgoing emails rejected or spamholed. Why? Because the big email players like GMail, Microsoft Outlook.com, etc use black-box heuristics of reputation datapoints that exist outside the boundaries of email settings such as... "amount of email volume", "# of spam abuse reports from ip block", etc.
Because "sender reputation" cannot be encoded into an email configuration (DKIM/SPF/DMARC/etc), that's why nobody can provide a convenient Docker container with a perfectly working self-hosted email server that can reliably send email. If such a thing existed, the spammers would use it as well!
A datapoint such as "volume of email from this ip" is an unstated behavior/activity number and not an identity setting like DKIM.
And the invisible heuristics keep changing which causes previous email setups that worked -- to later stop working for no obvious reason. Why? Because there's a constant arms race between spammers and email filter algorithms. This means others' email spam heuristics that keep evolving and that you don't control -- blocks your self-hosted outbound emails without warning.
That's why you have example of skilled admins who know what they're doing and had a working self-hosted setup for years suddenly getting their emails rejected: https://www.tablix.org/~avian/blog/archives/2019/04/google_i...
As to the contradicting anecdotes about the difficulties of self-hosting email, the issue is that the conversation shares the same unstated environments in comments about Uber or umbrellas that affects how the writer perceives the truth or relevance of their anecdote.
- "The problems of self-hosted email getting blocked is overstated. I've been doing it and it's working fine."
- "I'm not sure what value Uber provides. Taxi services have smartphone apps."
- "I'm not sure why people use umbrellas. Every time I walk outside, it's not raining."
As an example of evangelists and advice-givers not noticing their unstated environments... Back in October 2017, a commenter (lucb1e) argued[1] that I was exaggerating the difficulties of reliably sending email but a year later in 2019, he eventually confirmed the same difficulties! [2]
[1] https://news.ycombinator.com/item?id=15525505
[2] https://news.ycombinator.com/item?id=19757607
Oh yeah. This is very true, and getting worse every year.
I've had this discussion on HN before. It's gotten to the point where I've had to have my clients and their corporate lawyers go to bat against mail providers to maintain deliverability. No mail provider has any interest whatsoever in allowing an independent mailserver to continue delivering now.
So far, legal threats have worked when push came to shove against certain networks. But I imagine the difficulty is only going to increase.
I've had this discussion on HN before. It's gotten to the point where I've had to have my clients and their corporate lawyers go to bat against mail providers to maintain deliverability. No mail provider has any interest whatsoever in allowing an independent mailserver to continue delivering now.
So far, legal threats have worked when push came to shove against certain networks. But I imagine the difficulty is only going to increase.
Im extremely curious the legal precedents you used to accomplish this, particularly around forcing certain providers to un-spam or un-block your emails. What was the condition your legals found in order to do that?
That definitely outsources the most painful part of the problem. Though I'd probably use a forwarding-specific service - e.g. a lot of people swear by SES, I've used smtp2go just for Hotmail deliveries, I'm sure there are others.
You might have a bit of SPF fiddling to do, just because you might be fighting the default self-hosting assumption that incoming and outgoing servers are the same.
You might have a bit of SPF fiddling to do, just because you might be fighting the default self-hosting assumption that incoming and outgoing servers are the same.
The middle ground is to keep copies of all emails on your computer through a email client (via IMAP or POP), so that when you are locked out by your email provider, you can resume your business and communication with your contacts with your new email address. With IMAP, you can even upload your old emails to your next provider.
Incoming email is simple, MTAs have no problem at all delivering to residential IPs if that's what your MX says. So ideally you should run your own postfix + dovecot at your premises and point your MX to that. You have to take additional steps for sending email.
> would my custom domain be free from being marked by spam?
The receipient's mail service gets to choose if it thinks your email is spam, this will happen whatever your sending arrangements, outlook is not immune from sending spam and is no magic guarantee others will give it a free pass somehow.
Recipients score your email on a variety of characteristics, many of which are under your control. A major consideration is the sending netblock, eg, residential ADSL blocks are likely to be rejected or scored to hell. Garbage netblocks like linode with a terrible reputation likewise. A clean (no history of spamming) IP in a clean (reputable) netblock will be scored higher. You can look up sender reputations here, which is the service the big email providers use.
https://senderscore.org/
So to send your own mail, you should rent a dedicated server on your own IP, you can do this for $30/mo or so. All you need to run there is postfix + SASL auth to forward your (and only your) emails.
Then you must configure DKIM etc correctly and check your emails are validly signed, DKIM requires being able to add TXT fields to your DNS.
It's very possible to do this yourself securely after a bit of a learning curve and have it require minimal ongoing maintenance.
> would my custom domain be free from being marked by spam?
The receipient's mail service gets to choose if it thinks your email is spam, this will happen whatever your sending arrangements, outlook is not immune from sending spam and is no magic guarantee others will give it a free pass somehow.
Recipients score your email on a variety of characteristics, many of which are under your control. A major consideration is the sending netblock, eg, residential ADSL blocks are likely to be rejected or scored to hell. Garbage netblocks like linode with a terrible reputation likewise. A clean (no history of spamming) IP in a clean (reputable) netblock will be scored higher. You can look up sender reputations here, which is the service the big email providers use.
https://senderscore.org/
So to send your own mail, you should rent a dedicated server on your own IP, you can do this for $30/mo or so. All you need to run there is postfix + SASL auth to forward your (and only your) emails.
Then you must configure DKIM etc correctly and check your emails are validly signed, DKIM requires being able to add TXT fields to your DNS.
It's very possible to do this yourself securely after a bit of a learning curve and have it require minimal ongoing maintenance.
I think $30 a month is high just for email. I do $5/month now and have been doing this for over 20 years on my domain. (It was more expensive in the past.)
Yes, you can do it on a VPS much cheaper.
But this is your outgoing email authorized by DKIM... an attacker can use it to take over most of your accounts via Forgot Password flow. I think it is a false economy to have that depend on a shared VM.
But this is your outgoing email authorized by DKIM... an attacker can use it to take over most of your accounts via Forgot Password flow. I think it is a false economy to have that depend on a shared VM.
You don’t have to store your DKIM keys on the VPS. I keep my signing infrastructure local, and send outgoing mail over a WireGuard tunnel so it looks like it was sent from the VPS.
Do the same but use a http://postmarkapp.com or https://www.smtp2go.com account where you likely pay nothing unless you're a heavy sender.
Had an issue with my self hosted email going to spam and these services solved it.
Had an issue with my self hosted email going to spam and these services solved it.
I'm going to write this assuming you're a non-technical professional, lawyer perhaps, looking for a private email solution that doesn't rely on third parties.
Bottom line: There's no "middle ground", any middle ground you cede is allowing a third party some kind of access. Hosting your own email has become expensive and time-consuming (although IMHO it's still extremely worthwhile, and I do it in spite of what a pain in the ass it is). Be prepared to spend at least $50/mo and at least 6 hours in setup and 1-2 hours a month debugging if you do it personally. Or you can find someone to help (see below). You need your own IP address. You need a dedicated box, not a VPS. And check the IP address in advance to make sure it's clean, and not blacklisted. Tell the datacenter you're going to be doing email and ask them if they're okay with that for a clean IP. Use https://mxtoolbox.com/blacklists.aspx to test the IP address they're offering you, or IPs in their range. Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time.
My personal go-to would be dedicated hosting in the Netherlands, Switzerland, Isle of Man or Norway. Clean IPs, your own box, start with a clean server. But then you're talking $250/mo or so.
If you don't know how to set it up, there are people who can do it for you. You will need to essentially trust that person with access to all your correspondence, but if they do it properly, no one at the server farm[0] or elsewhere will have access to your correspondence... which puts you in the 0.01% of people on earth whose email isn't read by big tech companies.
[0] -who doesn't physically access the server: Look for ones in cages and ask who has physical access and why.
Bottom line: There's no "middle ground", any middle ground you cede is allowing a third party some kind of access. Hosting your own email has become expensive and time-consuming (although IMHO it's still extremely worthwhile, and I do it in spite of what a pain in the ass it is). Be prepared to spend at least $50/mo and at least 6 hours in setup and 1-2 hours a month debugging if you do it personally. Or you can find someone to help (see below). You need your own IP address. You need a dedicated box, not a VPS. And check the IP address in advance to make sure it's clean, and not blacklisted. Tell the datacenter you're going to be doing email and ask them if they're okay with that for a clean IP. Use https://mxtoolbox.com/blacklists.aspx to test the IP address they're offering you, or IPs in their range. Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time.
My personal go-to would be dedicated hosting in the Netherlands, Switzerland, Isle of Man or Norway. Clean IPs, your own box, start with a clean server. But then you're talking $250/mo or so.
If you don't know how to set it up, there are people who can do it for you. You will need to essentially trust that person with access to all your correspondence, but if they do it properly, no one at the server farm[0] or elsewhere will have access to your correspondence... which puts you in the 0.01% of people on earth whose email isn't read by big tech companies.
[0] -who doesn't physically access the server: Look for ones in cages and ask who has physical access and why.
> Be prepared to spend at least $50/mo and at least 6 hours in setup and 1-2 hours a month debugging if you do it personally. […] Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time.
That’s simply not true in that generality. I run my mail server on a VPS costing ~$7 per month (have been for ~20 years, switching the VPS provider once in that time) and mostly only have to do something when I major-upgrade Debian every 2-3 years. (Security updates are automated.) Some of my friends do the same. For the initial setup I would plan for more than 6 hours, it can certainly take some effort to work through all the details.
That’s simply not true in that generality. I run my mail server on a VPS costing ~$7 per month (have been for ~20 years, switching the VPS provider once in that time) and mostly only have to do something when I major-upgrade Debian every 2-3 years. (Security updates are automated.) Some of my friends do the same. For the initial setup I would plan for more than 6 hours, it can certainly take some effort to work through all the details.
> You need a dedicated box, not a VPS. … Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time.
I agree that hosting a mail server directly on a VPS compromises privacy and control. But there’s a better alternative: use VPSes for cheap static IPs, while hosting the server locally on hardware you physically control, using WireGuard tunnels and port forwarding to connect things. Port forward incoming SMTP over WireGuard to your real MX, and use MTA‐STS and DANE so that as many senders as possible will TLS‐encrypt mail they send you. Have your outgoing SMTP server handle DKIM signing, then send it out via WireGuard so it looks like it came from the VPS, while enforcing TLS encryption.
The VPS won’t be able to forge mail from you without your DKIM keys. It won’t be able to read your outgoing mail due to TLS. It won’t be able to read incoming mail that’s TLS encrypted. It will be able to read unencrypted mail, but the big providers that follow MTA‐STS will abort if the VPS attempts to block encrypted connections.
This has the added benefit of reducing your dependence on an external provider (the VPS company) for server setup. If you’re unhappy with a particular provider, just switch to another one. The issues associated with sending email from a brand new IP will be there, but you won’t have to set up complicated infrastructure on the new host, only a few WireGuard tunnels and firewall rules.
I agree that hosting a mail server directly on a VPS compromises privacy and control. But there’s a better alternative: use VPSes for cheap static IPs, while hosting the server locally on hardware you physically control, using WireGuard tunnels and port forwarding to connect things. Port forward incoming SMTP over WireGuard to your real MX, and use MTA‐STS and DANE so that as many senders as possible will TLS‐encrypt mail they send you. Have your outgoing SMTP server handle DKIM signing, then send it out via WireGuard so it looks like it came from the VPS, while enforcing TLS encryption.
The VPS won’t be able to forge mail from you without your DKIM keys. It won’t be able to read your outgoing mail due to TLS. It won’t be able to read incoming mail that’s TLS encrypted. It will be able to read unencrypted mail, but the big providers that follow MTA‐STS will abort if the VPS attempts to block encrypted connections.
This has the added benefit of reducing your dependence on an external provider (the VPS company) for server setup. If you’re unhappy with a particular provider, just switch to another one. The issues associated with sending email from a brand new IP will be there, but you won’t have to set up complicated infrastructure on the new host, only a few WireGuard tunnels and firewall rules.
That's a cool idea I'd never considered. And it's easy to set up. But then the main thing you're getting out of the VPS is the static IP. The main reason I was advising against VPS's, besides obvious vulnerabilities, was that their IP blocks get banned all the time. If you want to run a mailserver for a long time you need to cultivate that IP address's reputation for years, and you don't want it to be anywhere sketchy.
If you're paying for that, why not just pay for a static IP at home?
If you're paying for that, why not just pay for a static IP at home?
> the main thing you're getting out of the VPS is the static IP.
Yes, that and RDNS.
> If you're paying for that, why not just pay for a static IP at home?
That’s a good question. I too hear that mail providers consider IP blocks assigned to VPS providers less trustworthy than others. The reasons I don’t take the ISP/dedicated server route, aside from price, are:
• VPS providers are not tied to my physical location. If I move, I probably can’t take my ISP’s static IP with me (I may even move to somewhere they don’t service). Conversely, if I want to switch away from a local ISP, the selection of alternatives is extremely limited.
• Risk of neighboring IPs reducing the reputation of the block exists with server companies and local ISPs as well. I concede that the problem is probably worse with VPSes, but I hope to mitigate it somewhat by avoiding bottom‐of‐the‐barrel providers and by the fact that my own IP will never be used to spam.
• I’m somewhat worried about the possibility of DDOS, and VPS companies provide a lot of cheap bandwidth, so in case of attack I might be able to salvage the situation with careful firewalling on the VPS.
Yes, that and RDNS.
> If you're paying for that, why not just pay for a static IP at home?
That’s a good question. I too hear that mail providers consider IP blocks assigned to VPS providers less trustworthy than others. The reasons I don’t take the ISP/dedicated server route, aside from price, are:
• VPS providers are not tied to my physical location. If I move, I probably can’t take my ISP’s static IP with me (I may even move to somewhere they don’t service). Conversely, if I want to switch away from a local ISP, the selection of alternatives is extremely limited.
• Risk of neighboring IPs reducing the reputation of the block exists with server companies and local ISPs as well. I concede that the problem is probably worse with VPSes, but I hope to mitigate it somewhat by avoiding bottom‐of‐the‐barrel providers and by the fact that my own IP will never be used to spam.
• I’m somewhat worried about the possibility of DDOS, and VPS companies provide a lot of cheap bandwidth, so in case of attack I might be able to salvage the situation with careful firewalling on the VPS.
That all makes sense.
Although in my experience, if you're really being careful about not sending spam, you've got a lot more to worry about from your VPS being blacklisted than your own IP personally.
>> I’m somewhat worried about the possibility of DDOS
The one time I got severely dDoS'd, because I'd let a friend run a tiny static website off my server that attracted that kind of attention, the hosting company I was with shut my account down immediately and asked for $5000 in reparations. I had to backdoor into the server and salvage whatever I could. That was a hardened box in a military grade facility. I don't think a VPS is going to be kind. Push comes to shove, if it's in your house you can pull the cable.
>> I’m somewhat worried about the possibility of DDOS
The one time I got severely dDoS'd, because I'd let a friend run a tiny static website off my server that attracted that kind of attention, the hosting company I was with shut my account down immediately and asked for $5000 in reparations. I had to backdoor into the server and salvage whatever I could. That was a hardened box in a military grade facility. I don't think a VPS is going to be kind. Push comes to shove, if it's in your house you can pull the cable.
> Be prepared to spend at least $50/mo
> You need a dedicated box
> But then you're talking $250/mo
Nonsense.
You only need a $2-5/m cheap VPS for your server and have anything as MTA/smarthost for you. Eg: Fastmail, $5/m, just receive with a catch-all address, send through a SMTP with auth (with an app pass).
No bother with SPF, DKIM, PTR, SPAM and all this bullshit. Also you have a breaking glass access to your mail in case your server burst in flames.
Don't want Fastmail? Any other provider would do.
Want to tinker it yourself? Just buy anything anywhere, configure it as a smarthost.
Anecdata: I have a small business of ~15 people use Fastmail in the said configuration. The only difference is what they are on Exchange Server locally. The old one, on premises. They use it for almost 8 years without problems.
> You need a dedicated box
> But then you're talking $250/mo
Nonsense.
You only need a $2-5/m cheap VPS for your server and have anything as MTA/smarthost for you. Eg: Fastmail, $5/m, just receive with a catch-all address, send through a SMTP with auth (with an app pass).
No bother with SPF, DKIM, PTR, SPAM and all this bullshit. Also you have a breaking glass access to your mail in case your server burst in flames.
Don't want Fastmail? Any other provider would do.
Want to tinker it yourself? Just buy anything anywhere, configure it as a smarthost.
Anecdata: I have a small business of ~15 people use Fastmail in the said configuration. The only difference is what they are on Exchange Server locally. The old one, on premises. They use it for almost 8 years without problems.
To prevent 3rd party access you need to not only host your own email but all recipients of your mail need to be self hosted as well. Unless you are providing everyone you correspond with their own account, 3rd parties will be involved.
No, you just have to be aware of who you're sending to, and encrypt appropriately. I mean obviously if I send to someone @gmail, they're going to know I sent that person an email. But they can't scan my inbox for keywords.
If someone sends you an email from gmail, that’s also in gmail, and you can’t guarantee that the sender will encrypt their message.
That is a lot of money and surely few can afford it for their email. What are the advantages of it that make it worth the time and effort. Is it security? privacy? or just trying to keep things a bit more decentralised?
I just don't understand the attraction of self hosting email. The pain seems extreme, even for those who understand the considerable number of nuances.
To me the happy middle ground is email on your own domain but using an existing provider such as G / MS or whoever. That way you've got control but don't need to worry about the pain.
It does require paying for but really on balance not much. If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for.
The situation I've found frustrating is about family email on same domain. I've gone in a huge loop that has ended up back with GWorkspace which is quite costly for 3-4 family users. But still - not even close to the horror of self hosting...
To me the happy middle ground is email on your own domain but using an existing provider such as G / MS or whoever. That way you've got control but don't need to worry about the pain.
It does require paying for but really on balance not much. If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for.
The situation I've found frustrating is about family email on same domain. I've gone in a huge loop that has ended up back with GWorkspace which is quite costly for 3-4 family users. But still - not even close to the horror of self hosting...
I've selfhosted my own email for almost a decade now. I was spending about 5 min a month on it until my ip block rep died one day for little decernable reason. Then I wasted many hours on it so I switched to sending through amazon ses and I'm back to my 5 min a month (updating packages/checking logs).
It has a few advantages because I can run a bunch of automations and filters. Honestly I get less spam than in my gmail inbox, so I harvest spam from gmail to train my filters now. It's insane to me how much providers charge for an inbox and the tiny sizes many still offer
It has a few advantages because I can run a bunch of automations and filters. Honestly I get less spam than in my gmail inbox, so I harvest spam from gmail to train my filters now. It's insane to me how much providers charge for an inbox and the tiny sizes many still offer
If it was your IP block rep, then it could have been anyone else on your block who tanked everyone else on the blocks reputation. A few bad apples on a block /24 can tank the lot.
I also use SES for outbound. Haven't even thought of my selfhosted setup (run through Synology's MailPlus) since I first turned it on more than two years ago.
> If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for.
But what's the cost when Google's "AI" bans your account?
For paid email hosting I'd go to some provider with actual support...
But what's the cost when Google's "AI" bans your account?
For paid email hosting I'd go to some provider with actual support...
If you use your own domain (as the parent suggests), the cost is creating an account with any other e-mail provider that supports custom domains and updating DNS entries.
Since he uses his own domain, he can move elsewhere if there's a problem with Google. I do that too, and it's indeed much simpler than self-hosting.
On that note, I have amazon manage my custom domain.
Now I'm terrified that I'll return one too many pairs of socks or something, and get the retail consumer part of my account banned (we buy a lot of stuff on Amazon).
Does anyone have experience with what happens to AWS resources (specifically, domains) when that happens?
Now I'm terrified that I'll return one too many pairs of socks or something, and get the retail consumer part of my account banned (we buy a lot of stuff on Amazon).
Does anyone have experience with what happens to AWS resources (specifically, domains) when that happens?
It sounded like he's actually keeping the emails on google, even though they come on a custom domain. And that doesn't seem safe to me.
I use a client, so have a local IMAP backup, and regularly back this up to my NAS.
That's not really my worry, though. What scares me about being heavily invested in [email protected] or [email protected] is lockout from services. As long as I have my own domain then I can just switch out the DNS and I'll still be able to get into whatever web service I signed up to using my email address.
That's not really my worry, though. What scares me about being heavily invested in [email protected] or [email protected] is lockout from services. As long as I have my own domain then I can just switch out the DNS and I'll still be able to get into whatever web service I signed up to using my email address.
For backup he could get Thunderbird or similar to fetch the emails are regular interval. I think it's generally a good idea to do so.
The same thing camps happen with any company. If you have your own domain you can get a new email host, change your mx record. Update your email client to use the new credentials. If you use mobile email, upload existing recent email from your desktop or restore/ import from backup. This means not registering the domain with your email host to avoid problematic delays
The problem with google email hosting is you can no longer use that account for many of their services such as nest. They just don't support it. Same with google voice. I of course could create another gmail but that defeats the purpose of having my original email, I want everything in one place. If they fixed this I would say it's the best option.
I agree on your middle ground, with the added step to maintain a local backup of your mail (i.e. .eml files or similar).
> I just don't understand the attraction of self hosting email.
For several years I’ve hosted in the “middle ground” sense described by the OP, running my own incoming mail server and relaying outgoing mail through a big provider.
The main benefit for me (compared to using a big provider with my own domain) is personal privacy. When I used Google for mail, Google had access to so many pieces that make up my personal life: Purchase receipts. Flight itineraries. Conference registrations. Emails from my university. Emails from my realtor. Utility bills. Notifications for subscribed forum threads, GitHub repositories, Wikipedia pages. Whatever newsletters I chose to subscribe to. Theoretical access to any site with password reset by email. Running my own MX eliminates Google’s access to most of these things.
There are other some other benefits too. Free infinite aliases I can use to sign up on any website. No fear of dependence on features that might get paywalled. No sudden danger of having to migrate data to another provider.
> If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for.
Reducing my data footprint is something I care about enough to spend my spare time on.
For several years I’ve hosted in the “middle ground” sense described by the OP, running my own incoming mail server and relaying outgoing mail through a big provider.
The main benefit for me (compared to using a big provider with my own domain) is personal privacy. When I used Google for mail, Google had access to so many pieces that make up my personal life: Purchase receipts. Flight itineraries. Conference registrations. Emails from my university. Emails from my realtor. Utility bills. Notifications for subscribed forum threads, GitHub repositories, Wikipedia pages. Whatever newsletters I chose to subscribe to. Theoretical access to any site with password reset by email. Running my own MX eliminates Google’s access to most of these things.
There are other some other benefits too. Free infinite aliases I can use to sign up on any website. No fear of dependence on features that might get paywalled. No sudden danger of having to migrate data to another provider.
> If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for.
Reducing my data footprint is something I care about enough to spend my spare time on.
In my world (running a small digital agency), I've realised that even if I do all the things to reduce my data footprint (for instance, migrate all my docs to NextCloud, self host email, etc), it actually all breaks almost instantly - all it takes is a client to share a doc or folder with me where they use GDocs / Dropbox / whatever, and I'm effectively straight back in it.
My basic strategy is one of slight defeatism, I have to admit. I am 100% in to Google for their (really quite excellent) tools in Google Workspace: nothing is as good as GDocs, nothing is as good as Gmail, nothing is as good as Google Meet; but I do things to ensure I'm not utterly f**d if the Random Google AI Best happens to decide I'm some sort of unspecified menace. So for instance - I use Google Docs but only with .docx / .xlsx files rather than native .gdoc .gsheet files. I back this up automatically to my self-hosted NAS. I do this on a domain which I own, so can step away if things do happen to go south, or costs double or whatever.
Then I use kagi.com for search, and have a piHole / ublock / Brave to minimise footprint from a tracking POV.
I know, it's all probably moot given I just open up my inbox to Google, but I've tried and failed to find a provider that is even close to being the same balance of low price + utility. I got excited about Fastmail but turned out it was a combination of not very good AND really expensive once I factored in having several accounts on the same domain. I had a horrific experience with iCloud+ (they have a weird YEAR long account blocking issue thing that I won't go into now). M$ was awful and required me to send everything through GoDaddy's DNS. All the others were just underwhelming or expensive or both. So - sadly - I'm back in the G stable where I'll stay for the time being... :-)
My basic strategy is one of slight defeatism, I have to admit. I am 100% in to Google for their (really quite excellent) tools in Google Workspace: nothing is as good as GDocs, nothing is as good as Gmail, nothing is as good as Google Meet; but I do things to ensure I'm not utterly f**d if the Random Google AI Best happens to decide I'm some sort of unspecified menace. So for instance - I use Google Docs but only with .docx / .xlsx files rather than native .gdoc .gsheet files. I back this up automatically to my self-hosted NAS. I do this on a domain which I own, so can step away if things do happen to go south, or costs double or whatever.
Then I use kagi.com for search, and have a piHole / ublock / Brave to minimise footprint from a tracking POV.
I know, it's all probably moot given I just open up my inbox to Google, but I've tried and failed to find a provider that is even close to being the same balance of low price + utility. I got excited about Fastmail but turned out it was a combination of not very good AND really expensive once I factored in having several accounts on the same domain. I had a horrific experience with iCloud+ (they have a weird YEAR long account blocking issue thing that I won't go into now). M$ was awful and required me to send everything through GoDaddy's DNS. All the others were just underwhelming or expensive or both. So - sadly - I'm back in the G stable where I'll stay for the time being... :-)
I used to do something similar: I hosted my own IMAP, while using Gandi for receiving mail and sending mail. That meant I didn't lose mail on the receiving side if my mail server had an issue, and I didn't fail to deliver mail to others because I hadn't jumped through enough hoops. (Literally everyone I personally know who has run their own mail server has had one or both of those problems at least once.)
This worked well for me because it gave me the feeling of having more control and privacy and security over my email.
I switched away from that solution when I realized that in practice I have less ability to effectively provide security than the whole security and product teams of a major email provider.
This worked well for me because it gave me the feeling of having more control and privacy and security over my email.
I switched away from that solution when I realized that in practice I have less ability to effectively provide security than the whole security and product teams of a major email provider.
Whats's Gandi please?
https://www.gandi.net/
They're a domain registrar that also supplies email and hosting and similar services.
They're a domain registrar that also supplies email and hosting and similar services.
You can also try to use your domain registrar SMTP as relay, in case they offer you a mail service, and setting their SPF records to the domain. I have set it up with Gandi and works pretty good.
FWIW, a $4/m Exchange Online Plan 1 is probably enough. Or even the $1/m Exchange Online Protection if you don't have any need for a hosted mailbox. Anything that'll get you ongoing access to the Exchange Admin Center.
I use a single M365 Business Basic account, as a conventional mailbox, for one of my domains. From within the Exchange Admin Center there's extensive control over mail flow -- domains to accept mail for, inbound and outbound connectors for routing mail between on-prem mail servers. Best as I can tell, literally ongoing any subscription that gets you an account with access to EAC ought be enough to route any or all of your email through EO in either direction.
https://www.microsoft.com/en-us/microsoft-365/exchange/compa...
https://www.microsoft.com/en-us/microsoft-365/exchange/excha...
https://docs.microsoft.com/en-us/exchange/standalone-eop/sta...
https://docs.microsoft.com/en-us/exchange/mail-flow-best-pra...
I use a single M365 Business Basic account, as a conventional mailbox, for one of my domains. From within the Exchange Admin Center there's extensive control over mail flow -- domains to accept mail for, inbound and outbound connectors for routing mail between on-prem mail servers. Best as I can tell, literally ongoing any subscription that gets you an account with access to EAC ought be enough to route any or all of your email through EO in either direction.
https://www.microsoft.com/en-us/microsoft-365/exchange/compa...
https://www.microsoft.com/en-us/microsoft-365/exchange/excha...
https://docs.microsoft.com/en-us/exchange/standalone-eop/sta...
https://docs.microsoft.com/en-us/exchange/mail-flow-best-pra...
I've done this with Amazon SES, which is PAYG and costs me pennies.
https://www.pxeger.com/2020-07-02-hybrid-cloud-email-with-am...
It is a bit overcomplicated, because I also set up SES to receive email, but I could run that instead with an ordinary Postfix server. It would be much simpler for outgoing only, I think
https://www.pxeger.com/2020-07-02-hybrid-cloud-email-with-am...
It is a bit overcomplicated, because I also set up SES to receive email, but I could run that instead with an ordinary Postfix server. It would be much simpler for outgoing only, I think
SES is good for loads of outbound mail.
People don't know this, I mean no one knows this, but if you're running an EC2 instance with some allocated IPs, you can contact Amazon's customer service and ask them to unthrottle outbound mail on it. Typically they clamp down and stop connecting any SMTP deliveries off the EC2s if you do more than 10 emails/hr or something. But if you plan on keeping the IPs for awhile you can set up DKIM and SPF and all that, call Amazon and tell them you're sending and receiving legitimate business emails off that server. They may try to refer you to SES, but if you tell them you need to manage it on a private server for legitimate reasons, they have the ability to lift the block for you.
Do check your allocated IPs for blacklisting in advance, and obviously don't give them any cause for being blacklisted in the future.
People don't know this, I mean no one knows this, but if you're running an EC2 instance with some allocated IPs, you can contact Amazon's customer service and ask them to unthrottle outbound mail on it. Typically they clamp down and stop connecting any SMTP deliveries off the EC2s if you do more than 10 emails/hr or something. But if you plan on keeping the IPs for awhile you can set up DKIM and SPF and all that, call Amazon and tell them you're sending and receiving legitimate business emails off that server. They may try to refer you to SES, but if you tell them you need to manage it on a private server for legitimate reasons, they have the ability to lift the block for you.
Do check your allocated IPs for blacklisting in advance, and obviously don't give them any cause for being blacklisted in the future.
I know this is slightly off topic from OP's question, but I'm chiming in with one piece of first-hand advice:
You can self host mail alongside gmail/outlook on your own domain. More than one email service can run concurrently, without any problems.
That often overlooked fact allows you to quickly set up something like gmail on your domain, then use the trial period to see if you can self-host with any success. If you can, then you can shut down the trial, or move on to trial another paid service like 365 while you're still "trialing" your own host.
It really helped me make the transition.
You can self host mail alongside gmail/outlook on your own domain. More than one email service can run concurrently, without any problems.
That often overlooked fact allows you to quickly set up something like gmail on your domain, then use the trial period to see if you can self-host with any success. If you can, then you can shut down the trial, or move on to trial another paid service like 365 while you're still "trialing" your own host.
It really helped me make the transition.
I have good luck using https://forwardemail.net for having email address on my own custom domain. This will catch inbound email (configured from DNS MX record) to my gmail, and for outbound I set the custom domain as gmail alias.
Doesn't solve privacy, data ownership, nor google lock-in issue (but at least if I lost my gmail, I can move to a real email selfhost solution and keep my address). As my need is just to have custom domain address for the cool factor of it, this simple setup works flawlessly.
Doesn't solve privacy, data ownership, nor google lock-in issue (but at least if I lost my gmail, I can move to a real email selfhost solution and keep my address). As my need is just to have custom domain address for the cool factor of it, this simple setup works flawlessly.
The problem I'm having with a similar solution is that for some of my emails (that I used to test), are labeling the emails sent with the custom domain as gmail alias as "Promotion", so it doesn't go into the Inbox on gmail, goes to the Promotions tab - and makes no sense, since the only thing in my email structure I have is my name as a signature and the domain.
I do the same and I can't thank niftylettuce enough for it
You should be aware that Microsoft's SMTP servers parse the mime structure of the mails and restructure it in a non-standard way. I have no idea why, but for example this breaks PGP signatures.
This is due to a well known fact that Microsoft hates the email ecosystem.
I work on a email client and a large majority of "this email is broken" is due to weird outlook behavior. Most recently it's TNEF attachments: https://en.wikipedia.org/wiki/Transport_Neutral_Encapsulatio...
I work on a email client and a large majority of "this email is broken" is due to weird outlook behavior. Most recently it's TNEF attachments: https://en.wikipedia.org/wiki/Transport_Neutral_Encapsulatio...
They break way more than that, it's quite annoying. If you search in various MTA/client mailing lists you'll find a bunch of threads.
Though, others do violate standards as well, but not like this.
Though, others do violate standards as well, but not like this.
[deleted]
I have experience self hosting for incoming mails; outgoing mails are sent via mailchimp.
We have had this setup for several years. It is not difficult to setup, emails are delivered reliably, and email delivery cost is negligible.
Some of our users use Outlook / Thunderbird / Apple Mail as a client, some use GMail as a client (check external mail / send as user) and some use Rainloop which I set up on the mail server.
We have had this setup for several years. It is not difficult to setup, emails are delivered reliably, and email delivery cost is negligible.
Some of our users use Outlook / Thunderbird / Apple Mail as a client, some use GMail as a client (check external mail / send as user) and some use Rainloop which I set up on the mail server.
I use a free Gmail account as my mail client and direct all my custom domains from Fastmail to Gmail. Messages are delivered very quickly and I don't have spam issues.
Truth is I like Gmail but I think Google have dropped the ball with, "Let me point all my custom domains to a Gmail account. I would even pay you but I don't want Workspace".
Truth is I like Gmail but I think Google have dropped the ball with, "Let me point all my custom domains to a Gmail account. I would even pay you but I don't want Workspace".
Don't know about the effect on the spam ranking, but on the flip side you'd lose some privacy.
For example, email notifications sent by Stripe are delivered over TLS'd connections. My bank does this too. If you are to proxy these, the relay will obviously be in the loop on all emails that aren't local to your mail server.
For example, email notifications sent by Stripe are delivered over TLS'd connections. My bank does this too. If you are to proxy these, the relay will obviously be in the loop on all emails that aren't local to your mail server.
I think your best bet is Amazon ses. In my recent testing, all mail always go through - unless the recipient marks it thus.
It's damn cheap too, like almost free for low volumes.
I think Amazon uses this for their workmail also and has become pretty strict at policing abuse.
I am only speaking for gmail though, so ymmv for hotmail et al which I haven't checked.
It's damn cheap too, like almost free for low volumes.
I think Amazon uses this for their workmail also and has become pretty strict at policing abuse.
I am only speaking for gmail though, so ymmv for hotmail et al which I haven't checked.
Interesting just met a client who us using Amazon SES with a few domains, all of his emails go to spam, across all domains. Curious aye! Almost like these sender's aren't inpregnable and there is something else going on.
I agree. Amazingly cheap. Worth a little bit more technical hassle (looking at templating and batching for example IIRC you had to do for marketing or bulk transactional emails) but still amazing value for money.
I’d go AuthSMTP or similar rather than 365 for pure outbound as it will be much more cost effective.
Another alternative is https://thehelm.com:
Helm is a personal, private email server that won't share your data.
The Verge
Helm is a personal, private email server that won't share your data.
The Verge
I'd say just rent hosted dovecot from a reputable company and you're then free to build your own filters, tools, pipelines on top of that.
My favorite is mailbox.org
My favorite is mailbox.org
I did different setups over time, but currently settled on forwardemail.net for incoming catch-all custom domain and iCloud+ SMTP servers for sending.
I've done similar with AWS's SES before with no trouble - I imagine you'd have similar results with MSFT.
I’ve had severe deliverability issues with SES. You’re pooled with a lot of bad actors and occasionally you get a blacklisted outbound. They have a reputation system but it’s far from perfect and requires manual intervention if you’re blocked from sending which is a real pain.
For personal stuff I’m using iCloud+ domain hosting now after moving from fastmail which was also trouble free.
For personal stuff I’m using iCloud+ domain hosting now after moving from fastmail which was also trouble free.
This is how I do it, I can highly recommend it. My residential ISP provides a free relay service so I use that.
I'm still wondering why there's no go-to choice to spin up a docker container that has everything preconfigured.
Because the problem is mostly about the reputation of the IP address.
Setting up an email server is somewhat easy. Ensuring that other servers consider yours as legitimate that is the problem.
Setting up an email server is somewhat easy. Ensuring that other servers consider yours as legitimate that is the problem.
Spam is often sent from big providers, too. For several years I hosted my email the “middle ground” way (i.e., relaying outgoing mail via Google Workspace), and despite using DMARC correctly it was not infrequent that my emails would go to spam (even to GMail boxes) or never show up at all.
Obviously GMail is such a giant that email providers have to be very careful when blocking it, but enough spam comes from there that receivers clearly use some heuristics to block some of it. I’ve even received multiple rejection notices because the GMail server my email was sent through happened to be on a blacklist!
I switched last year to sending directly from my VPS. It was partly for privacy from Google, but moreso so I could enforce outgoing TLS. For the first few days they went to spam boxes or moderation queues, but I made sure they were rescued, and ever since I’ve had no deliverability issues sending to Google, some local ISPs, and even Microsoft (which seems crazy, as I never got a mail from my domain to show up in Outlook when I was relaying through Google).
I can only speak for my own experience, of course. But that is what I experienced.
Obviously GMail is such a giant that email providers have to be very careful when blocking it, but enough spam comes from there that receivers clearly use some heuristics to block some of it. I’ve even received multiple rejection notices because the GMail server my email was sent through happened to be on a blacklist!
I switched last year to sending directly from my VPS. It was partly for privacy from Google, but moreso so I could enforce outgoing TLS. For the first few days they went to spam boxes or moderation queues, but I made sure they were rescued, and ever since I’ve had no deliverability issues sending to Google, some local ISPs, and even Microsoft (which seems crazy, as I never got a mail from my domain to show up in Outlook when I was relaying through Google).
I can only speak for my own experience, of course. But that is what I experienced.
I have had very similar experiences.
Doing things perfectly and building up "reputation" helps a lot.
Doing things perfectly and building up "reputation" helps a lot.
At my company we are using dockerized mailcow on a Hetzner VPS and it has taken us some time to have the IP whitelisted in all major e-mail providers.
The easiest to work with have been Microsoft and Yahoo. I still haven't found a way to whitelist our IP on centurylink.net, charter.net and att.com (please let me know if you have any ideas)
Some links that might be of help to others:
[1] https://sendersupport.olc.protection.outlook.com/snds/data.a...
[2] https://io.help.yahoo.com/contact/index?page=confirmation&lo...
[3] https://postmaster.google.com/u/0/dashboards#st=userReported...
[4] https://support.microsoft.com/en-us/supportrequestform/8ad56...
[5] https://www.mail-tester.com/
The easiest to work with have been Microsoft and Yahoo. I still haven't found a way to whitelist our IP on centurylink.net, charter.net and att.com (please let me know if you have any ideas)
Some links that might be of help to others:
[1] https://sendersupport.olc.protection.outlook.com/snds/data.a...
[2] https://io.help.yahoo.com/contact/index?page=confirmation&lo...
[3] https://postmaster.google.com/u/0/dashboards#st=userReported...
[4] https://support.microsoft.com/en-us/supportrequestform/8ad56...
[5] https://www.mail-tester.com/
Cannot upvote you enough for the yahoo link.
You could build one for SES. You'd need to write an Amazon CloudFormation template that configured all the things correctly.
The result would be a screen or two of auditable gobbledygook that took a week or so to write. Somewhere in there, there'd be a pointer to an EC2 or ECS image, among other things.
It'd be a learning experience, and very tied to AWS. I wonder if someone else already did this.
The result would be a screen or two of auditable gobbledygook that took a week or so to write. Somewhere in there, there'd be a pointer to an EC2 or ECS image, among other things.
It'd be a learning experience, and very tied to AWS. I wonder if someone else already did this.
Mailinabox and Mailu are both popular.
I was wondering if anyone had any experience combining the microsoft 365 business basic (6$ a month) with self-hosted email server? By relaying SMTP through the Microsoft provided outlook server, would my custom domain be free from being marked by spam?
[1]: https://news.ycombinator.com/item?id=31180379