Unicorns, Startups and Hosted Email(blog.thinkst.com)
blog.thinkst.com
Unicorns, Startups and Hosted Email
http://blog.thinkst.com/2015/10/unicorns-startups-and-hosted-email.html
52 comments
E-mail software is not complex. It's all based on a simple line-based text protocol. The stack is also pretty simple to learn if you spend an hour on it.
The administrative headaches range from general user woes to networks that don't want your mail or have various ways of handling it, and you regularly have to address new and unusual problems. But this has nothing to do with the stack or protocol. This is because not all mail is made the same, not everyone does it right, and spam is really god damn annoying.
Don't run your own mail server unless you want to find out what makes BOFHs cry.
The administrative headaches range from general user woes to networks that don't want your mail or have various ways of handling it, and you regularly have to address new and unusual problems. But this has nothing to do with the stack or protocol. This is because not all mail is made the same, not everyone does it right, and spam is really god damn annoying.
Don't run your own mail server unless you want to find out what makes BOFHs cry.
At one point I "inherited" an email system that had ~120 different domains hundreds of servers worldwide and pretty much every email program known (including Lotus Notes) - migrating that to one email system while keeping the whole thing running was no fun at all - largely as nobody cares that much about email until it doesn't work then they get very upset and shouty.
The end result is, you need to invest significant amount of time and know-how into setting it up and then even more for maintenance if you want your e-mails to be ever read by anyone. Because spam is really god damn annoying.
You clearly never run your company's mail server. It needs regular attention, and 1 hour is not enough to jump in.
Have you had a look at sovereign (on github)? It's easy to set up and offers much more than just email.
I don't think this is that surprising. A lot of those firms would already have large-ish engineering teams already but managing the complexity of the mail servers is just not a good use of that resource.
Mail is a mostly solved technical problem, and having 'better' mail servers than the next guy isn't going to put food on the table any more.
Mail is a mostly solved technical problem, and having 'better' mail servers than the next guy isn't going to put food on the table any more.
Mail is a commodity now. It would be like trying to program your own switchboard or run your own phone lines,
> It would be like trying to program your own switchboard or run your own phone lines
You mean like a PBX? Most companies with >= 50 employees do this. Smaller/newer ones will definitely tend towards a managed service for this though.
You mean like a PBX? Most companies with >= 50 employees do this. Smaller/newer ones will definitely tend towards a managed service for this though.
What's that complexity that you are implying?
The comment there by Anonymous:
"Reminds me of this: "The Hostile Email Landscape," by Jody Ribton http://liminality.xyz/the-hostile-email-landscape/
In summary: Start sending email from a new mail server and the established players will likely mark it as spam. Market captured."
"Reminds me of this: "The Hostile Email Landscape," by Jody Ribton http://liminality.xyz/the-hostile-email-landscape/
In summary: Start sending email from a new mail server and the established players will likely mark it as spam. Market captured."
> Start sending email from a new mail server and the established players will likely mark it as spam. Market captured."
Oh because it most likely is?
Oh and I wonder why people use gmail, between having to use tools that are absolutely user unfriendly and a hosted solution, the option for 99% is clear.
(However I don't advocate using gmail/google apps)
Oh because it most likely is?
Oh and I wonder why people use gmail, between having to use tools that are absolutely user unfriendly and a hosted solution, the option for 99% is clear.
(However I don't advocate using gmail/google apps)
The article I've linked documents why it wasn't supposed to be a spam server. Specifically, it seems that the author had a financial interest to keep the server, not to push the spam and burn:
"Earlier this year I moved my personal email from Google Apps to a self-hosted server, with hopes of launching a paid mail service à la Fastmail on the same infrastructure. I've done this before, and this server was configured perfectly: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: https://www.mail-tester.com/ and Port25 are great for checking your setup.)
I had no issues sending to other servers running Postfix or Exim; SpamAssassin happily gave me a 0.0 score, but most big services and corporate mail servers were rejecting my mail, or flagging it as spam:
- Outlook.com accepted my email, but discarded it.
- GMail flagged me as spam.
- MimeCast put my mail into a perpetual greylist.
- Corporate networks using Microsoft's Online Exchange Protection bounced my mail."
"Earlier this year I moved my personal email from Google Apps to a self-hosted server, with hopes of launching a paid mail service à la Fastmail on the same infrastructure. I've done this before, and this server was configured perfectly: not on any blacklists, reverse DNS set up, SPF, DKIM and DMARC policies in place, etcetera. (Side note: https://www.mail-tester.com/ and Port25 are great for checking your setup.)
I had no issues sending to other servers running Postfix or Exim; SpamAssassin happily gave me a 0.0 score, but most big services and corporate mail servers were rejecting my mail, or flagging it as spam:
- Outlook.com accepted my email, but discarded it.
- GMail flagged me as spam.
- MimeCast put my mail into a perpetual greylist.
- Corporate networks using Microsoft's Online Exchange Protection bounced my mail."
True, I agree that this case is not spam
And the 'hints' are pretty strong it isn't
At the same time sometimes there are some gotchas that people might not be aware (and things that the big providers have to deal with and that to them are sign of spam - this should definitely be more publicized, however they want to protect this because of the actual spam senders)
Now, if their domain previously sent mail, then now it gets from a new domain this might make warnings go off
Politely contacting those email providers might help
One small anecdote: I've had Gmail flag (legitimate) mail from 'google.com' not only as Spam but as Phishing as well.
And the 'hints' are pretty strong it isn't
At the same time sometimes there are some gotchas that people might not be aware (and things that the big providers have to deal with and that to them are sign of spam - this should definitely be more publicized, however they want to protect this because of the actual spam senders)
Now, if their domain previously sent mail, then now it gets from a new domain this might make warnings go off
Politely contacting those email providers might help
One small anecdote: I've had Gmail flag (legitimate) mail from 'google.com' not only as Spam but as Phishing as well.
> Politely contacting those email providers might help
Really? Anybody who tried contacting Google and/or Outlook.com to claim some e-mail server is legit cares to comment? Is it possible? Is it different than what the article I've liked claims:
"The standard response from all of the above boiled down to this, from Microsoft's Postmaster Troubleshooting page:
- IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.
How to build a reputation for not sending spam when they're already flagging, bouncing, or deleting my mail was unclear."
Really? Anybody who tried contacting Google and/or Outlook.com to claim some e-mail server is legit cares to comment? Is it possible? Is it different than what the article I've liked claims:
"The standard response from all of the above boiled down to this, from Microsoft's Postmaster Troubleshooting page:
- IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, Outlook.com will typically allow for a better email delivery experience.
How to build a reputation for not sending spam when they're already flagging, bouncing, or deleting my mail was unclear."
> How to build a reputation for not sending spam when they're already flagging, bouncing, or deleting my mail was unclear."
- Have people click 'not spam' on the email in GMail/Outlook
- Have people send email from those domains to yours
- Have people click 'not spam' on the email in GMail/Outlook
- Have people send email from those domains to yours
Outlook.com discarded his e-mails completely. The corporate systems bounced it. You can't train the system that rejects the e-mails completely. There's nothing where "not spam" can be clicked.
Sounds like potentially, either:
1) The individual IP address(es) used by your server were used for spamming/phishing/bad things.
or
2) The larger address space owned/used by your hosting company is seen as sullied.
1) The individual IP address(es) used by your server were used for spamming/phishing/bad things.
or
2) The larger address space owned/used by your hosting company is seen as sullied.
The guy who wrote the article is in South Africa. It wouldn't be surprising that big parts of the world are grey or blacklisted for e-mail. But note that having an e-mail server in the US often negates the benefit of maintaining an own e-mail server for anybody not in the US.
I am often installing whole new email servers (author of https://poste.io here)... And thats not true, at least from my experience - well unless your first email is not "super promo whatever"...
Does the IP location of the server matter? The guy who wrote the article is in South Africa, if I understood correctly. Note that if the company is outside of the US and wants to keep control of their own e-mail, it would want to have the mail server locally, not in the US. Having the server in the US is otherwise not significantly different than having it in gmail.
Will be interesting times for EU startups when at the end of January 2016 model clauses and corporate bindings break down due to the 29 Working Group and EuGH decisions.
Could you please elaborate and provide links about what happens then? This sounds important, yet I have never heard about these things...
IANAL.
The EuGH court made it's decision that safe harbor is no longer considered lawful (triggered by a law suit by an Austrian against Facebook in Ireland). It basically says that EU data protection agencies can investigate companies for data protection issues even if the EU company uses an US company that is Safe Harbor certified.
This is due to the fact that the EuGH considers NSA snooping unlawful, especially that EU citizizens do not know about the spying and have no legal way in the US.
Beside Safe Harbor US companies provide data protection for EU companies based on EU model clauses. US enterprises share information about their employees back into the US based on 'corporate bindings'.
The 29 Working Group of EU data protection agencies issued their opinion last week on the EuGH decision
http://ec.europa.eu/justice/data-protection/article-29/press...
Interesting parts:
- Safe Harbor is unlawful for EU companies ("In any case, transfers that are still taking place under the Safe Harbour decision after the CJEU judgment are unlawful.")
- Model clauses and corporate bindings can only be used until the end of January 2016
"In the meantime, the Working Party will continue its analysis on the impact of the CJEU judgment on other transfer tools. During this period, data protection authorities consider that Standard Contractual Clauses and Binding Corporate Rules can still be used [...]
If by the end of January 2016, no appropriate solution is found with the US authorities and depending on the assessment of the transfer tools by the Working Party, EU data protection authorities are committed to take all necessary and appropriate actions, which may include coordinated enforcement actions."
Most unlikely those will be extended beyond January 2016 if one reads the opinion of different national agencies.
PS: Gmail is based on these legal frameworks.
The EuGH court made it's decision that safe harbor is no longer considered lawful (triggered by a law suit by an Austrian against Facebook in Ireland). It basically says that EU data protection agencies can investigate companies for data protection issues even if the EU company uses an US company that is Safe Harbor certified.
This is due to the fact that the EuGH considers NSA snooping unlawful, especially that EU citizizens do not know about the spying and have no legal way in the US.
Beside Safe Harbor US companies provide data protection for EU companies based on EU model clauses. US enterprises share information about their employees back into the US based on 'corporate bindings'.
The 29 Working Group of EU data protection agencies issued their opinion last week on the EuGH decision
http://ec.europa.eu/justice/data-protection/article-29/press...
Interesting parts:
- Safe Harbor is unlawful for EU companies ("In any case, transfers that are still taking place under the Safe Harbour decision after the CJEU judgment are unlawful.")
- Model clauses and corporate bindings can only be used until the end of January 2016
"In the meantime, the Working Party will continue its analysis on the impact of the CJEU judgment on other transfer tools. During this period, data protection authorities consider that Standard Contractual Clauses and Binding Corporate Rules can still be used [...]
If by the end of January 2016, no appropriate solution is found with the US authorities and depending on the assessment of the transfer tools by the Working Party, EU data protection authorities are committed to take all necessary and appropriate actions, which may include coordinated enforcement actions."
Most unlikely those will be extended beyond January 2016 if one reads the opinion of different national agencies.
PS: Gmail is based on these legal frameworks.
The people saying the software isn't that complex are missing the point. Everything's easy for the person who has never done it. Even if you include all of the DNS/SPF/DKIM garbage the initial setup can be done quickly. Then the support time sink begins. One person's outbound email bounced. Another person is getting too much spam. A third person is missing perfectly valid email because it keeps getting marked as spam. Third parties are complaining at you, or trying to hack you, or both. Oops, time to upgrade because of another TLS bug. You get the idea. Burning a couple of hours one time is no big deal. Burning half of someone's day, every day, is a problem.
Still, a lot of developers' addiction to gmail in particular continues to mystify me. I work on open source, so I don't care if people see what I put up on Google Drive for discussion, but there's no way I'd choose to put company-confidential email on a competitor's servers. Google competes with a lot of other companies. I guess infosec just isn't as important to some people as aesthetics.
Still, a lot of developers' addiction to gmail in particular continues to mystify me. I work on open source, so I don't care if people see what I put up on Google Drive for discussion, but there's no way I'd choose to put company-confidential email on a competitor's servers. Google competes with a lot of other companies. I guess infosec just isn't as important to some people as aesthetics.
The frustrating thing about this is that we're starting to see companies that make google accounts mandatory.
I've already been in the position of being told "make a google account or lose your job" once. I chose the job, I have rent to pay. But I really don't want a google account, and critically, I _do not_ agree with googles terms of use.
Unfortunately, there's no way for me to flag to google that I have made the account under duress, and thus I will be subject to their data fracking techniques against my will.
Facebook was talking a few years ago about developing Facebook for business, I can see that taking off like a rocket too.
I'd really like to see some federal level intervention on the topic of employers strong-arming employees into legally binding agreements with third parties.
I've already been in the position of being told "make a google account or lose your job" once. I chose the job, I have rent to pay. But I really don't want a google account, and critically, I _do not_ agree with googles terms of use.
Unfortunately, there's no way for me to flag to google that I have made the account under duress, and thus I will be subject to their data fracking techniques against my will.
Facebook was talking a few years ago about developing Facebook for business, I can see that taking off like a rocket too.
I'd really like to see some federal level intervention on the topic of employers strong-arming employees into legally binding agreements with third parties.
I don't see the problem here. You use third party products and services all the time in your professional capacity.
If your employer wants you to have a google account, i assume you'll use it for some job-related things. So it doesn't need any of your private data/usage profile (use company address/phone nr etc).
If your employer wants you to have a google account, i assume you'll use it for some job-related things. So it doesn't need any of your private data/usage profile (use company address/phone nr etc).
The difference is in who the legal agreement is between, from googles (and the courts) perspective.
For example, I do, in my professional capacity, use perforce. However, the agreement that perforce has is with my company. I'm just a number on the sheet of "how many users".
A gmail account is a privately held account. The agreement is between me and google. Any liability is mine, and google invests heavy effort into continuing to track my via a plethora of cookies, fingerprinting techniques and other methods even after I log out. They do that on the basis that I personally have agreed to submit to it, my employer isn't involved.
It's on the same level as if an employer mandated which phone company you could use. I don't mind my employer insisting on me using a specific provider, if he is providing the phone and paying for the plan. I might choose to leave that phone at work when I'm not on call though. But if it's my phone and I'm paying for the plan, I should have the right to choose who I give my support to.
The alternative road takes us down a dodgy path of corporate alliances, where companies agree to have each others employees scratch their backs, and to hell with consumer rights. We're already seeing it with companies in the US paying wages on pre-paid visa/AA/mastercard cards. Inevitably, the employee ends up paying all the fees. $5 to check your balance, $10 to transfer it elsewhere etc.
For example, I do, in my professional capacity, use perforce. However, the agreement that perforce has is with my company. I'm just a number on the sheet of "how many users".
A gmail account is a privately held account. The agreement is between me and google. Any liability is mine, and google invests heavy effort into continuing to track my via a plethora of cookies, fingerprinting techniques and other methods even after I log out. They do that on the basis that I personally have agreed to submit to it, my employer isn't involved.
It's on the same level as if an employer mandated which phone company you could use. I don't mind my employer insisting on me using a specific provider, if he is providing the phone and paying for the plan. I might choose to leave that phone at work when I'm not on call though. But if it's my phone and I'm paying for the plan, I should have the right to choose who I give my support to.
The alternative road takes us down a dodgy path of corporate alliances, where companies agree to have each others employees scratch their backs, and to hell with consumer rights. We're already seeing it with companies in the US paying wages on pre-paid visa/AA/mastercard cards. Inevitably, the employee ends up paying all the fees. $5 to check your balance, $10 to transfer it elsewhere etc.
In practice, there is pretty much zero chance you'd have any problems with Google. But I support your stance as a general rule - an employee should not force you to enter into personal commitments as a part of the job.
GMail or Google Apps?
Are you aware they have different terms of use?
> and thus I will be subject to their data fracking techniques against my will
To the data you present there, which is mostly your company data.
> I'd really like to see some federal level intervention on the topic of employers strong-arming employees into legally binding agreements with third parties.
Like when an employer requires a background check? (done via a website, for example?) Been there done that, didn't like it. I agree, there might be a need for regulation
But I'm much more bothered by the background check than a Google Apps account.
Are you aware they have different terms of use?
> and thus I will be subject to their data fracking techniques against my will
To the data you present there, which is mostly your company data.
> I'd really like to see some federal level intervention on the topic of employers strong-arming employees into legally binding agreements with third parties.
Like when an employer requires a background check? (done via a website, for example?) Been there done that, didn't like it. I agree, there might be a need for regulation
But I'm much more bothered by the background check than a Google Apps account.
Running a mail server isn't a black art. With projects like sovereign (on github), it's even easier.
Companies and people throw away their privacy so lightly - it shows a puzzling negligence.
The internet is at its strongest when it's decentralized. Stop using proprietary services when decentralized alternatives are readily available!
PS: I think Hillary Clinton did the right thing by running her own mail server, she shouldn't have used it for classified documents (if she did), however.
Companies and people throw away their privacy so lightly - it shows a puzzling negligence.
The internet is at its strongest when it's decentralized. Stop using proprietary services when decentralized alternatives are readily available!
PS: I think Hillary Clinton did the right thing by running her own mail server, she shouldn't have used it for classified documents (if she did), however.
Not surprising at all.
Hands down, Google Apps (not Gmail) is the best hosted email service for businesses. That's why companies use it. It's so good that wasting precious engineering resources on reinventing the wheel makes absolute no sense whatsoever. Oh, and don't forget email deliverability, spam detection, and a slew of other features that your cobblestone of postfix, dovecot, and roundcube will never even come close to reaching feature parity with.
Hands down, Google Apps (not Gmail) is the best hosted email service for businesses. That's why companies use it. It's so good that wasting precious engineering resources on reinventing the wheel makes absolute no sense whatsoever. Oh, and don't forget email deliverability, spam detection, and a slew of other features that your cobblestone of postfix, dovecot, and roundcube will never even come close to reaching feature parity with.
People focus on different priorities. I don't think it's hard to understand. They just care less about privacy. Scaling is more important to them. And your emails can be encrypted on your client side if you want to. Just make sure those encrypted emails aren't so suspicious that they want to crack them.
honest question, what's an "Unicorn"?
A startup with a $1B valuation.
To be a bit more specific; a privately owned startup valued at $1B or more.
I think I would prefer to stick with the traditional meaning --- it makes HN's headlines more entertainingly surreal.
> How to Build a Unicorn and Walk Away with Nothing
> GitHub's Unicorn Setup
> Tell HN: Unicorns aren't ad-supported
> In Search of Uber’s Unicorn
And nobody is going to tell me neigh.
> How to Build a Unicorn and Walk Away with Nothing
> GitHub's Unicorn Setup
> Tell HN: Unicorns aren't ad-supported
> In Search of Uber’s Unicorn
And nobody is going to tell me neigh.
Ok. I will invest 1$ in random startup for 1/1000000000 of the company - congratulations we have a new unicorn.
Reminds me of something Michael Lewis wrote:
“You have a dog, and I have a cat. … You sell me the dog for a billion, and I sell you the cat for a billion. Now we are no longer pet owners but Icelandic banks, with a billion dollars in new assets.”
“You have a dog, and I have a cat. … You sell me the dog for a billion, and I sell you the cat for a billion. Now we are no longer pet owners but Icelandic banks, with a billion dollars in new assets.”
Is there a canonical source for that definition, or at least showing that it's accepted slang in SV? I'm not saying I don't believe you; I'd only like to know the extent to which this is the accepted meaning.
It's a term that was introduced by Aileen Lee: http://techcrunch.com/2013/11/02/welcome-to-the-unicorn-club...
Thanks!
A legendary animal.
> So every time a twitter executive sends an email,
> people at Google can read it?
Legally I'm sure there's something in the TOS or contract that allows them to access the data if necessary.
Realistically, the number of people who can actually access customer data such as email and documents is probably quite small and heavily audited. Even accessing scrubbed telemetry data is probably audited.
Realistically, the number of people who can actually access customer data such as email and documents is probably quite small and heavily audited. Even accessing scrubbed telemetry data is probably audited.
>Realistically, the number of people who can actually access customer data such as email and documents is probably quite small and heavily audited.
http://www.theguardian.com/technology/2013/oct/30/google-rep...
Realistically, Google had no idea who had access to their customer data back then.
http://www.theguardian.com/technology/2013/oct/30/google-rep...
Realistically, Google had no idea who had access to their customer data back then.
You never know if there is backdoor and someone might be reading all the emails.
The US doesn't have a data protection law, nor a general privacy law. "Trade secret" protection and regular copyright are all you have.
Edit in response to downvote: OK, there's the https://en.wikipedia.org/wiki/Stored_Communications_Act . Whic prevents them from 'divulging' the contents of the email. However, it doesn't prevent automated processing (as used by gmail advertising). So you can see where this is going: is it illegal to ask the computer for a count of the uses of the word "merger" in emails belonging to users of a particular domain?
Edit in response to downvote: OK, there's the https://en.wikipedia.org/wiki/Stored_Communications_Act . Whic prevents them from 'divulging' the contents of the email. However, it doesn't prevent automated processing (as used by gmail advertising). So you can see where this is going: is it illegal to ask the computer for a count of the uses of the word "merger" in emails belonging to users of a particular domain?
As a non-US citizen, this shocks me, even if it shouldn't by now.
Does this indeed make it legal for a "chief corporate espionage officer" at Google to have a handy little dashboard of competitors inboxes? Can trade secrets only be defined by courts retrospectively?
Does this indeed make it legal for a "chief corporate espionage officer" at Google to have a handy little dashboard of competitors inboxes? Can trade secrets only be defined by courts retrospectively?
Maybe I am wrong regarding traditional email though, is there a simple and easy DIY email stack around these days?