The only safe email is text-only email(theconversation.com)
theconversation.com
The only safe email is text-only email
https://theconversation.com/the-only-safe-email-is-text-only-email-81434
120 comments
Second link points out a problem I was afraid of existing - big providers just stump on personal email servers.
>Hotmail is typically deleting all emails that I am sending. ... Monopolists like gmail.com won't accept any messages sent from my mail server.
>Hotmail is typically deleting all emails that I am sending. ... Monopolists like gmail.com won't accept any messages sent from my mail server.
I feel like I'm making this comment once a week on HN but I host my own email and I haven't had any major issue so far with "big email".
The main caveat is that you will have a very hard time getting your email accepted if it comes from a home connection IP range instead of some host provider but if you do have a dedicated server and follow the guidelines (SMTPS, DKIM, SPF etc...) it just works, at least in my experience.
The main caveat is that you will have a very hard time getting your email accepted if it comes from a home connection IP range instead of some host provider but if you do have a dedicated server and follow the guidelines (SMTPS, DKIM, SPF etc...) it just works, at least in my experience.
Hosting one's own e-mail server is a totally opaque random crapshot. You may
not have any trouble, but some other dude or gal will get their e-mail marked
as spam without any way to tell what exactly is wrong and what to change.
Maybe https://www.mail-tester.com/ would help. Also https://mxtoolbox.com/diagnostic.aspx .
If neither of these tools highlight any issues it's pretty strange indeed.
If neither of these tools highlight any issues it's pretty strange indeed.
First step when getting an ip for a server that will be a mail server is to check if the ip is not already blacklisted.
You can always get it unlisted.
After that don't start to send hundred of email by day. You need to build a reputation for your domain and ip.
As the parent comment says, set up directly spf, dkim and dmarc (also arc if you can). Rspamd can help you do that.
I've been running a personal mail server for 5 years with simply following those rules.
You can always get it unlisted.
After that don't start to send hundred of email by day. You need to build a reputation for your domain and ip.
As the parent comment says, set up directly spf, dkim and dmarc (also arc if you can). Rspamd can help you do that.
I've been running a personal mail server for 5 years with simply following those rules.
I've been running a personal mail server for twice as long with simply
following those rules and my e-mail is still tagged as spam in Gmail when
it's me who initiates contact (once the other party sends me an e-mail, reply
or otherwise, I no longer get tagged as spam).
As I said, it's totally opaque crapshot.
As I said, it's totally opaque crapshot.
I'm in a similar boat. I've moved ISPs a few times throughout the years, and each move it takes a few months to finally get things settled. I'm in the US, and had the best luck for deliverability when I was on a Comcast Business account.
My current ISP, a local fiber provider, was not great getting going. Most of the IPs that they have are in at least 1 spam database, and it took a while for the ISP to reach out to the database maintainers themselves. Even then, since they're a small ISP, the IPs are still blacklisted. The ISP wasn't even a company when the IPs were added to these blacklists.
After a few months they were able to assign me an IP that wasn't in a blacklist somewhere. I still randomly have issues with the big providers though - gmail is probably the most annoying. Like dozzie, my SPF, DKIM, and DMARC setups are all valid.
Overall, I really enjoying running my own mail server. Every now and then there are a few annoyances, but it's worth it in the long run.
My current ISP, a local fiber provider, was not great getting going. Most of the IPs that they have are in at least 1 spam database, and it took a while for the ISP to reach out to the database maintainers themselves. Even then, since they're a small ISP, the IPs are still blacklisted. The ISP wasn't even a company when the IPs were added to these blacklists.
After a few months they were able to assign me an IP that wasn't in a blacklist somewhere. I still randomly have issues with the big providers though - gmail is probably the most annoying. Like dozzie, my SPF, DKIM, and DMARC setups are all valid.
Overall, I really enjoying running my own mail server. Every now and then there are a few annoyances, but it's worth it in the long run.
There have been a few discussions regarding deliverability:
Why does Gmail hate my domain? | https://news.ycombinator.com/item?id=9855030
How to Avoid Spam Filters | https://news.ycombinator.com/item?id=10465639
Hotmail | https://news.ycombinator.com/item?id=14210939
ESP | https://news.ycombinator.com/item?id=14201704
If there are others I'd appreciate a link as I try to connect them!
Why does Gmail hate my domain? | https://news.ycombinator.com/item?id=9855030
How to Avoid Spam Filters | https://news.ycombinator.com/item?id=10465639
Hotmail | https://news.ycombinator.com/item?id=14210939
ESP | https://news.ycombinator.com/item?id=14201704
If there are others I'd appreciate a link as I try to connect them!
Are you using DKIM and SPF properly? I've never had any problems with this (although, I'm not mailing from home ISP ranges)
Yes.
That's very strange, then.
I'd recommend trying http://dkimvalidator.com/ to make sure everything is perfect; having proper certs on your MXs seems to help too.
I run a pretty large email infra (~500k/minute) and I'd help you out if I can...
If you're still having problems with all that in place then why don't you just change IPs?
I'd recommend trying http://dkimvalidator.com/ to make sure everything is perfect; having proper certs on your MXs seems to help too.
I run a pretty large email infra (~500k/minute) and I'd help you out if I can...
If you're still having problems with all that in place then why don't you just change IPs?
As I said, Gmail is opaque with regard to its spam detection.
As for your help offer, thank you, but I'm good. It's usually other people who want to contact me, I rarely send the first message ever.
As for your help offer, thank you, but I'm good. It's usually other people who want to contact me, I rarely send the first message ever.
Hmm well, I've found it to be pretty straightforward, I'm curious why this is happening to you though and there must be a solution...
I mean.. If you get a solid result from dkimvalidator but google is still shitlisting you then I'd definitely consider moving to another host/dc/isp at least.
Depending on your size, it might be best just to make this someone else's problem (if you can) -- like google, o365, etc..
I mean.. If you get a solid result from dkimvalidator but google is still shitlisting you then I'd definitely consider moving to another host/dc/isp at least.
Depending on your size, it might be best just to make this someone else's problem (if you can) -- like google, o365, etc..
The real problem is Google is opaque in dropping such mail in recipients spam folder. Hotmail can sometimes be annoying in this regard too. As far as I've managed neither does the sane thing and add a header that report on how they've divided that "this mail is spam" - so the recipient doesn't have anything to go on either, other than hoping "mark this as bot spam/add to address book/send email/reply to address" help Google/Ms treat it as "not spam".
Maybe it's worse for non-English mail (maybe the statistical models are biased against "not English" even for people whom don't have English as a native language).
But I'm quite convinced google's (and to a lesser extent Hotmail/ms') "magic" spamfiltering is subtly (but annoyingly) broken.
Maybe it's worse for non-English mail (maybe the statistical models are biased against "not English" even for people whom don't have English as a native language).
But I'm quite convinced google's (and to a lesser extent Hotmail/ms') "magic" spamfiltering is subtly (but annoyingly) broken.
I'm definitely not giving to somebody else my /var/log/mail.log, IMAP/Mutt
access, sieve rules, nor ad hoc dedicated aliases for every website account
I create. And on top of that, I would land on some US-based server for ease of
illegal spying and my mail would be harvested for some advertising crap, all
to solve something that is not a problem for me. Not happening.
[deleted]
> You can always get it unlisted.
How do you do that? The vast majority of blocklists I've interacted with have been unwilling to deal with questions, instead responding only that if you fix "something" (not always specified) automated measures will remove it from the list eventually.
In my experience it has also been extremely difficult to deal with people using blocklists. It's easy to find a bunch of people using .tor.dan.me.uk rather than .torexit.dan.me.uk "just to be safe". Frankly, I'm not sure why the former list exists in the first place other than to be an arse? What threats do entry/relay nodes pose to you?
How do you do that? The vast majority of blocklists I've interacted with have been unwilling to deal with questions, instead responding only that if you fix "something" (not always specified) automated measures will remove it from the list eventually.
In my experience it has also been extremely difficult to deal with people using blocklists. It's easy to find a bunch of people using .tor.dan.me.uk rather than .torexit.dan.me.uk "just to be safe". Frankly, I'm not sure why the former list exists in the first place other than to be an arse? What threats do entry/relay nodes pose to you?
I worked for two very large ESPs and never really had many problems with blacklists from a deliverability perspective.
Whenever one of our mailer IPs was blacklisted by one of the big targets (hotmail, gmail, etc) it would only be for 24hours after which I'd put it back into the pools (although, at least initially perhaps for our more reputable clients to warm it back up before letting the more dodgy stuff through it again). If you host your own NS's for the sender domains flipping SPF ips/ranges is not too hard (it's all automated for us, anyway).
The big boys work that way at least in my experience. I'm sure sometimes you'd hit a 'somedomain.foo' domain which is using a blocklist style thing that their (usually inexperienced) sysadmins think prevents them from receiving spam; but they're not worth arguing with. If you're doing email at volume shifting sends to another range for that client is usually 'enough' to get them through if they care that much about that one segment to ask you to do it.
If it's not then we'd usually refund instead of trying to negotiate with such admins.
Honestly though, blacklists have never been an issue for me and I've sent a ridiculous amount of email over the last 10 years...
Whenever one of our mailer IPs was blacklisted by one of the big targets (hotmail, gmail, etc) it would only be for 24hours after which I'd put it back into the pools (although, at least initially perhaps for our more reputable clients to warm it back up before letting the more dodgy stuff through it again). If you host your own NS's for the sender domains flipping SPF ips/ranges is not too hard (it's all automated for us, anyway).
The big boys work that way at least in my experience. I'm sure sometimes you'd hit a 'somedomain.foo' domain which is using a blocklist style thing that their (usually inexperienced) sysadmins think prevents them from receiving spam; but they're not worth arguing with. If you're doing email at volume shifting sends to another range for that client is usually 'enough' to get them through if they care that much about that one segment to ask you to do it.
If it's not then we'd usually refund instead of trying to negotiate with such admins.
Honestly though, blacklists have never been an issue for me and I've sent a ridiculous amount of email over the last 10 years...
If I ran an evil email monopoly, I would systematically drop all email from say _half_ of the email servers out there. This way, nobody would be able to make a case that we're abusing our monopoly position, but at the same time running an email server is hard and the internet is filled with wildly varying experience reports (which is the best FUD there is).
This way, fewer people start email servers and, therefore, fewer potential competitors grow up.
Obviously I have no way to tell whether Google really does this, but if they would, the result would look exactly like this HN thread.
I'm probably totally wrong here, but note that there is absolutely no incentive for the large email providers to try to fix this mess and make email the free distributed network it once was.
This way, fewer people start email servers and, therefore, fewer potential competitors grow up.
Obviously I have no way to tell whether Google really does this, but if they would, the result would look exactly like this HN thread.
I'm probably totally wrong here, but note that there is absolutely no incentive for the large email providers to try to fix this mess and make email the free distributed network it once was.
I've gotten DKIM, SPF, DMARC, etc all verified on my Digital Ocean droplet (and removed from various greylists) but still get spam checked by AT&T / SBC Global, who is the main offender of keeping a private list.
I've been removed for now from their lists... but they tend to re-add IPs from previous ranges for no reason.
I've been removed for now from their lists... but they tend to re-add IPs from previous ranges for no reason.
This is the most German thing I've seen since Octoberfest 2016.
It seems weird to me to have to find out the preferred way of communication to this level of detail, I already loathe that one cient of mine that only uses Facebook messenger for all communication.
It seems weird to me to have to find out the preferred way of communication to this level of detail, I already loathe that one cient of mine that only uses Facebook messenger for all communication.
Thunderbird has settings for each of your contacts for this reason. You can set if they want HTML or text only mails.
I would love to only use plain text e-mail. However, how does this work in practice when you are not a well known person (albeit in a niche) that decides the rules?
In work I have to accept whatever my colleagues, clients and bosses will send me. For personal communication I use mail with two people. And services that let you choose whether they'll send you plain text are practically inexistent.
In work I have to accept whatever my colleagues, clients and bosses will send me. For personal communication I use mail with two people. And services that let you choose whether they'll send you plain text are practically inexistent.
Many (most?) emails can be read in plain-text format even if they were written in HTML. The email often comes encoded with an "alternative" plain text version; of course you need a client that will show you that version instead. In my experience most personal email (i.e. not automated/form mails) I receive has a plain text alternative version that works fine.
For emails sent by an evil client that doesn't provide a plain text version, you can consider a tool that attempts to convert HTML to plain text (though I don't have a suggestion for this yet). Fall back on reading the HTML version only if the above fail.
- experience from working on / using my own mutt-like terminal email client.
For emails sent by an evil client that doesn't provide a plain text version, you can consider a tool that attempts to convert HTML to plain text (though I don't have a suggestion for this yet). Fall back on reading the HTML version only if the above fail.
- experience from working on / using my own mutt-like terminal email client.
> For emails sent by an evil client that doesn't provide a plain text version, you can consider a tool that attempts to convert HTML to plain text (though I don't have a suggestion for this yet).
I use emacs to read my email, and w3m to render HTML email as text. It does a good job. Newer emacs includes its own web browser (eww) but I have not tried it for HTML email rendering since I'm happy with w3m.
Replies always go out in plain text (my preference) but if you want to send HTML there are ways to e.g. write your email in org-mode markup and have it converted to HTML when you send. But IMO if you want to send "rich" email then just use a client that does that natively.
I use emacs to read my email, and w3m to render HTML email as text. It does a good job. Newer emacs includes its own web browser (eww) but I have not tried it for HTML email rendering since I'm happy with w3m.
Replies always go out in plain text (my preference) but if you want to send HTML there are ways to e.g. write your email in org-mode markup and have it converted to HTML when you send. But IMO if you want to send "rich" email then just use a client that does that natively.
Mutt + Lynx does it well, too
Similar to the other child comment: elinks can format HTML as plaintext.
I have this as my .mailcap and it Just Works (TM) with mutt:
I have this as my .mailcap and it Just Works (TM) with mutt:
text/html; elinks -dump %s; nametemplate=%s.html; copiousoutputYou can also use pandoc, I had better results with this:
> text/html; iconv -f %{charset} -t UTF-8 | pandoc -f html -t plain --wrap=preserve; copiousoutput; nametemplate=%s.html; description="HTML eMail";
> text/html; iconv -f %{charset} -t UTF-8 | pandoc -f html -t plain --wrap=preserve; copiousoutput; nametemplate=%s.html; description="HTML eMail";
I automatically delete anything with an attachment, that's so effective I didn't bother going any further with it.
I'm still waiting for people like that to automatically email back a CAPTCHA to the sender, if the sender is unknown, and not on a whitelist yet.
The proof of work approach is better.
How much work would you propose is sufficient that grandma doesn't mind but spammers will be severely hampered in their sending and in a manner that doesn't require the receiver to store too much state?
Only a small amount is necessary. There is a protocol called hashcash (used by Bitcoin but generally applicable) that can be easily used with email [1]. You basically get a header like this:
X-Hashcash: 0:030626:[email protected]:6470e06d773e05a8
You can choose how much work you want to do and the recepient can specify thresholds for minimum work required. This header is all they have to store, and it's easily stored through existing email infrastructure. Your mail server can do it without your client's help, too. If you spent a second on a proof I'm sure grandma wouldn't notice, but it would be very difficult for spammers to do the same.
[1] http://www.hashcash.org/
X-Hashcash: 0:030626:[email protected]:6470e06d773e05a8
You can choose how much work you want to do and the recepient can specify thresholds for minimum work required. This header is all they have to store, and it's easily stored through existing email infrastructure. Your mail server can do it without your client's help, too. If you spent a second on a proof I'm sure grandma wouldn't notice, but it would be very difficult for spammers to do the same.
[1] http://www.hashcash.org/
That does not sound terrible useful.
For one, it'll only work if both sender and receiver use it. Which means for 99.99% of mail traffic right now, it is utterly meaningless.
edit: Plus you also literally wasted everyone's time and energy.
I'd rather favor some kind of mechanism like grey-lists that don't require sender opt-in otherwise it'll be a dead technology just like GPG. (You can probably count the number of GPG emails within 1000 average mails on one hand)
For one, it'll only work if both sender and receiver use it. Which means for 99.99% of mail traffic right now, it is utterly meaningless.
edit: Plus you also literally wasted everyone's time and energy.
I'd rather favor some kind of mechanism like grey-lists that don't require sender opt-in otherwise it'll be a dead technology just like GPG. (You can probably count the number of GPG emails within 1000 average mails on one hand)
How about sending (by email) a link to a proof-of-work-website, so the hash function can be computed in javascript/wasm in a browser?
Anyway, I agree on the waste of time+energy.
Anyway, I agree on the waste of time+energy.
We make incremental change. Adoption starts slow, then some middle players pick it up, one big player grabs it and then it proliferates. I've been working on a mail client for some time and I have plans to write a mail server, and both will have first class, opt-out support for hashcash.
That sounds like a lovely pipe-dream.
I severely doubt any big player picks it up on grounds of it being cheaper and less energy intensive to simply greylist or blacklist.
I severely doubt any big player picks it up on grounds of it being cheaper and less energy intensive to simply greylist or blacklist.
Thanks. I love the examples.
Every time I use some kind of graphical/web email client, I'm appalled by how hard it is for me to find the "show me all the headers" feature. In mutt when I have doubts about something I just hit "h".
Admittedly this requires a level of knowledge that the average user may be missing. But I find it really helps inform my opinion of any borderline-suspicious emails.
Admittedly this requires a level of knowledge that the average user may be missing. But I find it really helps inform my opinion of any borderline-suspicious emails.
I've always thought native handling of markdown for email would be pretty cool.
The MUA could strip out any HTML embedded in the actual markdown, render to HTML locally (for display) and you have nice formatting without the risks or cruft of email HTML.
And of course if you choose to view in text only mode, you don't actually lose any semantic meaning.
The MUA could strip out any HTML embedded in the actual markdown, render to HTML locally (for display) and you have nice formatting without the risks or cruft of email HTML.
And of course if you choose to view in text only mode, you don't actually lose any semantic meaning.
Huh, looks like text/markdown email is coming slowly:
https://stackoverflow.com/a/25812177
https://tools.ietf.org/html/rfc7763
https://tools.ietf.org/html/rfc7764
(March 2016)
Previous HN discussion: https://news.ycombinator.com/item?id=13176743
https://stackoverflow.com/a/25812177
https://tools.ietf.org/html/rfc7763
https://tools.ietf.org/html/rfc7764
(March 2016)
Previous HN discussion: https://news.ycombinator.com/item?id=13176743
The RFC's are just about defining the Media Type 'text/markdown' aren't they?
Just because they used to be called MIME types and were designed for email, I wouldn't treat those RFC's as being about email support specifically.
Just because they used to be called MIME types and were designed for email, I wouldn't treat those RFC's as being about email support specifically.
Sure, it could be used elsewhere. OTOH 7763 calls out email as a use case three times and in section 5 Examples "email attachment" is the only example.
In plain text, Thunderbird recognises * and / as bold and italic, which sometimes produces interesting results for badly-formatted multi-part emails.
I switched to using mutt with gmail and have never looked back. Mutt actually does a brilliant job of converting all html to plaintext.
Here's the entry that I wrote about it (includes my mutt config): https://smalldata.tech/blog/2016/09/10/gmail-with-mutt
Here's the entry that I wrote about it (includes my mutt config): https://smalldata.tech/blog/2016/09/10/gmail-with-mutt
This might be true, but I think that ship has sailed. Email for 99% of internet users is html. Thinking that some large fraction of news letters, outlook emails will ever be plaintext is just naive. I use html emails in outlook simply because I don't want my emails within the corporation to appear differnet from anyone elses. I certainly don't want to return something that looks different from what the sender wrote,.
The mail client is a web browser. In many cases it's an actual web app in an actual web browser. The solution has to be to make them safer for the user. Maybe a subset of html can be whitelisted, maybe link targets should be rendered more clearly, maybe something else.
The mail client is a web browser. In many cases it's an actual web app in an actual web browser. The solution has to be to make them safer for the user. Maybe a subset of html can be whitelisted, maybe link targets should be rendered more clearly, maybe something else.
Over 99.5% of the email I receive is either plaintext or renders perfectly legible in plain. This suggests that a large fraction of newsletters are already plaintext. The mail client is primarily a program to display and send email. The web browser is a web browser.
If the user wants safety, switching to plaintext is a very wide step forward.
If the user wants safety, switching to plaintext is a very wide step forward.
> Over 99.5% of the email I receive is either plaintext or renders perfectly legible in plain.
Is that representative? Where do you get email from? Did you change preferences on things like mailing lists in order to get to that percentage?
Also: what kind of client do you use? iOS mail does send plain text emails as text/plain (which is fantastic) but if you look at e.g. inbox (gmail) it doesn't even allow sending plain text emails as far as I'm aware.
Is that representative? Where do you get email from? Did you change preferences on things like mailing lists in order to get to that percentage?
Also: what kind of client do you use? iOS mail does send plain text emails as text/plain (which is fantastic) but if you look at e.g. inbox (gmail) it doesn't even allow sending plain text emails as far as I'm aware.
Emails can send in both formats for a single message, so it's possible and even likely that most mailing lists, etc. he receives send in both formats. In my experience, even most marketing e-mails are at least somewhat good about this. I'm 99% sure that Gmail will still send a plaintext e-mail inferred from your HTML content whenever you send, so it's not quite accurate to say it doesn't send in plaintext.
...and if you're PayPal or eBay, you offer the customer a choice of receiving plain text emails. Then you send them multipart emails with a blank text/plain part.
Not that this surprises me in the least :-)
Not that this surprises me in the least :-)
> I'm 99% sure that Gmail will still send a plaintext e-mail inferred from your HTML content whenever you send, so it's not quite accurate to say it doesn't send in plaintext.
I think that might be possible in the "old" gmail, I was referring to their newer "Inbox by gmail" client. I believe it never sends a text/plain email regardless of content (And it being their "newer" revamped client kind of shows what googles take on this is...)
I think that might be possible in the "old" gmail, I was referring to their newer "Inbox by gmail" client. I believe it never sends a text/plain email regardless of content (And it being their "newer" revamped client kind of shows what googles take on this is...)
I run two email addresses: one business, one personal. As you may imagine, the emails I get between the two are quite different. I'm using Thunderbird and always, when given the choice, opt for plain text email. I find Thunderbird does a very good job of rendering HTML email legible (easy to read) and functional (links work, etc).
Perhaps once a month, maybe less, I have to use the toolbar button "Show HTML Temp" to do a one-off rendering of an email as basic HTML. Almost always the same offenders.
I find it such a success that I recommend it to my customers - none of which are computer experts, and most are not what I'd call "computer savvy". A process has to meet a high bar for me to recommend something like this to my customers.
Shame that when I explain how it can help in improving one's security that a vanishingly-small number of customers take me up on the offer!
Perhaps once a month, maybe less, I have to use the toolbar button "Show HTML Temp" to do a one-off rendering of an email as basic HTML. Almost always the same offenders.
I find it such a success that I recommend it to my customers - none of which are computer experts, and most are not what I'd call "computer savvy". A process has to meet a high bar for me to recommend something like this to my customers.
Shame that when I explain how it can help in improving one's security that a vanishingly-small number of customers take me up on the offer!
I think the first reason people don't care for installing e.g. Thunderbird is that they check mail on a) mobile, b) work (which is usually outlook, or at least mandated by someone else).
If I asked 100 friends "did you use email today" 99 would say yes. If I said "did you do it on a computer" then 50 would say yes (those that work at an office). If I said "was it your own computer" then maybe 1 would say yes.
I think for us that sit at our own computers regularly, it's hard to imagine just how extremely rare it is for people to actually use desktop/laptops for personal use these days.
If I asked 100 friends "did you use email today" 99 would say yes. If I said "did you do it on a computer" then 50 would say yes (those that work at an office). If I said "was it your own computer" then maybe 1 would say yes.
I think for us that sit at our own computers regularly, it's hard to imagine just how extremely rare it is for people to actually use desktop/laptops for personal use these days.
I'm not sure the ship has sailed. If HSBC switched to only mailing out text-only emails with URLs written out in full, after a while HSBC users would get used to only receiving text correspondence from their bank. I think that would be a step towards reducing phishing attempts, though certainly not a complete answer.
Disclosure: I work for a fintech company that utilizes HTML emails almost exclusively for customer communications.
I think one thing that is not being considered is that for most customers, branding and the consistency thereof are key indicators of trustworthiness - especially when dealing with financial information. HN users are rare creatures, they have technical context the average end user does not have. The rise of phishing has lead users to pay a great amount of attention to subtle hints of impropriety, like being taken from one sort of visual experience to a vastly different one. We saw vast improvement across all meaningful metrics when we switched from plain text to HTML emails that utilized branding consistent with our website.
As with everything that humans deal with, there are tradeoffs here. And I'm extremely concerned that this position taken to it's logical extreme would lead to the web being transformed into something that is "safer" but much less useful and dynamic. One outcome of this could be the slow death of the open web in favor of siloed networks and platforms serving actually functional content in "safe" ways.
I think one thing that is not being considered is that for most customers, branding and the consistency thereof are key indicators of trustworthiness - especially when dealing with financial information. HN users are rare creatures, they have technical context the average end user does not have. The rise of phishing has lead users to pay a great amount of attention to subtle hints of impropriety, like being taken from one sort of visual experience to a vastly different one. We saw vast improvement across all meaningful metrics when we switched from plain text to HTML emails that utilized branding consistent with our website.
As with everything that humans deal with, there are tradeoffs here. And I'm extremely concerned that this position taken to it's logical extreme would lead to the web being transformed into something that is "safer" but much less useful and dynamic. One outcome of this could be the slow death of the open web in favor of siloed networks and platforms serving actually functional content in "safe" ways.
That would require HSBC to value some kind of improved security so much that they'd accept not having the HSBC logo in the email. That's what I think is out of the question.
You could maybe see banks having plaintext communication as an optional, but I doubt they'd make it default (allowing users to switch to html).
Isn't this problem already solved with certificates online? Shouldn't this be solvable the same way? E.g. a bank sends an email containing a link to the content with some special attribute. The web browser displays the content if and only if the sender domain of the email (e.g. hsbc.com) is also the domain from which the content will be downloaded.
You could maybe see banks having plaintext communication as an optional, but I doubt they'd make it default (allowing users to switch to html).
Isn't this problem already solved with certificates online? Shouldn't this be solvable the same way? E.g. a bank sends an email containing a link to the content with some special attribute. The web browser displays the content if and only if the sender domain of the email (e.g. hsbc.com) is also the domain from which the content will be downloaded.
Problem is you've already received the rogue html before accessing the secure webpage. It's a shame email signing and encryption never took off.
The solution assumed mail clients would be adapted to enforce this. So if you send me a forged email claiming to be from hsbc, the mail client would allow showing html content only from a https connection to somewhere on hsbc.
Kind of like the same-origin policy but where the origin is the domain the email claims it came from.
They could still have the HSBC logo, it would just need to be in ascii....
If you like this approach and are using Mail.app on OSX then I recommend the following:
First...
1. In the menu, go: Mail > Preferences > Viewing.
2. Uncheck "Load remote content in messages".
3. Move to the "Composing" tab.
4. Select "Plain Text" for Message Format.
5. Uncheck "Use the same message format as the original message".
Second...
In Terminal, copy/paste this command to set plain-text preference to true:
First...
1. In the menu, go: Mail > Preferences > Viewing.
2. Uncheck "Load remote content in messages".
3. Move to the "Composing" tab.
4. Select "Plain Text" for Message Format.
5. Uncheck "Use the same message format as the original message".
Second...
In Terminal, copy/paste this command to set plain-text preference to true:
defaults write com.apple.mail PreferPlainText -bool true
I use plaintext mail heavily. My only complaint is clients that don't support format=flowed[1] wrap text weirdly and make it look bad. The biggest ones are Outlook and the Windows 10 Mail app, though some webmail handles it poorly too. It gets bad when conversations start accumulating nested quotes.
I've found a quick tell if someone uses Outlook is if I send them a text/plain message, they'll send one back and there's no format=flowed. At that point I'll usually send them HTML mail.
1. https://joeclark.org/ffaq.html
I've found a quick tell if someone uses Outlook is if I send them a text/plain message, they'll send one back and there's no format=flowed. At that point I'll usually send them HTML mail.
1. https://joeclark.org/ffaq.html
I think this is entirely dependent on the context of that particular E-mail. Email has evolved quite a lot.
A good example is a newsletter from say, Quora or Medium; Newsletters usually have links to a story or a news feed article. If this was done using plain text, the link would be one long mashup of characters, because they usually include an authentication token or something like that. In this case, using an html link or button is clearly the better option.
HTML is like structured text for web pages.
A good example is a newsletter from say, Quora or Medium; Newsletters usually have links to a story or a news feed article. If this was done using plain text, the link would be one long mashup of characters, because they usually include an authentication token or something like that. In this case, using an html link or button is clearly the better option.
HTML is like structured text for web pages.
> A good example is a newsletter from say, Quora or Medium; Newsletters usually have links to a story or a news feed article. If this was done using plain text, the link would be one long mashup of characters, because they usually include an authentication token or something like that. In this case, using an html link or button is clearly the better option.
Well, that's a yet more hostile thing that one has to put up with. I follow news through newsletters, and I'd prefer they were in plain text and with normal links. My reader knows how to wrap lines and make link-like things in plain text clickable. But unfortunately if I wanted to enforce that I'd have to avoid news...
Well, that's a yet more hostile thing that one has to put up with. I follow news through newsletters, and I'd prefer they were in plain text and with normal links. My reader knows how to wrap lines and make link-like things in plain text clickable. But unfortunately if I wanted to enforce that I'd have to avoid news...
Yah,consequently. But last week someone posted a link here, a text only version of CNN, I looked it up, here it is; http://lite.cnn.io/en. We could have more of these soon, which is a good thing.
Some client software hides the From and Reply-To addresses, only showing the name by default. A friend's accountant got hit because the From had my friend's name, but the address itself was bogus but hidden, so he opened the attachment.
So keep the main headers text-only as well (which most sane software does anyway).
So keep the main headers text-only as well (which most sane software does anyway).
I never click on any link in an email I didn't ask for. If I get a notification email from a party I have business with, I go to their website manually and check for messages.
E-Mail should've been replaced by something more suitable more than a decade ago.
I sometimes wonder how long it's going to stay. My guess is: until the end of the internet.
I sometimes wonder how long it's going to stay. My guess is: until the end of the internet.
and to get there, we need email clients with usable UIs. Is there a way to make outlook or apple mail or the gmail web app grab the equivalent of `document.innerText` on every piece of email? Who can tell, without spending hours hunting through increasingly obscure menu forests?
I never stopped using pine. Who knew that I was actually ahead of the game?
wouldn't it be easier to have your email reader only render it in plain text?
Reminds me of the ascii ribbon campaign against non-human readable formats in email.
I switched to a text only email client a few months back. I really don't think I am missing anything. HTML content tends to be chaff/advertising.
I switched to a text only email client a few months back. I really don't think I am missing anything. HTML content tends to be chaff/advertising.
I recently switched to mutt and using plain-text email as well. So far there have been very few times that I've felt the need to jump back to the webmail to send something (e.g. inline images). Overall, I've thoroughly enjoyed the simplicity of plain text email using my text editor of choice.
In my experience, for personal email, text-only is a non-issue; for work stuff, it's not even an option.
Can you elaborate why?
I sometimes abuse this option by including screenshots in the body of e-mail, but I could as well add it as an attachment. Other than that I see no reason to use HTML in e-mail conversations at work.
Intra-corporate e-mail is often used as an ad hoc document collaboration tool.
"See my comments in blue", strikethroughs, inline diagrams, bullet points, big red font for emphasis.
Of course that should all be done in a dedicated application, but who is going to provision and authorise users for that versus just adding Bob to the cc line and giving him implicit editing capabilities?
The functional overloading of corporate e-mail is a user-driven reaction to the awfulness of most " collaborative' software.
"See my comments in blue", strikethroughs, inline diagrams, bullet points, big red font for emphasis.
Of course that should all be done in a dedicated application, but who is going to provision and authorise users for that versus just adding Bob to the cc line and giving him implicit editing capabilities?
The functional overloading of corporate e-mail is a user-driven reaction to the awfulness of most " collaborative' software.
I've always had my work email client set to text only. Never been a problem for me (unless you consider missing out on a lot of unnecessary smilies, fonts, and colors a problem).
I too am confused. The only time I've used HTML in email was when I used the web interface and plain text wasn't an option.
[deleted]
I was expecting to read about some brilliant new way to solve all of the normalisation problems with Unicode, and make plain text safe again. What a let down!
Annoyingly, the Gmail Android app sends things multipart, even though I'm not sure it's really possible to have any formatting (except maybe links?). I wish there wan option to send text only, it might even make a (small) difference for people with limited data.
There's a certain zen to going back to basics and using plaintext. It's always my default choice whenever I'm given the option.
I'd argue in most cases you really don't need any fancy styles and markup. Although upon writing this I'm now wondering if unstyled HTML might provide improved accessibility over plaintext. What are people's experiences on the matter?
Although I respect that some people may find greater value in carefully styled marketing emails, to me it just feels a bit overengineered at times. How many man-hours have been spent trying to get marketing emails to look the same across all major clients? Are slight variations really such a horrible thing?
As we're on the subject, I'll add the following suggestion to gmail users: go to Settings and changing the Images option to "Ask before displaying external images". External images are regularly abused to track if the email has been viewed, which I consider creepy.
I'd argue in most cases you really don't need any fancy styles and markup. Although upon writing this I'm now wondering if unstyled HTML might provide improved accessibility over plaintext. What are people's experiences on the matter?
Although I respect that some people may find greater value in carefully styled marketing emails, to me it just feels a bit overengineered at times. How many man-hours have been spent trying to get marketing emails to look the same across all major clients? Are slight variations really such a horrible thing?
As we're on the subject, I'll add the following suggestion to gmail users: go to Settings and changing the Images option to "Ask before displaying external images". External images are regularly abused to track if the email has been viewed, which I consider creepy.
If nothing else, styling emails serves an economic purpose through facilitating signaling. The more a company expects that consumers will value it in the long run the more incentive it has to signal that fact, and a signal is only as good as it is expensive in appearance to the people being signaled to.
In this case, things like a marketing email or simply the follow-up email to signing up to some service provide that signal since the user can see how much effort was invested in making the communication look attractive.
In this case, things like a marketing email or simply the follow-up email to signing up to some service provide that signal since the user can see how much effort was invested in making the communication look attractive.
As a recipient of those emails -- not really. Clients block remote content by default and there's almost never a reason to load them, especially when they're used for tracking. Those 'pretty' emails will almost always look like a soup of image placeholders interspersed with text.
> External images are regularly abused to track if the email has been viewed, which I consider creepy.
My understanding was that external images are automatically fetched and cached on their servers by Google, so they can't be reliably used to track message views [1]. Has this changed?
[1] https://gmail.googleblog.com/2013/12/images-now-showing.html
My understanding was that external images are automatically fetched and cached on their servers by Google, so they can't be reliably used to track message views [1]. Has this changed?
[1] https://gmail.googleblog.com/2013/12/images-now-showing.html
AFAIK tracking images are usually sent via a recipient-unique URL, which allows services like Mailchimp to provide open stats for individuals. Since the URLs are unique, the host servers will get hit at least once per recipient that opens the email, as Google downloads and caches the image.
That depends on how the fetching and caching is implemented. If it's done when you open your email, the tracking would still work. If the trigger is unrelated to anything correlating with your opening of the email it wouldn't.
It will still prevent the sender from setting cookies for you, or learning your IP and user agent.
It will still prevent the sender from setting cookies for you, or learning your IP and user agent.
They are proxied by Google's servers, but not (necessarily) cached.
I am in the process of learning everything-marketing and sending my own emails from my own server using my own hosted software.
I can confirm that every time you open email, open is get tracked. (what is not get tracked is your IP/OS/etc)
So in fact Gmail made online marketers job easier.
I can confirm that every time you open email, open is get tracked. (what is not get tracked is your IP/OS/etc)
So in fact Gmail made online marketers job easier.
> Although upon writing this I'm now wondering if unstyled HTML might provide improved accessibility over plaintext.
Yep. Email formatted for 65 to 78 monospaced characters with hard carriage returns (as per *nix mailing lists) look much worse on devices that are narrower or wider than this. HTML paragraphs don't have the problem.
Yep. Email formatted for 65 to 78 monospaced characters with hard carriage returns (as per *nix mailing lists) look much worse on devices that are narrower or wider than this. HTML paragraphs don't have the problem.
I think this is the problem format=flowed was created to solve. But it doesn’t seem to help. I wonder why it doesn’t have wider implementation
This is silly. The authors establish that phishing is a serious problem (duh), and that this problem is caused by the absence of reliable authentication of messages (a worthwhile observation, albeit one that the industry is already aware of and doing its best to patch over), but they fail to establish that text-only email solves this problem in any meaningful way.
Text-only emails can and will still contain links, which users will still click on. Misleading domain names will work just as well in the email body as they do in the address bar. Even if (as this article seems to imply should be done) mail clients don't make the links clickable, users will still copy-paste them. (Not to mention that the usability benefits of making links clickable are significant enough that mail clients won't forgo them just for a speculative hypothetical security benefit.)
The authors seem to think that inserting a "speed bump" here will cause users to pay closer attention and not be fooled. This is not how humans work, especially very busy humans who get too much email and just want to get through it as quickly as possible.
Also, the reference to JavaScript in email leads me to question whether the authors have any idea what they're talking about. Mail clients don't execute JavaScript.
Text-only emails can and will still contain links, which users will still click on. Misleading domain names will work just as well in the email body as they do in the address bar. Even if (as this article seems to imply should be done) mail clients don't make the links clickable, users will still copy-paste them. (Not to mention that the usability benefits of making links clickable are significant enough that mail clients won't forgo them just for a speculative hypothetical security benefit.)
The authors seem to think that inserting a "speed bump" here will cause users to pay closer attention and not be fooled. This is not how humans work, especially very busy humans who get too much email and just want to get through it as quickly as possible.
Also, the reference to JavaScript in email leads me to question whether the authors have any idea what they're talking about. Mail clients don't execute JavaScript.
> but they fail to establish that text-only email solves this problem in any meaningful way
The quote from US-CERT isn't "meaningful"?
FWIW, I'm the first to bitch about security experts that sacrifice usability in the name of security any day, but for once I completely agree with them.
> Also, the reference to JavaScript in email leads me to question whether the authors have any idea what they're talking about. Mail clients don't execute JavaScript.
https://stackoverflow.com/questions/3054315/is-javascript-su...
And that's only until someone finds a way to make them execute Javascript anyway. I don't think it ever actually happened, but not using an HTML engine drastically reduces the attack surface for sure.
The quote from US-CERT isn't "meaningful"?
FWIW, I'm the first to bitch about security experts that sacrifice usability in the name of security any day, but for once I completely agree with them.
> Also, the reference to JavaScript in email leads me to question whether the authors have any idea what they're talking about. Mail clients don't execute JavaScript.
https://stackoverflow.com/questions/3054315/is-javascript-su...
And that's only until someone finds a way to make them execute Javascript anyway. I don't think it ever actually happened, but not using an HTML engine drastically reduces the attack surface for sure.
Text-only emails can and will still contain links, which users will still click on. Misleading domain names will work just as well in the email body as they do in the address bar.
...that brings up the other elephant in the room: Unicode. More specifically, https://en.wikipedia.org/wiki/IDN_homograph_attack
The 0/O/i/1/l/I distinction is a classic one that can be somewhat mitigated by good font choice (plaintext emails are often presented in monospace, so that helps a little), but some of the other sets of Unicode characters are designed to look identical to ones in the ASCII range, often using the exact same glyphs.
For those who use non-English characters exclusively, it is not possible to use ASCII only emails, but non-Unicode encodings are still helpful in this situation --- e.g. the Cyrillic characters most known for IDN homograph attacks aren't even encodable in 8859-1 or Shift-JIS.
...that brings up the other elephant in the room: Unicode. More specifically, https://en.wikipedia.org/wiki/IDN_homograph_attack
The 0/O/i/1/l/I distinction is a classic one that can be somewhat mitigated by good font choice (plaintext emails are often presented in monospace, so that helps a little), but some of the other sets of Unicode characters are designed to look identical to ones in the ASCII range, often using the exact same glyphs.
For those who use non-English characters exclusively, it is not possible to use ASCII only emails, but non-Unicode encodings are still helpful in this situation --- e.g. the Cyrillic characters most known for IDN homograph attacks aren't even encodable in 8859-1 or Shift-JIS.
At least with plain-text, it's a slightly harder to forge links. (i.e. you have to do http://www.megabank.com.phishingattempt.io instead of <a href="http://www.phishingattempt.io"><img src="megabank.com/logo.png"></a> )
Speaking of JavaScript in e-mail, gmail doesn't allow you to send or receive .js files (even tarred up), which is somewhat inconvenient, and I'm not really sure what attack that prevents. Maybe there is a mail client out there that will happily execute attached js?
Speaking of JavaScript in e-mail, gmail doesn't allow you to send or receive .js files (even tarred up), which is somewhat inconvenient, and I'm not really sure what attack that prevents. Maybe there is a mail client out there that will happily execute attached js?
I got a phising SMS today (please login to verify unusual account activity) with a link to mybank.online-eauth.co.uk. I knew from before even seeing that it was probably not legit, but I did check the domain whois as UK banks are rather arcane, and I wouldn't be surprised if they had a completely different domain for something like this. I know when I buy something online, my bank has a different domain for Verified by Visa.
Also, I gather that non-text emails make it possible to disguise the link target when you hover over it to see where the link goes. Whether that is using css or js, I'm not sure.
On mobile, there's no hover, so HTML mail means I just have to guess if I think the link will go where it says or not.
In iOS Mail, you don't have to guess where the link goes, press and hold on a link will show the URL.
Considering how much email is checked on mobile I am surprised your point is not being mentioned more frequently.
You can't falsify the browser indicators of link targets (status bar on desktop, modal dialog from holding down the link on mobile) without JavaScript.
You can create a misleading tooltip in HTML: <a href="https://www.megabank.com.phishingattempt.io" title="https://www.megabank.com">https://www.megabank.com</a>. But since modern browsers don't use tooltips to indicate link targets, users probably won't be looking there in the first place.
You can create a misleading tooltip in HTML: <a href="https://www.megabank.com.phishingattempt.io" title="https://www.megabank.com">https://www.megabank.com</a>. But since modern browsers don't use tooltips to indicate link targets, users probably won't be looking there in the first place.
The real URL will appear in the browser address bar anyway before the user gets the chance to disclose any information. I don't know exactly what proportion of users will notice a well-disguised phishing URL in the email body but not in the address bar, but I bet it's not that high.
The attack prevented is simply having the user open the attachment, allowing the sender to execute arbitrary JavaScript on their machine in the file:// context. (Modern browsers have made the security consequences of this somewhat less dire than they once were, but it's still not something you want to do if you can help it.)
The attack prevented is simply having the user open the attachment, allowing the sender to execute arbitrary JavaScript on their machine in the file:// context. (Modern browsers have made the security consequences of this somewhat less dire than they once were, but it's still not something you want to do if you can help it.)
That involves the rather large and utterly baseless assumption that users look at the address bar at all. You probably do. Does your somewhat less-savvy next-door neighbour?
This observation I made a while ago shows that a considerable number of "average users" used to, and even understood how URLs are formed:
https://news.ycombinator.com/item?id=7678729
If an increasing number of users aren't, then that is certainly a problem.
https://news.ycombinator.com/item?id=7678729
If an increasing number of users aren't, then that is certainly a problem.
They don't really look at the URL in the email body either, especially if it's long and intimidatingly technical-looking.
> The real URL will appear in the browser address bar anyway before the user gets the chance to disclose any information
Which is already too late for anyone compromised by a drive-by download attack.
Which is already too late for anyone compromised by a drive-by download attack.
The threat model here is phishing, not drive-by downloads. Browsers have a much greater ability to mitigate those. Also, a drive-by download email doesn't have to impersonate any particular sender, it just has to look like something that a user might want to click on.
> The threat model here is phishing, not drive-by downloads.
I think you missed my point. What if it isn't a phishing attack? Or even, what if it isn't just a phishing attack?
Your suggestion leaves users vulnerable by encouraging them to open suspicious looking links on the off chance it is, at most, a phishing attack.
> Browsers have a much greater ability to mitigate those.
Except when they don't. For what it's worth, I've also seen mobiles fall victim to drive-by download attacks.
> Also, a drive-by download email doesn't have to impersonate any particular sender, it just has to look like something that a user might want to click on.
E-mail worms are spread by the trust relationship between people known to each other (ie a user opening an attachment because it's from a recipient they know). I don't see why drive-by download attacks couldn't exploit the same human condition (ie "Hey bob, check out this link. It's awesome").
In fact I have seen that kind of malware in the wild, now I think about it.
I think you missed my point. What if it isn't a phishing attack? Or even, what if it isn't just a phishing attack?
Your suggestion leaves users vulnerable by encouraging them to open suspicious looking links on the off chance it is, at most, a phishing attack.
> Browsers have a much greater ability to mitigate those.
Except when they don't. For what it's worth, I've also seen mobiles fall victim to drive-by download attacks.
> Also, a drive-by download email doesn't have to impersonate any particular sender, it just has to look like something that a user might want to click on.
E-mail worms are spread by the trust relationship between people known to each other (ie a user opening an attachment because it's from a recipient they know). I don't see why drive-by download attacks couldn't exploit the same human condition (ie "Hey bob, check out this link. It's awesome").
In fact I have seen that kind of malware in the wild, now I think about it.
I don't know what you mean by "encouraging". I am taking it as a given that users cannot be reliably prevented from clicking on links in emails. The security benefit of forcing the URL to be displayed before the click is extremely minimal.
A small, but not insignificant, proportion of malware and phishing emails I receive do not show any links at all in their plaintext part. Some of the links are probably easy to skim over and miss the fact they're suspect, but some really do look very dodgy indeed (coupled with very poor plaintext formatting), facts that I would expect a non-expert would notice in some cases. Certainly enough to make it worthwhile.
"Mail clients don't execute JavaScript."
Maybe not on purpose, but EmailPrivacyTester.com has found several clients in the past (usually webmail) which do.
Maybe not on purpose, but EmailPrivacyTester.com has found several clients in the past (usually webmail) which do.
But even there you've got to be careful. For example if you're using GNU Emacs to read your email you could have been vulnerable to arbitrary code execution for the past few years:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28350
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28350
The vulnerability is in the handling of text/enriched parts - not quite text/plain.
https://www-user.tu-chemnitz.de/~heha/email.en.htm
http://problemkaputt.de/email.htm
https://www.gaertner.de/~neitzel/email-to-mn.html
http://www.karo-electronics.de/448.html
...and of course there's this:
http://arc.pasp.de/