Spam blacklisting is out of control(blog.roastidio.us)
blog.roastidio.us
Spam blacklisting is out of control
https://blog.roastidio.us/posts/spam_blacklists_are_out_of_control/
424 comments
I manage an outbound mail server for a mid-sized company. I happen to also use it for my own personal mail.
We have had on and off deliverability issues for years (AT&T and Comcast being the worst).
As head of IT it fell to me to post whitelisting requests and try to get mail delivering again. I decided after awhile that this really isn't my job, and made a suggestion to the CEO which he took to heart:
There is another solution besides changing IPs, using a paid sender, or filling out whitelist requests into the void: Get your legal department involved. We have repeatedly been taken off various public and private blacklists by having lawyers do their job. Once we went this path, it was like magic. Same day responses from those companies, and we haven't been on any blacklists for a couple of years.
We have had on and off deliverability issues for years (AT&T and Comcast being the worst).
As head of IT it fell to me to post whitelisting requests and try to get mail delivering again. I decided after awhile that this really isn't my job, and made a suggestion to the CEO which he took to heart:
There is another solution besides changing IPs, using a paid sender, or filling out whitelist requests into the void: Get your legal department involved. We have repeatedly been taken off various public and private blacklists by having lawyers do their job. Once we went this path, it was like magic. Same day responses from those companies, and we haven't been on any blacklists for a couple of years.
>Get your legal department involved. We have repeatedly been taken off various public and private blacklists by having lawyers do their job.
What's the particular law that makes those curators of blacklists pay attention to your company's lawyers? Do you have example text of those legal requests?
What's the particular law that makes those curators of blacklists pay attention to your company's lawyers? Do you have example text of those legal requests?
It's not a law, it's who handles the request. There's two types of employees in any company with very different KPIs. The first has their performance measured in number of tickets closed. The second has their performance measured in number of incidents allowed. Support tickets get handled by the first type of employee, anything that's plausibly a legal threat gets handled by the second.
The easiest way to close a ticket is a quick wontfix. The easiest way to avoid a possible six-figure minimum legal issue is to spend 15 minutes figuring out what they want and giving it to them. The law is almost entirely irrelevant; the cost-benefit analysis doesn't change enough when comparing a frivolous vs meritorious lawsuit.
The easiest way to close a ticket is a quick wontfix. The easiest way to avoid a possible six-figure minimum legal issue is to spend 15 minutes figuring out what they want and giving it to them. The law is almost entirely irrelevant; the cost-benefit analysis doesn't change enough when comparing a frivolous vs meritorious lawsuit.
This is brilliant logic. It considers all parties involved and what they want.
Just guessing, but maybe threatening to sue for damages might work?
You don't even need to threaten. Just a letter of representation from a lawyer is magically effective for things like this.
ETA: We refer to it as "6mins and a stamp".
ETA: We refer to it as "6mins and a stamp".
Hmm. If lawyer isn't a protected title in my home country, I wonder if it's possible for anyone to take a template and refer to themselves as a lawyer with an email to legal. I suspect they'd see through it, but it might convince someone in some cases?
That's not smart for a number of reasons. But why not just give the template to legal and let them do their job?
Or just write and snail-mail a letter from yourself. People respond surprisingly well to requests and offers in a personal/non-bulk letters.
Or just write and snail-mail a letter from yourself. People respond surprisingly well to requests and offers in a personal/non-bulk letters.
But the companies aren't obligated to accept your email are they? What grounds do you have to ask for damages?
The main benefit of the lawyer here is to break through the bureaucracy on the other end. Your lawyer talks to their lawyer, their lawyer talks to someone on their end who is actually authorized and able to make changes (not necessarily to make them change it - possibly just to figure out what is going on).
This causes a human with authority to actually look at your case and likely go "yeah, this is silly, I've taken care of it". Lawyer is very happy because something that looked like a lot of expensive work just went away.
This causes a human with authority to actually look at your case and likely go "yeah, this is silly, I've taken care of it". Lawyer is very happy because something that looked like a lot of expensive work just went away.
I guess you could try to go for the tortious interference angle:
I have a contract with the customer which involves interacting with them by email. Customer uses EmailCompany. EmailCompany unfairly blocks me from emailing customer. EmailCompany is interfering with the contract. Both myself and the customer have a reasonable expectation that we should be able to carry out our contract by emailing back and forth, so EmailCompany is obstructing the contract.
Any of this sort of "advocacy" only works if the customer actually cares about receiving your emails though.
In my experience, the customer will just switch email providers when I tell them we're blocked. I had an email provider just the other day who wasn't accepting my emails. I tried to get them to unblock us, but to no avail. I told my customer and he went and registered a new email elsewhere.
I have a contract with the customer which involves interacting with them by email. Customer uses EmailCompany. EmailCompany unfairly blocks me from emailing customer. EmailCompany is interfering with the contract. Both myself and the customer have a reasonable expectation that we should be able to carry out our contract by emailing back and forth, so EmailCompany is obstructing the contract.
Any of this sort of "advocacy" only works if the customer actually cares about receiving your emails though.
In my experience, the customer will just switch email providers when I tell them we're blocked. I had an email provider just the other day who wasn't accepting my emails. I tried to get them to unblock us, but to no avail. I told my customer and he went and registered a new email elsewhere.
Lawyers typically will only talk to other lawyers when interacting with other companies (often are ethically compelled to).
An email from [email protected] to [email protected] will be read.
An email from [email protected] to [email protected] will be read.
But this thread is about companyB.com blocking incoming email delivery from companyA.com :D
You need to involve legal.com.
You need to involve legal.com.
Yeah I guess that’s when you rain down BigLegal.com.
Or you send certified mail with return signature requested. Snail mail to the rescue?
Or you send certified mail with return signature requested. Snail mail to the rescue?
Good point, generally. However, it depends on the list or the tactic that's used. Blacklisting an ISP's ASN or entire network range because the blacklist creator set an arbitrary threshold of acceptable number of spam activity from 1 or more IPs. I'm really not sure about the legal aspects here, but there are so many practices that blatantly approach extortion. And it's skillfully done in the name of "online etiquette" or a "safe internet".
Obviously, legitimate use-cases exist for such services, provided that they are operated by faithful people/entities with some level of credibility.
Obviously, legitimate use-cases exist for such services, provided that they are operated by faithful people/entities with some level of credibility.
No, they aren't. This tactic isn't effective when you've truly pissed off the other party. If it could plausibly be an innocent mistake, this strategy will cheaply and quickly get things resolved.
[deleted](1)
Is it a tort of negligent interference? Libel?
Do you know what the lawyers said beyond "I'm a lawyer and would like you to edit the blacklist" ? Are these companies doing something illegal by blacklisting you unfairly, or do you have grounds for some sort of civil suit (if so, what grounds)?
I would think "Tortious interference" is the most likely legal basis to complain about it.
"Tortious interference is a common law tort allowing a claim for damages against a defendant who wrongfully interferes with the plaintiff's contractual or business relationships"
"Tortious interference is a common law tort allowing a claim for damages against a defendant who wrongfully interferes with the plaintiff's contractual or business relationships"
IANAL. But you don't have a contract with the receiver for the receiver to receive your mail. You send mail from an IP on a subnet you control to AT&T. ATT has previously received spam from this subnet (maybe before you acquired the subnet), and they've marked it in a spam list. You're not _paying_ ATT to deliver your mail; that's when you can invoke a lawyer and say that ATT is not doing what you're paying them to do. ATT is under no obligation to route your incoming mail to its destination without any form of payment. Their services are offered to their customers, not to you, the sender.
I'm not saying this situation is _good_. It results in massive consolidation and gatekeeping by Tier 1s for mail. The problem is that it's much easier to send mail over SMTP than receive it, so the cost is on the receiver to filter mail. That means the receiver has every incentive to be overly defensive. I think a hashcash/PoW mechanism or a cryptocurrency deposit mechanism could be a great way to fight some of these problems. Pay a crypto deposit and receive a tunnel to send mail through. If you violate any spam policies, your deposit is lost and the tunnel is closed. Or charge cryptocurrency (quadratically?) per mail sent on this tunnel.
I'm not saying this situation is _good_. It results in massive consolidation and gatekeeping by Tier 1s for mail. The problem is that it's much easier to send mail over SMTP than receive it, so the cost is on the receiver to filter mail. That means the receiver has every incentive to be overly defensive. I think a hashcash/PoW mechanism or a cryptocurrency deposit mechanism could be a great way to fight some of these problems. Pay a crypto deposit and receive a tunnel to send mail through. If you violate any spam policies, your deposit is lost and the tunnel is closed. Or charge cryptocurrency (quadratically?) per mail sent on this tunnel.
You and the receiver are in a business relationship. AT&T, through its “arbitrary” blocklist, is getting in the way. Without further justifications (and “oh it’s the algorithm” is not enough!) they are interfering with your business relationship.
That’s what tortuous interference is, getting in the way of third parties doing business. You could argue about technical details around the server but a judge will properly understand the social constructs at play.
That’s what tortuous interference is, getting in the way of third parties doing business. You could argue about technical details around the server but a judge will properly understand the social constructs at play.
All reasons why you might lose a lawsuit. However, a lawsuit, or even a cease and desist letter is still a nuisance for whoever you sue. Enough that their legal department might ask their ops department to stop blocking the emails. Since that's probably the cheapest route to make you go away.
But AT&T is being paid or has an agreement to operate the recipients' mailboxes.
TOS are gonna TOS, but if the small print reserves the right to block spam, or traffic from spammers, then proof of not being a spammer, or at least proof of a legitimate use, might create an obligation ATT has to its users to not block those addresses, even if there's no obligation to the 3rd party making the request.
The knowledge affects ATT's obligations to others, not the requestor.
Idk where that gets you in a practical sense, but its at least a line of reasoning that's beyond "ATT can block whatever it wants".
TOS are gonna TOS, but if the small print reserves the right to block spam, or traffic from spammers, then proof of not being a spammer, or at least proof of a legitimate use, might create an obligation ATT has to its users to not block those addresses, even if there's no obligation to the 3rd party making the request.
The knowledge affects ATT's obligations to others, not the requestor.
Idk where that gets you in a practical sense, but its at least a line of reasoning that's beyond "ATT can block whatever it wants".
Safe your 500 bucks per hour.
A lawyer will not do anything.
A blacklist is not directly interfering with your business. They just provide a list that contains IPs, your IP, and say we have seen spam traffic from it in the last x hours. The mail receiver, who trusts and uses the list might be interfering, but it is his right to pick and choose who he is accepting email from. Same right you have to pick and choose who you let into your bar, apartment, club, house, ... what so ever.
A blacklist is not directly interfering with your business. They just provide a list that contains IPs, your IP, and say we have seen spam traffic from it in the last x hours. The mail receiver, who trusts and uses the list might be interfering, but it is his right to pick and choose who he is accepting email from. Same right you have to pick and choose who you let into your bar, apartment, club, house, ... what so ever.
Safe your 500 bucks per hour. A lawyer will not do anything
But I do know the "enemy" of my "enemy" is not necessarily my "friend".
I've seen the nightmare of the next Internet. Whether it's micropayments for digital stamps, or slowly refilling quotas, or hard hierarchical controls, it's more power to central authority.
I know I'll never win this argument.
The Internet will keep fracturing and normal citizens will get more frustrated. There will be a change. But just like the hatred of social media, lurking underneath it is an insatiable thirst for power.
[deleted]
Not everyone can spend $500 on lawyer billable hours per SMTP destination multiplied by N number of destinations.
I also think that the likelihood of success in sending legal threats to somebody that demand they accept your SMTP traffic will not stand up in court, if you ever escalated it that far.
As somebody who runs postfix MX on the receiving side of things, I can guarantee you that the day I receive a legal threat from some unknown third party with which I don't have a pre-existing business/contract relationship, demanding that I accept their email, is the day that I blacklist their entire organization and tell them "okay, I'll await service of your statement of claim".
You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs? Companies with which you don't have a signed service order contract and/or master services agreement?
You say you're a mid sized company. I think you're running a huge legal risk of angering a Comcast or AT&T size entity that has much deeper pockets and legal resources than you. The day that one of those giants calls you out on your bluff is going to be very expensive.
On an ISP-to-ISP relationship level, this is not how you solve SMTP flow traffic problems. I can tell you that if I went to a NANOG conference representing my AS and proudly told other people "oh yeah, we've started sending threats from our lawyers to $OTHERISP1 and $OTHERISP2 because they won't take our mail traffic", that I would quickly be treated as a pariah.
I also think that the likelihood of success in sending legal threats to somebody that demand they accept your SMTP traffic will not stand up in court, if you ever escalated it that far.
As somebody who runs postfix MX on the receiving side of things, I can guarantee you that the day I receive a legal threat from some unknown third party with which I don't have a pre-existing business/contract relationship, demanding that I accept their email, is the day that I blacklist their entire organization and tell them "okay, I'll await service of your statement of claim".
You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs? Companies with which you don't have a signed service order contract and/or master services agreement?
You say you're a mid sized company. I think you're running a huge legal risk of angering a Comcast or AT&T size entity that has much deeper pockets and legal resources than you. The day that one of those giants calls you out on your bluff is going to be very expensive.
On an ISP-to-ISP relationship level, this is not how you solve SMTP flow traffic problems. I can tell you that if I went to a NANOG conference representing my AS and proudly told other people "oh yeah, we've started sending threats from our lawyers to $OTHERISP1 and $OTHERISP2 because they won't take our mail traffic", that I would quickly be treated as a pariah.
You have this all backward.
> You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs?
It's not a "network engineering problem" if administrators and managers are the ones making the decisions to provide no reasonable recourse for a ban, on top of actively ignoring or denying legitimate requests for removal. If the third party hasn't broken any rules, then the only resort is getting attorneys involved, because there are literally no other options if the provider refuses to act in good faith.
> Companies with which you don't have a signed service order contract and/or master services agreement?
If you are providing an email service and your customers are not receiving the emails they're expecting, all because you refused to acknowledge a removal request, then it shouldn't matter if you have a relationship with the third party. You've failed your own customers, and for no legitimate, logical or even conceivable reason, other than to assert some personal dominance over another group of engineers that you see as "lesser."
None of this is an engineering problem. It's an asshole problem.
> You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs?
It's not a "network engineering problem" if administrators and managers are the ones making the decisions to provide no reasonable recourse for a ban, on top of actively ignoring or denying legitimate requests for removal. If the third party hasn't broken any rules, then the only resort is getting attorneys involved, because there are literally no other options if the provider refuses to act in good faith.
> Companies with which you don't have a signed service order contract and/or master services agreement?
If you are providing an email service and your customers are not receiving the emails they're expecting, all because you refused to acknowledge a removal request, then it shouldn't matter if you have a relationship with the third party. You've failed your own customers, and for no legitimate, logical or even conceivable reason, other than to assert some personal dominance over another group of engineers that you see as "lesser."
None of this is an engineering problem. It's an asshole problem.
> all because you refused to acknowledge a removal request
But the receiving MSP can't acknowledge the removal request, in the scenario reported by the author. They are not the ones operating the blocklist. That's UCEPROTECT, not Comcast or whoever.
But the receiving MSP can't acknowledge the removal request, in the scenario reported by the author. They are not the ones operating the blocklist. That's UCEPROTECT, not Comcast or whoever.
You have this backward, as well, because that's not how any of this works. The UCEPROTECT list is a literal text file that gets ingested by the mail provider. The provider is under zero obligation to use the entirety of the list and, in fact, is still 100% responsible for maintaining their own list in a way that complies with international laws, ICANN rules, service agreements, etc. UCEPROTECT even offers a very blatant disclaimer on their site.
> USING OUR BLACKLISTS, YOU ARE AT YOUR OWN RISK. WE WILL NOT BE HELD RESPONSIBLE FOR ANY DAMAGE OR PROFIT OR LOSSES. SHOULD THIS BE UNACCEPTABLE TO YOU, YOU ARE NOT ALLOWED TO USE OUR BLACKLISTS.
> USING OUR BLACKLISTS, YOU ARE AT YOUR OWN RISK. WE WILL NOT BE HELD RESPONSIBLE FOR ANY DAMAGE OR PROFIT OR LOSSES. SHOULD THIS BE UNACCEPTABLE TO YOU, YOU ARE NOT ALLOWED TO USE OUR BLACKLISTS.
> The UCEPROTECT list is a literal text file that gets ingested by the mail provider.
I don't know whether that is true; but I do know that it is usually used as a DNSBL - a DNS lookup for an IP address, that answers whether that address is or is not in the list. In general, mail providers do not "ingest" entire blocklists.
> responsible for maintaining their own list in a way that complies with international laws [etc.]
Actually, anyone can publish a list of anything; unless that publication amounts to a contract (statement of purpose, assertion of fitness for purpose), then I'm not aware of any international "law" that says you can't put anything you like in a publicly-acccessible list. And anyway, there's no contract without an exchange of considerations - baiscally, you have to cough-up if you want to assert a contract.
> even offers a very blatant disclaimer on their site
Have you ever read a FOSS licence? UCEPROTECT are simply stating that as a free user, you can't hold them responsible for the accuracy of their lists; in the same way that FOSS authors disclaim responsibility for fitness-for-purpose.
I don't know whether that is true; but I do know that it is usually used as a DNSBL - a DNS lookup for an IP address, that answers whether that address is or is not in the list. In general, mail providers do not "ingest" entire blocklists.
> responsible for maintaining their own list in a way that complies with international laws [etc.]
Actually, anyone can publish a list of anything; unless that publication amounts to a contract (statement of purpose, assertion of fitness for purpose), then I'm not aware of any international "law" that says you can't put anything you like in a publicly-acccessible list. And anyway, there's no contract without an exchange of considerations - baiscally, you have to cough-up if you want to assert a contract.
> even offers a very blatant disclaimer on their site
Have you ever read a FOSS licence? UCEPROTECT are simply stating that as a free user, you can't hold them responsible for the accuracy of their lists; in the same way that FOSS authors disclaim responsibility for fitness-for-purpose.
> In general, mail providers do not "ingest" entire blocklists.
Completely false. They ingest lists[1] into their own local daemon. There is not a major mail provider on the planet who is querying a third party DNSBL every time an email comes in. It's also more than a little ridiculous that you admitted to not knowing if something were true, but then decided to confidently (and incorrectly) explain it anyway, rather than taking ten seconds to look it up.
> Actually, anyone can publish a list of anything; unless that publication amounts to a contract (statement of purpose, assertion of fitness for purpose), then I'm not aware of any international "law" that says you can't put anything you like in a publicly-acccessible list.
Both wrong and irrelevant. I explained how Comcast is responsible for maintaining their own blacklist, which is NOT public, and as an ISP they are absolutely under all sorts of legal regulations, in addition to having service agreements with end users and other network providers. Yet, you responded with a "point" about how UCEPROTECT (a completely different company) is allowed to post whatever they want in a text file. Do you seriously not see the disconnect between what people are actually saying and how you interpret them? Because it's beyond frustrating.
> Have you ever read a FOSS licence? UCEPROTECT are simply stating that as a free user, you can't hold them responsible for the accuracy of their lists
Entirely irrelevant. The UCEPROTECT disclaimer is not for the end user -- it's for the ISPs. If someone sues Comcast, they're not trying to hold UCEPROTECT responsible for creating a text file, they're trying to hold Comcast responsible for acting in bad faith by not allowing reasonable recourse for removal from Comcast's copy of the text file. So, the end user in this situation is not a "free user of UCEPROTECT," but instead a paid user of a network provider with a service agreement in place.
[1] http://www.uceprotect.net/en/index.php?m=6&s=10
Completely false. They ingest lists[1] into their own local daemon. There is not a major mail provider on the planet who is querying a third party DNSBL every time an email comes in. It's also more than a little ridiculous that you admitted to not knowing if something were true, but then decided to confidently (and incorrectly) explain it anyway, rather than taking ten seconds to look it up.
> Actually, anyone can publish a list of anything; unless that publication amounts to a contract (statement of purpose, assertion of fitness for purpose), then I'm not aware of any international "law" that says you can't put anything you like in a publicly-acccessible list.
Both wrong and irrelevant. I explained how Comcast is responsible for maintaining their own blacklist, which is NOT public, and as an ISP they are absolutely under all sorts of legal regulations, in addition to having service agreements with end users and other network providers. Yet, you responded with a "point" about how UCEPROTECT (a completely different company) is allowed to post whatever they want in a text file. Do you seriously not see the disconnect between what people are actually saying and how you interpret them? Because it's beyond frustrating.
> Have you ever read a FOSS licence? UCEPROTECT are simply stating that as a free user, you can't hold them responsible for the accuracy of their lists
Entirely irrelevant. The UCEPROTECT disclaimer is not for the end user -- it's for the ISPs. If someone sues Comcast, they're not trying to hold UCEPROTECT responsible for creating a text file, they're trying to hold Comcast responsible for acting in bad faith by not allowing reasonable recourse for removal from Comcast's copy of the text file. So, the end user in this situation is not a "free user of UCEPROTECT," but instead a paid user of a network provider with a service agreement in place.
[1] http://www.uceprotect.net/en/index.php?m=6&s=10
> If you are providing an email service and your customers are not receiving the emails they're expecting, all because you refused to acknowledge a removal request,
And on this I concur with you, because if the sender of the email is actually sending spam and has ended up on some smtp-receiving deny lists for well founded reasons, they are indeed failing their customers. Failing them through their own lack of procedure and policy, such as not having a click to remove link in the mail, and not removing addresses/putting addresses into a scrub-before-sending list.
If that is the case, they have nobody to blame but themselves for that and should not be sending legal demands to other peoples' ISPs such as at&t or Comcast.
And on this I concur with you, because if the sender of the email is actually sending spam and has ended up on some smtp-receiving deny lists for well founded reasons, they are indeed failing their customers. Failing them through their own lack of procedure and policy, such as not having a click to remove link in the mail, and not removing addresses/putting addresses into a scrub-before-sending list.
If that is the case, they have nobody to blame but themselves for that and should not be sending legal demands to other peoples' ISPs such as at&t or Comcast.
You're assuming facts not in evidence, such as that we don't have policies for unsubscribing, or that we're sending spam in the first place. To be clear, we've never complained to anyone except on behalf of customers who complained to us that they weren't getting our mail.
He didn’t say threaten them.
It’s pretty easy to envision a situation where a lawyer sends a quite friendly and factual email to a company, that is literally identical to the one the IT head would have sent, but because it’s coming from a lawyer the recipient uses completely different internal routing to process the request. So someone actually takes the request seriously.
Seems both plausible and a reasonable thing to do for a company large enough to have a legal department.
People pay attention to lawyer letters. You’ve pretty much confirmed as much by noting that letters from a lawyer are so concerning to you that the mere mention of one makes you assume it’s a threat.
If you got a letter from an attorney asking for something nicely, and it was a reasonable request you would automatically reply “SUE ME” just on principle? What’s the principle?
It’s pretty easy to envision a situation where a lawyer sends a quite friendly and factual email to a company, that is literally identical to the one the IT head would have sent, but because it’s coming from a lawyer the recipient uses completely different internal routing to process the request. So someone actually takes the request seriously.
Seems both plausible and a reasonable thing to do for a company large enough to have a legal department.
People pay attention to lawyer letters. You’ve pretty much confirmed as much by noting that letters from a lawyer are so concerning to you that the mere mention of one makes you assume it’s a threat.
If you got a letter from an attorney asking for something nicely, and it was a reasonable request you would automatically reply “SUE ME” just on principle? What’s the principle?
In the American legal system, if somebody spends the money to take the time to have their lawyer hand craft and send me a letter about something such as this, I'm going to take it as a threat whether or not it specifically contains one.
The implication is that if you do not do whatever is demanded in the letter, the next step will be the client of said lawyer escalating the situation to paying their lawyer to actually sue you.
The implication is that if you do not do whatever is demanded in the letter, the next step will be the client of said lawyer escalating the situation to paying their lawyer to actually sue you.
I mean, not really. In the world of companies talking to other companies lawyers are involved all the time. In some scenarios (real estate transactions, or M&A, for example) it would just be completely routine for two companies on exceedingly good terms to communicate back and forth via attorneys.
Your main premise seems to be that the recipient should take all this very personally. But it’s not personal, these are businesses, discussing a business related topic.
The VAST majority of business to business communications that involve attorneys don’t go anywhere near an actual filed lawsuit.
Your main premise seems to be that the recipient should take all this very personally. But it’s not personal, these are businesses, discussing a business related topic.
The VAST majority of business to business communications that involve attorneys don’t go anywhere near an actual filed lawsuit.
An intentionally initiated transaction where everyone already knows each other, and knows in advance that lawyers will be involved (as you say, m&a, real estate, etc) is a very different thing than receiving a demand letter from a previously unknown party out of the blue.
Like I said in my original reply here, we are talking about sending threats to third-party isps, with which the originator of the smtp traffic has no existing business or contractual relationship.
Like I said in my original reply here, we are talking about sending threats to third-party isps, with which the originator of the smtp traffic has no existing business or contractual relationship.
I made all kinds of good faith efforts personally to contact someone at AT&T through their forms and phone numbers, without ever even speaking to anyone who could resolve the problem if they wanted to. Lacking any other way to get through to them, legal seemed like the only recourse.
I would think that if you were the ISP owner blocking our mail to your customer, a single phone call from the customer or from us would convince you that our IPs were not a spam threat. That's certainly the way I prefer to deal with anything. But when dealing with something as monolithic and totally deaf as AT&T, lawyers were the only way.
I would think that if you were the ISP owner blocking our mail to your customer, a single phone call from the customer or from us would convince you that our IPs were not a spam threat. That's certainly the way I prefer to deal with anything. But when dealing with something as monolithic and totally deaf as AT&T, lawyers were the only way.
I would think an attorney wouldn’t get involved until other avenues were exhausted. As alluded to in the comment. If it’s out of the blue, check your spam folder.
Yes that’s the idea. They’ve spent a bit of money to make a sincere and well-founded request that will cause you to consult your own lawyer who will explain to you why you’re probably doing something wrong and need to do what the letter asks of you.
Of course there’s a lot of garbage out there too.
Of course there’s a lot of garbage out there too.
The fact that you would automatically react in this way is itself a reason to send it to the lawyers and have it go through a different department and chain of command.
Make it a legal and business decision instead of a technical one. And find an audience that knows the practical benefits of quick resolutions to reasonable requests.
And I say this as a person that would likely respond in a similar punitive and/or principled way to what you described upthread. Which is exactly why I leave it to the lawyers.
Make it a legal and business decision instead of a technical one. And find an audience that knows the practical benefits of quick resolutions to reasonable requests.
And I say this as a person that would likely respond in a similar punitive and/or principled way to what you described upthread. Which is exactly why I leave it to the lawyers.
>> Make it a legal and business decision instead of a technical one.
I'd go further and say that choosing to unblock some IP addresses out of a block you don't like is always going to be a business decision. The whole rationale for blocking in the first place was that a sender is untrustworthy.
Untrustworthy senders generally don't have a team of lawyers, and if they do (and they're actually sending spam) you can show what spam they sent and keep them blocked.
All the legal letter does is force someone who's otherwise too busy to look at this particular case for the 5 seconds it takes to determine that it's not a threat. Sad to say that being head of IT for a company for a decade doesn't get you the same level of respect; it might if they even bothered to open your email, but there's something about postal and legal letterhead.
I'd go further and say that choosing to unblock some IP addresses out of a block you don't like is always going to be a business decision. The whole rationale for blocking in the first place was that a sender is untrustworthy.
Untrustworthy senders generally don't have a team of lawyers, and if they do (and they're actually sending spam) you can show what spam they sent and keep them blocked.
All the legal letter does is force someone who's otherwise too busy to look at this particular case for the 5 seconds it takes to determine that it's not a threat. Sad to say that being head of IT for a company for a decade doesn't get you the same level of respect; it might if they even bothered to open your email, but there's something about postal and legal letterhead.
But lawyers don’t cost nearly as much outside the US. We moved from the US to the NL a few years ago. Legal representation is pennies compared to the US. So just because it’s expensive where you are, doesn’t mean it is everywhere else.
It doesn’t mean they are thinking to sue you. Not even close. When you actually get served, that’s when they’re thinking about it, because actually serving you is a few bits of paper and requires a few brain cells. Sending you what amounts to a form letter? No brain cells required.
It doesn’t mean they are thinking to sue you. Not even close. When you actually get served, that’s when they’re thinking about it, because actually serving you is a few bits of paper and requires a few brain cells. Sending you what amounts to a form letter? No brain cells required.
The rationale for involving legal is to place some accountability and consequences where they belong.
Currently, countless people essentially commit countless abuses for free because the actor is hidden behind a machine or a process. But somewhere it's a humans decision to institute an abusive protocol, and it seems pretty fair fo me to make that human accountable for their action. Not just email but all kinds of things.
You are probably merely a dick but still a legal dick if you wantonly block email for yourself. But the second you are responsible for even one other person's correspondence reaching them, I say you should be legally culpable for any failure to deliver.
Currently, countless people essentially commit countless abuses for free because the actor is hidden behind a machine or a process. But somewhere it's a humans decision to institute an abusive protocol, and it seems pretty fair fo me to make that human accountable for their action. Not just email but all kinds of things.
You are probably merely a dick but still a legal dick if you wantonly block email for yourself. But the second you are responsible for even one other person's correspondence reaching them, I say you should be legally culpable for any failure to deliver.
I'm a dick?
I question whether you or what other percentage of the commenters in this thread represent any specific ASN with its own IP space that it cares about keeping clean, and have bgp relationships with other ISPs.
Or whether they're actually end users only.
Have you actually encountered this problem as a service provider in the past and implemented solutions to it, or are you just sharing your opinion as a possibly-frustrated end user of email?
I question whether you or what other percentage of the commenters in this thread represent any specific ASN with its own IP space that it cares about keeping clean, and have bgp relationships with other ISPs.
Or whether they're actually end users only.
Have you actually encountered this problem as a service provider in the past and implemented solutions to it, or are you just sharing your opinion as a possibly-frustrated end user of email?
I avoided the problem by only ever sending email on behalf of customers, not receiving. IE, saas app can send out quotes and invoices and things but any replies go to the end users own address not any of our servers, and didn't host our own email.
What I didn't do was ignore the problem or think there was no problem. Things still got blocked elsewhere but I (personally or my company) didn't do it.
I recognized that handling someone elses correspondence is a non-trivial responsibility. Yes the job is hard, and so you either decide that is your whole job, or you outsource it to someone whos whole job it is. You don't just do a poor job because it's hard and not your all day every day job.
What I didn't do was ignore the problem or think there was no problem. Things still got blocked elsewhere but I (personally or my company) didn't do it.
I recognized that handling someone elses correspondence is a non-trivial responsibility. Yes the job is hard, and so you either decide that is your whole job, or you outsource it to someone whos whole job it is. You don't just do a poor job because it's hard and not your all day every day job.
> I say you should be legally culpable for any failure to deliver.
So you think an MSP's advertised policy should be "We guarantee that anything sent to you will be delivered, including spam"? That no MSP should provide spam-filtering, at risk of legal culpability?
If that's not what you mean, then presumably you are requiring all MSPs to block only spam, and to deliver all legitimate email. But that is impossible, because nobody has figured out a way of reliably distinguishing spam from ham.
If I received such a letter from your legal department, I would laugh, and reply: "I am awaiting your writ." If I got the note from a recipient or their postmaster, I'd be much more inclined to try to help, but I never try to negotiate with lawyers brandishing threats.
So you think an MSP's advertised policy should be "We guarantee that anything sent to you will be delivered, including spam"? That no MSP should provide spam-filtering, at risk of legal culpability?
If that's not what you mean, then presumably you are requiring all MSPs to block only spam, and to deliver all legitimate email. But that is impossible, because nobody has figured out a way of reliably distinguishing spam from ham.
If I received such a letter from your legal department, I would laugh, and reply: "I am awaiting your writ." If I got the note from a recipient or their postmaster, I'd be much more inclined to try to help, but I never try to negotiate with lawyers brandishing threats.
The MSP's policy can be whatever they want as long as it IS advertised.
If they say "We drop 10% of messages at random, and you knowingly choose to take that, then that is just a stupid arrangement you should never agree to, but they aren't doing something unexpected.
I decline to believe you are as stupid as that remark.
Good faith best effort is perfectly reasonable. Not knowingly and intentionaly discarding mail is all that's required.
If you're the mailman, you do not have to garantee that you will never lose a single letter in a car accident.
But you can certainly garantee, absolutely, that you never go through the bag and throw away all the spam and sometimes mistake a legit letter from a lawyer for lawyer spam.
You can ceetainly garantee, absolutely, that you never apply utterly thoughtless rules like "we got these scam letters from Nigeria so now we just throw away anything from Nigeria."
You should absolutely be responsible for other peoples stuff that is in your hands while it is in your hands. It's not yours to dispose of, even when part of your explicit job is to filter. If that sounds onerous, that's why you get paid money for the responsibility. If it's too hard to bear this responsibility properly, then you have no business doing that job. Do the job right or don't do the job. There is nothing unreasonable about those two choices. It is not at all required to do the job, but poorly or carelessly.
If they say "We drop 10% of messages at random, and you knowingly choose to take that, then that is just a stupid arrangement you should never agree to, but they aren't doing something unexpected.
I decline to believe you are as stupid as that remark.
Good faith best effort is perfectly reasonable. Not knowingly and intentionaly discarding mail is all that's required.
If you're the mailman, you do not have to garantee that you will never lose a single letter in a car accident.
But you can certainly garantee, absolutely, that you never go through the bag and throw away all the spam and sometimes mistake a legit letter from a lawyer for lawyer spam.
You can ceetainly garantee, absolutely, that you never apply utterly thoughtless rules like "we got these scam letters from Nigeria so now we just throw away anything from Nigeria."
You should absolutely be responsible for other peoples stuff that is in your hands while it is in your hands. It's not yours to dispose of, even when part of your explicit job is to filter. If that sounds onerous, that's why you get paid money for the responsibility. If it's too hard to bear this responsibility properly, then you have no business doing that job. Do the job right or don't do the job. There is nothing unreasonable about those two choices. It is not at all required to do the job, but poorly or carelessly.
> The MSP's policy can be whatever they want as long as it IS advertised.
No.
Suppose the MSP uses bayesian filtering? How would one go about advertising a policy that depends on bayesian filtering? You'd have to publish the contents of your filter table. The only people who could benefit from that would be spammers, who could use the data to customise their spam.
In general, telling the world what your filtering policies are is just going to cause spammers to try to sidestep your policies. The only policy that you maybe ought to publish would be along the lines of "We filter out spam; that sometimes results in false positives. Sorry."
No.
Suppose the MSP uses bayesian filtering? How would one go about advertising a policy that depends on bayesian filtering? You'd have to publish the contents of your filter table. The only people who could benefit from that would be spammers, who could use the data to customise their spam.
In general, telling the world what your filtering policies are is just going to cause spammers to try to sidestep your policies. The only policy that you maybe ought to publish would be along the lines of "We filter out spam; that sometimes results in false positives. Sorry."
Then you mean "yes".
Did you read the whole article? The OP wasn't being blacklisted by an operator. He was being blacklisted by a company that charges you a $25/month fee to not be blacklisted by lazy operators who program their systems to curl their for-profit blacklist.
People have been making legal threats at, and trying to sue, RBL operators since 1997 or so. It's a well known thing. All I say is "good luck" if you think legally threatening a maintainer of a list of IP CIDR prefixes that are used by a third party is going to solve your problems. It hasn't worked for the last 25 years and I don't see how it'll start working now.
No I think the solution is to not get a $5/month vps and expect it to have a good reputation. Maybe if you went with the $100/month hosting provider you wouldn't have needed to spend $500/month on legal threats. Internet addresses aren't fungible. It's a well known concept in telecommunications. One of the reasons why people have always paid a lot more money to have 212 numbers versus 646 numbers. It's the reason why if you want to set up a respectable business, you don't rent property in a high crime neighborhood. Browsers and email clients should ideally have more transparency about the ASNs and service providers who host the websites that people are visiting, so that everyone can develop a mental model of which ones are associated with good things, because right now the only people who are able to have any kind of awareness of this thing are operators who regularly monitor traffic.
This is completely valid, but if you go hunting for an IP block that has never been used for spam at this point you're going to be looking for a long time. It should not be the #1 consideration when choosing a hosting provider just because someone abused their IP block in the past (possibly before they even owned it). In any case, trying to run mail off a VPS would be stupid and that's not what I'm saying. I'm saying we don't all need to capitulate to paying Amazon or Google to forward our outbound mail, and it would be better if we did not, even considering the struggles attached to bucking the trend.
Literally prejudice ad a service, however.
> Not everyone can spend $500 on lawyer billable hours per SMTP destination multiplied by N number of destinations.
You likely wouldn't do that - just get a template version that gets reused, just like you pay once for a contract / t&c you reuse with multiple parties.
You likely wouldn't do that - just get a template version that gets reused, just like you pay once for a contract / t&c you reuse with multiple parties.
> You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs?
If the remote mailserver is explicitly bouncing your mails, it's not a network engineering problem.
If you're able to complete a TCP 3 way handshake, it's not a network engineering problem.
If you're being prevented from completing a TCP 3 way handshake because of a filter list on the MX server, it's not a network engineering problem. Admittedly, you can't tell that one without asking a network engineer (on the remote network) to validate that for you, or asking the application owner to advise.
Contrary to popular advice (thanks MSFT), you probably shouldn't contact your network administrator. We probably have priv 15, but we dont necessarily have root.
If the remote mailserver is explicitly bouncing your mails, it's not a network engineering problem.
If you're able to complete a TCP 3 way handshake, it's not a network engineering problem.
If you're being prevented from completing a TCP 3 way handshake because of a filter list on the MX server, it's not a network engineering problem. Admittedly, you can't tell that one without asking a network engineer (on the remote network) to validate that for you, or asking the application owner to advise.
Contrary to popular advice (thanks MSFT), you probably shouldn't contact your network administrator. We probably have priv 15, but we dont necessarily have root.
Tangentially related:
Somebody created a github containing domains he thought should be blocked: https://github.com/chadmayfield/my-pihole-blocklists
A year later I bought a domain that had expired under my country TLD. It turns out that domain for some reason was previously added to that list.
Now, as you can see, the man behind that Github repo has decided to archive that repo and therefore make it read only.
As you can tell from both pull requests and issues on that repo, people has asked him to remote legitimate domains (e.g. *.urbandictionary.com). But those calls remain unanswered. It is fruitless to contact the author.
So this random dude causes real problems for legitimate business and individuals and we should just accept it?
Obviously he doesn't act on friendly geek requests. It seems lawyering up would be the only recourse in this situation. I find it analogous to somebody standing on a soap box in a village and announcing: "Don't trust James. Don't trust Mary either. There's problems with Charles" and James, Mary, and Charles have no way of stopping his libel.
I don't have the funds for legal action, but it is obviously wrong that he can announce "these domains are bad" and offer no way of fixing mistakes. He should take down the repo, but oh that sweet sweet Github karma probably discourages him from doing so.
Somebody created a github containing domains he thought should be blocked: https://github.com/chadmayfield/my-pihole-blocklists
A year later I bought a domain that had expired under my country TLD. It turns out that domain for some reason was previously added to that list.
Now, as you can see, the man behind that Github repo has decided to archive that repo and therefore make it read only.
As you can tell from both pull requests and issues on that repo, people has asked him to remote legitimate domains (e.g. *.urbandictionary.com). But those calls remain unanswered. It is fruitless to contact the author.
So this random dude causes real problems for legitimate business and individuals and we should just accept it?
Obviously he doesn't act on friendly geek requests. It seems lawyering up would be the only recourse in this situation. I find it analogous to somebody standing on a soap box in a village and announcing: "Don't trust James. Don't trust Mary either. There's problems with Charles" and James, Mary, and Charles have no way of stopping his libel.
I don't have the funds for legal action, but it is obviously wrong that he can announce "these domains are bad" and offer no way of fixing mistakes. He should take down the repo, but oh that sweet sweet Github karma probably discourages him from doing so.
> I find it analogous to somebody standing on a soap box in a village and announcing: "Don't trust James. Don't trust Mary either.
You're speaking of some random domain-list on github - that isn't even maintained? Taken from someone's private pihole?
Anyone using a list of that kind to block is simply incompetent. Even as part of a scoring system, it's pretty silly. Before adding a blocklist, a postmaster needs to familiarize herself with the list's policies. Are list entries aged-out? How quickly? Do they use spamtraps, or user-reports? Or is it just the whim of the list-maintainer? Do they block individual addresses, whole domains, or entire allocations?
> So this random dude causes real problems for legitimate business and individuals and we should just accept it?
So you're having problems sending mail to a domain where the postmaster cares more about rejecting spam than she does about receiving legitimate email. That's a matter for your recipient to take up with their MSP. And if the recipient wants to receive mail from small-time domains, they need to accept that they're going to receive some spam as well; but maybe they need to switch to an MSP that only rejects on strong evidence.
My point is that it's your recipient's choice to use an MSP that blocks using some crazy list they found on github.
Some postmasters will block everything from selected countries; at one time I would block everything from Romania, because none of my users had correspondents in Romania, and email from Romania at that time was 100% spam. But I wasn't providing service to the public. I knew all my users.
Different MTAs have different users, and different patterns of abusive email. So if you want to use a custom blocklist, make your own, based on your own incoming spam (and then you can honour removal requests yourself). Otherwise use a public blocklist, based on multiple spamtraps in multiple ISPs.
So yes, you should just accept it. You don't have a right to have mail delivered by any MSP you send to; they're private organisations or individuals, and they're entitled to determine what their own policies are. In the world of email, nobody is entitled to protection from the foolishness of others.
You're speaking of some random domain-list on github - that isn't even maintained? Taken from someone's private pihole?
Anyone using a list of that kind to block is simply incompetent. Even as part of a scoring system, it's pretty silly. Before adding a blocklist, a postmaster needs to familiarize herself with the list's policies. Are list entries aged-out? How quickly? Do they use spamtraps, or user-reports? Or is it just the whim of the list-maintainer? Do they block individual addresses, whole domains, or entire allocations?
> So this random dude causes real problems for legitimate business and individuals and we should just accept it?
So you're having problems sending mail to a domain where the postmaster cares more about rejecting spam than she does about receiving legitimate email. That's a matter for your recipient to take up with their MSP. And if the recipient wants to receive mail from small-time domains, they need to accept that they're going to receive some spam as well; but maybe they need to switch to an MSP that only rejects on strong evidence.
My point is that it's your recipient's choice to use an MSP that blocks using some crazy list they found on github.
Some postmasters will block everything from selected countries; at one time I would block everything from Romania, because none of my users had correspondents in Romania, and email from Romania at that time was 100% spam. But I wasn't providing service to the public. I knew all my users.
Different MTAs have different users, and different patterns of abusive email. So if you want to use a custom blocklist, make your own, based on your own incoming spam (and then you can honour removal requests yourself). Otherwise use a public blocklist, based on multiple spamtraps in multiple ISPs.
So yes, you should just accept it. You don't have a right to have mail delivered by any MSP you send to; they're private organisations or individuals, and they're entitled to determine what their own policies are. In the world of email, nobody is entitled to protection from the foolishness of others.
[deleted]
>> per SMTP destination multiplied by N number of destinations
The number of people using anything other than google, apple, microsoft or yahoo for personal email these days is vanishingly small; and of the remainder, almost all of them are on major broadband providers. If you were some local ISP blocking us, that wouldn't be worth the trouble of sending a legal letter. I'd just tell that particular customer to contact you and ask why you weren't delivering our mail.
This only leaves other corporations that provide their own mail services to people who work there, and if those people want to get their mail they can call IT.
The number of people using anything other than google, apple, microsoft or yahoo for personal email these days is vanishingly small; and of the remainder, almost all of them are on major broadband providers. If you were some local ISP blocking us, that wouldn't be worth the trouble of sending a legal letter. I'd just tell that particular customer to contact you and ask why you weren't delivering our mail.
This only leaves other corporations that provide their own mail services to people who work there, and if those people want to get their mail they can call IT.
So what's the advice for an avg Joe for getting a reply from ATT, Comcast, etc when they unjustly blacklist you and ignore all correspondence?
1. Host your mx somewhere that isn't on any blacklists. This means a small to medium sized isp, where you can directly contact the people who run the core network operations there, and who truly do care about kicking off abusive other customers very quickly. Ideally I would go with an ISP in your own region and home business area. Best chances of success if it's a hosting ISP where random customers cannot sign up online with just a name and a credit card, but it's more of a "contact us for a custom price quotation for your colocation needs" type of hosting operation.
2. Possibly run all your outbound smtp through a trusted third party service that you pay for such relay. Leaves a bad taste in my mouth but that's where we are at in 2022.
3. Be absolutely certain that your own smtp, spf, dkim, dmarc configuration is flawless and you've never been a source of spam.
2. Possibly run all your outbound smtp through a trusted third party service that you pay for such relay. Leaves a bad taste in my mouth but that's where we are at in 2022.
3. Be absolutely certain that your own smtp, spf, dkim, dmarc configuration is flawless and you've never been a source of spam.
Host your mx somewhere that isn't on any blacklists. This means a small to medium sized isp, where you can directly contact the people who run the core network operations there, and who truly do care about kicking off abusive other customers very quickly. Ideally I would go with an ISP in your own region and home business area.
That's a nice idea. In fact, it's what my businesses have done for years. I have personally met several of the senior staff at the service I use and I know that they are both technically excellent and very serious about preventing abuse on their network.
We haven't been able to deliver mails to customers at certain mail services for years. 100% blackholed every time. And it's the same usual suspects that others have mentioned in this very discussion.
The problem is that the hosting service we use has many customers. Occasionally one of those customers makes a mistake and one of their systems gets compromised for a short time, until they or the hosting service detects the unusual traffic and intervenes. And that's enough to get someone's IP block on a blacklist it will evidently never leave.
This is of course absolutely no different to any hosting service like AWS or Azure, except that no-one is going to blanket block all servers from an organisation of that scale even though the exact same problems happen there too.
That's a nice idea. In fact, it's what my businesses have done for years. I have personally met several of the senior staff at the service I use and I know that they are both technically excellent and very serious about preventing abuse on their network.
We haven't been able to deliver mails to customers at certain mail services for years. 100% blackholed every time. And it's the same usual suspects that others have mentioned in this very discussion.
The problem is that the hosting service we use has many customers. Occasionally one of those customers makes a mistake and one of their systems gets compromised for a short time, until they or the hosting service detects the unusual traffic and intervenes. And that's enough to get someone's IP block on a blacklist it will evidently never leave.
This is of course absolutely no different to any hosting service like AWS or Azure, except that no-one is going to blanket block all servers from an organisation of that scale even though the exact same problems happen there too.
The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness. It's usually wrong to assume that everything in a /24 is controlled by one botnet, and it's not that hard to check whether it was just one or two particular addresses that were compromised. But if you're an ISP and you want to take the nuclear option to every spam threat, at least be willing to listen to your own customers when they complain that they're expecting mail, and there's absolutely NO reason to assign "group punishment" to everyone using the same service provider. I think the thought was that that would make service providers more accountable, but it's totally unfair to use everyday customers as pawns in a war between ISPs.
> The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness.
Yes, in that situation it's laziness, but on the part of your ISP. ISPs already spend huge amounts of time and money dealing with spam, hacking attempts, phishing attacks, etc. If your ISP is irresponsible and isn't doing their job keeping those things from leaving their network then your ISP is gong to find their entire IP space blocked and that's 100% reasonable. Why should we ever accept traffic from an ISP that refuses to keep their corner of the network clean when it's just going to cause problems for us and our users?
That's the situation for every ISP on the internet. Keep your users in line, keep trash off your network or else no one is going to accept traffic from you. You could call it laziness, but it simply isn't worth it. We'd rather spend our time cleaning up abuse on our own network and working with ISPs who are doing their job than dealing with the problems we get from bad actors.
If you own an IP surrounded by a bunch of spammers and it's giving you trouble step 1 should be to contact your ISP and tell them to get their shit in order or you'll take your business to an ISP who does their job. Step two is to switch to a new ISP if they don't. No one has the right to force us to accept traffic from anyone else. It's every ISPs responsibility to make sure the traffic leaving their network isn't more trouble than it's worth. Good ISPs are rewarded because users will give them their business and stay and bad ISPs are punished because users will drop their service when they see they are blocked.
The goal isn't to punish the poor sucker who signed on with an irresponsible host, but to cut down on the number of bad ISPs on the internet and the amount of work we have to deal with coming from them.
Yes, in that situation it's laziness, but on the part of your ISP. ISPs already spend huge amounts of time and money dealing with spam, hacking attempts, phishing attacks, etc. If your ISP is irresponsible and isn't doing their job keeping those things from leaving their network then your ISP is gong to find their entire IP space blocked and that's 100% reasonable. Why should we ever accept traffic from an ISP that refuses to keep their corner of the network clean when it's just going to cause problems for us and our users?
That's the situation for every ISP on the internet. Keep your users in line, keep trash off your network or else no one is going to accept traffic from you. You could call it laziness, but it simply isn't worth it. We'd rather spend our time cleaning up abuse on our own network and working with ISPs who are doing their job than dealing with the problems we get from bad actors.
If you own an IP surrounded by a bunch of spammers and it's giving you trouble step 1 should be to contact your ISP and tell them to get their shit in order or you'll take your business to an ISP who does their job. Step two is to switch to a new ISP if they don't. No one has the right to force us to accept traffic from anyone else. It's every ISPs responsibility to make sure the traffic leaving their network isn't more trouble than it's worth. Good ISPs are rewarded because users will give them their business and stay and bad ISPs are punished because users will drop their service when they see they are blocked.
The goal isn't to punish the poor sucker who signed on with an irresponsible host, but to cut down on the number of bad ISPs on the internet and the amount of work we have to deal with coming from them.
That seems a lot of rationalisation for a situation where a genuine sender on another system sends legitimate mail to a genuine recipient on your system, that mail is not properly delivered, and it's your fault.
There is a reason that collective punishment is considered immoral by civilised cultures. It hurts the innocent and often fails to achieve its original goal anyway.
There is a reason that collective punishment is considered immoral by civilised cultures. It hurts the innocent and often fails to achieve its original goal anyway.
> and it's your fault
Or maybe your policy?
> There is a reason that collective punishment is considered immoral by civilised cultures
Rejecting email submissions isn't punishment, collective or otherwise. It's something you have to do if you run a mailserver. In the same way, I'm not punishing trespassers if I secure my front-door with a deadlock.
Or maybe your policy?
> There is a reason that collective punishment is considered immoral by civilised cultures
Rejecting email submissions isn't punishment, collective or otherwise. It's something you have to do if you run a mailserver. In the same way, I'm not punishing trespassers if I secure my front-door with a deadlock.
No, but if you secure a cellar with a padlock on hundreds of people, thats called kidnapping and possibly murder.
> That seems a lot of rationalisation for a situation where a genuine sender on another system sends legitimate mail to a genuine recipient on your system, that mail is not properly delivered, and it's your fault.
It's how the internet stays functional. If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture. Bad actors have been ostracized from communities for as long communities have existed. If you want reliable service, choose a responsible ISP. Blacklists have enabled the internet and email to remain useful for most users most of the time. Without them, email wouldn't be usable. If you have a better solution for spam, the whole world would love to hear it.
It's how the internet stays functional. If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture. Bad actors have been ostracized from communities for as long communities have existed. If you want reliable service, choose a responsible ISP. Blacklists have enabled the internet and email to remain useful for most users most of the time. Without them, email wouldn't be usable. If you have a better solution for spam, the whole world would love to hear it.
It's how the internet stays functional.
[citation needed]
If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture.
1. It's way more than 0.001%. Like, several orders of magnitude more.
2. What overwhelming amounts of spam/attacks? Those of us using traditional mail systems with traditional spam filtering seem to be doing OK not getting overwhelmed by incoming spam without the kind of "help" you advocate.
3. You don't get to decide what's acceptable to everyone else. Except that apparently you've decided you do, which is why regulation is needed to remove that ability from service providers who can't do their jobs properly and hurt others as a result.
If you want reliable service, choose a responsible ISP.
This is a poor argument. Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them. The decent ones will identify the problem and block it reasonably quickly, but there is plenty of evidence that they can still be blacklisted and it can still be difficult or impossible to get removed from those blacklists again after the problem is fixed.
The kind of policy you advocate punishes small ISPs just for being ISPs. I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
Or maybe the rest of us should apply the same policy to organisations that do what you advocate. If they won't deliver mail reliably, we won't forward any mail to them at all, so their mail service becomes useless. Except of course it's some of the biggest mail services that do this, so just like no-one's going to block incoming mail from AWS or MailChimp, no-one's going to block outgoing mail to Google or Microsoft.
If you have a better solution for spam, the whole world would love to hear it.
Block actual spam sources and provide a reasonable method for removing blocks that are no longer necessary. Don't carpet bomb whole chunks of the Internet just because there are a few bad actors around. Don't fire and forget. It's really not difficult and plenty of small organisations operate just fine on this basis every day.
[citation needed]
If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture.
1. It's way more than 0.001%. Like, several orders of magnitude more.
2. What overwhelming amounts of spam/attacks? Those of us using traditional mail systems with traditional spam filtering seem to be doing OK not getting overwhelmed by incoming spam without the kind of "help" you advocate.
3. You don't get to decide what's acceptable to everyone else. Except that apparently you've decided you do, which is why regulation is needed to remove that ability from service providers who can't do their jobs properly and hurt others as a result.
If you want reliable service, choose a responsible ISP.
This is a poor argument. Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them. The decent ones will identify the problem and block it reasonably quickly, but there is plenty of evidence that they can still be blacklisted and it can still be difficult or impossible to get removed from those blacklists again after the problem is fixed.
The kind of policy you advocate punishes small ISPs just for being ISPs. I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
Or maybe the rest of us should apply the same policy to organisations that do what you advocate. If they won't deliver mail reliably, we won't forward any mail to them at all, so their mail service becomes useless. Except of course it's some of the biggest mail services that do this, so just like no-one's going to block incoming mail from AWS or MailChimp, no-one's going to block outgoing mail to Google or Microsoft.
If you have a better solution for spam, the whole world would love to hear it.
Block actual spam sources and provide a reasonable method for removing blocks that are no longer necessary. Don't carpet bomb whole chunks of the Internet just because there are a few bad actors around. Don't fire and forget. It's really not difficult and plenty of small organisations operate just fine on this basis every day.
> What overwhelming amounts of spam/attacks?
The majority of all email has been spam, for more than a decade.
> You don't get to decide what's acceptable to everyone else
If you operate a mailserver, you do actually get to decide what kind of stuff you are willing to accept, and from who. "Everybody else" does not get an automatic right to inject data into my computer.
> The kind of policy you advocate punishes small ISPs
Not at all (perhaps you meant MSPs?) Blocklisting Google was a reasonable policy, at one time. Blocklisting MailChimp is perfectly reasonable now.
> Block actual spam sources
Of course, good plan. Unless the sender's ISP is in the habit of moving spammers from one address to another, so they can evade blocks. Then you have to block the ISP, or eat their spam.
Postmasters can't inspect every inbound spam!
The majority of all email has been spam, for more than a decade.
> You don't get to decide what's acceptable to everyone else
If you operate a mailserver, you do actually get to decide what kind of stuff you are willing to accept, and from who. "Everybody else" does not get an automatic right to inject data into my computer.
> The kind of policy you advocate punishes small ISPs
Not at all (perhaps you meant MSPs?) Blocklisting Google was a reasonable policy, at one time. Blocklisting MailChimp is perfectly reasonable now.
> Block actual spam sources
Of course, good plan. Unless the sender's ISP is in the habit of moving spammers from one address to another, so they can evade blocks. Then you have to block the ISP, or eat their spam.
Postmasters can't inspect every inbound spam!
> [citation needed]
Nearly 85% of all emails are spam. source: https://dataprot.net/statistics/spam-statistics/
At times that number has been even higher with over 90% of all messages sent over the internet being spam. If 90% of all the messages in your inbox were spam how long would you continue to use it? Email systems can't bear the costs that spam forces on them. Even with tools like blacklisting which you think shouldn't exist that cost is measured in tens of billions annually. source: https://www.aeaweb.org/articles?id=10.1257/jep.26.3.87
If not for the ability to filter common sources of spam, email would never have survived as a viable means of communication.
> It's way more than 0.001%. Like, several orders of magnitude more.
Whatever the actual number, it's clearly acceptable to us because blocking irresponsible networks is standard practice. We depend on it.
> What overwhelming amounts of spa
Again, 80-90% of all mail is spam, costing billions. If you're able to run a mail system without blacklists that's great for you, but it clearly doesn't work for everyone.
> You don't get to decide what's acceptable to everyone else.
That's the beauty of the internet. I don't have the power to force an abusive network to do their job and prevent spam from leaving their network and that abusive network can't force me to accept mail from them. No one can force anyone to do anything. All we have is a loose set of standards and expectations and it's up to each network to decide what to accept or not based on how well those standards and expectations are followed.
> Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them.
A responsible ISP identifies those users and prevents them continuing to cause problems. If they refuse to do that their reputation suffers and they will get blocked. If they do their job too slowly or too poorly they will be blocked. Is it possible for a responsible ISP to end up on blacklists? Yes, it is, and there are blacklists that don't maintain their lists well. That's fine too because no one is forced to use them. It's still the case that every network has the choice of what blacklists they will or won't use and how they use them. They can whitelist blacklisted IPs they decide to trust and they can use blacklists to greylist instead of block.
> The kind of policy you advocate punishes small ISPs just for being ISPs
Nope. Even small very ISPs can staff their internet abuse departments adequately and implement anti-spam technologies to prevent their IP space from becoming a safe haven for hackers and spammers. If they choose not to do that they will and should be blocked.
> I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
I'll agree that there are problems when certain services (either cloud providers or mail providers like Gmail) become "too big to blacklist". We've had that problem with AOL and we have it now with Google. Personally, I'd prefer to hold them to the same standards as everyone else, but the problem of the largest players throwing their weight around giving them unfair advantages exists in every industry and until someone comes up with a solution for it, we're all just stuck playing along.
> Block actual spam sources
If your ISP is a safe haven for spammers and hackers their IP space is the spam source.
> provide a reasonable method for removing blocks that are no longer necessary.
So your alternative to blacklists is just more blacklists that are run better? I think everyone who depends on blacklists would like those blacklists to be better at detecting spam sources and better at clearing unnecessary listings. The good news is that badly run blacklists don't tend to get widely adopted because they cause more trouble for ISPs than they are worth.
If some network won't accept your mail and you're convinced that your ISP is acting responsibility and that it's the blacklist that's wrong, you can have the person you're trying to reach contact their ISP to get your mail server whitelisted. If an ISP sees that a blacklist they use is catching too many messages that it shouldn't they'll adjust their thresholds or stop using that list.
It's not a perfect system, but it's the best one we have.
Nearly 85% of all emails are spam. source: https://dataprot.net/statistics/spam-statistics/
At times that number has been even higher with over 90% of all messages sent over the internet being spam. If 90% of all the messages in your inbox were spam how long would you continue to use it? Email systems can't bear the costs that spam forces on them. Even with tools like blacklisting which you think shouldn't exist that cost is measured in tens of billions annually. source: https://www.aeaweb.org/articles?id=10.1257/jep.26.3.87
If not for the ability to filter common sources of spam, email would never have survived as a viable means of communication.
> It's way more than 0.001%. Like, several orders of magnitude more.
Whatever the actual number, it's clearly acceptable to us because blocking irresponsible networks is standard practice. We depend on it.
> What overwhelming amounts of spa
Again, 80-90% of all mail is spam, costing billions. If you're able to run a mail system without blacklists that's great for you, but it clearly doesn't work for everyone.
> You don't get to decide what's acceptable to everyone else.
That's the beauty of the internet. I don't have the power to force an abusive network to do their job and prevent spam from leaving their network and that abusive network can't force me to accept mail from them. No one can force anyone to do anything. All we have is a loose set of standards and expectations and it's up to each network to decide what to accept or not based on how well those standards and expectations are followed.
> Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them.
A responsible ISP identifies those users and prevents them continuing to cause problems. If they refuse to do that their reputation suffers and they will get blocked. If they do their job too slowly or too poorly they will be blocked. Is it possible for a responsible ISP to end up on blacklists? Yes, it is, and there are blacklists that don't maintain their lists well. That's fine too because no one is forced to use them. It's still the case that every network has the choice of what blacklists they will or won't use and how they use them. They can whitelist blacklisted IPs they decide to trust and they can use blacklists to greylist instead of block.
> The kind of policy you advocate punishes small ISPs just for being ISPs
Nope. Even small very ISPs can staff their internet abuse departments adequately and implement anti-spam technologies to prevent their IP space from becoming a safe haven for hackers and spammers. If they choose not to do that they will and should be blocked.
> I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
I'll agree that there are problems when certain services (either cloud providers or mail providers like Gmail) become "too big to blacklist". We've had that problem with AOL and we have it now with Google. Personally, I'd prefer to hold them to the same standards as everyone else, but the problem of the largest players throwing their weight around giving them unfair advantages exists in every industry and until someone comes up with a solution for it, we're all just stuck playing along.
> Block actual spam sources
If your ISP is a safe haven for spammers and hackers their IP space is the spam source.
> provide a reasonable method for removing blocks that are no longer necessary.
So your alternative to blacklists is just more blacklists that are run better? I think everyone who depends on blacklists would like those blacklists to be better at detecting spam sources and better at clearing unnecessary listings. The good news is that badly run blacklists don't tend to get widely adopted because they cause more trouble for ISPs than they are worth.
If some network won't accept your mail and you're convinced that your ISP is acting responsibility and that it's the blacklist that's wrong, you can have the person you're trying to reach contact their ISP to get your mail server whitelisted. If an ISP sees that a blacklist they use is catching too many messages that it shouldn't they'll adjust their thresholds or stop using that list.
It's not a perfect system, but it's the best one we have.
> The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness.
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive. It's to incentivize the sending MSP to remove their spammer (or move them to address-space where they can be blocked without causing collateral damage).
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive. It's to incentivize the sending MSP to remove their spammer (or move them to address-space where they can be blocked without causing collateral damage).
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive.
Unfortunately the people it punishes are the legitimate users of both systems who only wanted the system to do its job and let them communicate. The bad actors will just move on and abuse another system instead.
Unfortunately the people it punishes are the legitimate users of both systems who only wanted the system to do its job and let them communicate. The bad actors will just move on and abuse another system instead.
> who only wanted the system to do its job and let them communicate
"The system" you are referring to consists of a bunch of private networks. Your opinion about what the job of those networks is may not coincide with the opinions of the operators of those networks.
Email is not a public service, and there is no entitlement to send whatever "vital business communications" you like to anyone you want. It's not even reasonable to require a postmaster to state what their rejection policy is; that would just tell spammers what they have to do to evade your blocks.
If email doesn't work for your business, then switch to another channel, such as huge billboards or whatever. Ranting about blocklists isn't going to help, people have been doing that for two decades.
I take it you've never run a mailserver?
"The system" you are referring to consists of a bunch of private networks. Your opinion about what the job of those networks is may not coincide with the opinions of the operators of those networks.
Email is not a public service, and there is no entitlement to send whatever "vital business communications" you like to anyone you want. It's not even reasonable to require a postmaster to state what their rejection policy is; that would just tell spammers what they have to do to evade your blocks.
If email doesn't work for your business, then switch to another channel, such as huge billboards or whatever. Ranting about blocklists isn't going to help, people have been doing that for two decades.
I take it you've never run a mailserver?
Could you get a lawyer to draft you one template that looks scary while also not leading to much follow up unless you really want to be whitelisted by that particular entity?
A decent company and big enough already has at least one lawyer on payroll, so no need to be billed the additional 500$/hour
> Get your legal department involved.
Excellent idea, thank you.
Excellent idea, thank you.
Too late to edit my original post, but I want to address a few questions that were brought up by several people.
1. I don't know how the lawyers worded the letters. Obviously it's not illegal for a host to block inbound mail. But clearly the legal team made someone look more closely at our particular case. If you're not doing anything nefarious, it's easy for someone mid-level to clear you off a blacklist if they want to. The company I work for isn't a household name outside a few states, but it's large and respectable enough that getting a cursory glance from someone with the power to say "take them off the blacklist" is probably enough.
2. We've had these IPs for > 5 years. Our reverse DNS, DKIM, SPF are all in order. The mail comes from the same IPs as the company's main website/app services. The company uses Constant Contact for marketing emails, so our outbound server only sends things like receipts for purchases, service agreements, confirmations of renewal/cancellation, verification codes for login (password reset / new signup / new device), and intra-company mail which sometimes goes to employees' personal addresses. Being able to firmly state that we never send marketing off this server, and we regularly assess exim logs and we are sure we've never been hijacked has usually gotten good results, even without lawyers.
3. When we first got these IPs, around 2015, there were more problems. I don't know who they belonged to before. For the first year it was whack-a-mole with the blacklists. But truthfully, I've never had a problem personally getting us taken off any of the public blacklists shown on mxtoolbox. It's the private blacklists of mail providers like AT&T, Comcast and AOL that caused repeated issues. There's no public way to check if you're on these, and as far as I can tell their removal forms are black holes. The removal form for AT&T looks like it hasn't been updated since the '90s, and I've never received even an automated response from using it.
4. We did, for awhile, advise customers to get other email addresses. Unfortunately, the customers with an @comcast or @att account are also probably the least tech-savvy. It's just not feasible to have customers calling their local retail outlet and having a manager spend an hour trying to walk them through why they didn't get their signup verification code, and three days later this bubbles up to corporate and finally to me. And it isn't a good look for us as a company to say "sorry, Comcast doesn't like us for some reason."
5. For awhile we had customers write to Comcast and AT&T asking why they were bouncing important emails from us. Overall this had basically no effect. The customers would get back automated emails telling them how to configure Outlook. However, one such customer was a lawyer who I ended up corresponding with personally (through my gmail account, since he couldn't get my emails), who got such a bug up his butt about it that he seemed to get AT&T to unblock us, which only lasted a month or so. This was where I got the idea to approach the CEO about unleashing the legal department.
6. Variations of "Don't waste $500/hr. on a lawyer". IMHO lawyers are pretty terrible to have against you, and pretty awesome to have on your side. I'm glad I work for a company whose philosophy is to keep as much of our infrastructure as possible in-house, and which also refuses to pony up something that feels like a ransom, even if it would be more expedient. I personally don't have $500/hr. to pay a lawyer, and it would be a much worse uphill battle for a sole proprietor or small company these days trying to manage their own email server than it was ten years ago, or is now even with the backing of a larger company. But it's important to us that we don't get pushed around by big providers, and retain as much autonomy as possible in our business dealings with customers. That's part of the philosophy I brought to the company, it's why we run dedicated hardware, and I'm the one who pushed to spend the money to have the lawyers get involved. If more mid-sized companies took this approach, and if other IT leads pushed for running their own metal and their own mailservers (granted, yeah, it's an endless pain in the ass), then big email providers would be forced to actually take things on a case by case basis the way they did ten years ago, rather than block blacklisting whole /24s with the assumption that every company and individual will eventually cave and be forced into their monopoly. So to whoever says that decline is inevitable, save your money, lawyers suck, etc, without offering any positive solutions: you're welcome.
1. I don't know how the lawyers worded the letters. Obviously it's not illegal for a host to block inbound mail. But clearly the legal team made someone look more closely at our particular case. If you're not doing anything nefarious, it's easy for someone mid-level to clear you off a blacklist if they want to. The company I work for isn't a household name outside a few states, but it's large and respectable enough that getting a cursory glance from someone with the power to say "take them off the blacklist" is probably enough.
2. We've had these IPs for > 5 years. Our reverse DNS, DKIM, SPF are all in order. The mail comes from the same IPs as the company's main website/app services. The company uses Constant Contact for marketing emails, so our outbound server only sends things like receipts for purchases, service agreements, confirmations of renewal/cancellation, verification codes for login (password reset / new signup / new device), and intra-company mail which sometimes goes to employees' personal addresses. Being able to firmly state that we never send marketing off this server, and we regularly assess exim logs and we are sure we've never been hijacked has usually gotten good results, even without lawyers.
3. When we first got these IPs, around 2015, there were more problems. I don't know who they belonged to before. For the first year it was whack-a-mole with the blacklists. But truthfully, I've never had a problem personally getting us taken off any of the public blacklists shown on mxtoolbox. It's the private blacklists of mail providers like AT&T, Comcast and AOL that caused repeated issues. There's no public way to check if you're on these, and as far as I can tell their removal forms are black holes. The removal form for AT&T looks like it hasn't been updated since the '90s, and I've never received even an automated response from using it.
4. We did, for awhile, advise customers to get other email addresses. Unfortunately, the customers with an @comcast or @att account are also probably the least tech-savvy. It's just not feasible to have customers calling their local retail outlet and having a manager spend an hour trying to walk them through why they didn't get their signup verification code, and three days later this bubbles up to corporate and finally to me. And it isn't a good look for us as a company to say "sorry, Comcast doesn't like us for some reason."
5. For awhile we had customers write to Comcast and AT&T asking why they were bouncing important emails from us. Overall this had basically no effect. The customers would get back automated emails telling them how to configure Outlook. However, one such customer was a lawyer who I ended up corresponding with personally (through my gmail account, since he couldn't get my emails), who got such a bug up his butt about it that he seemed to get AT&T to unblock us, which only lasted a month or so. This was where I got the idea to approach the CEO about unleashing the legal department.
6. Variations of "Don't waste $500/hr. on a lawyer". IMHO lawyers are pretty terrible to have against you, and pretty awesome to have on your side. I'm glad I work for a company whose philosophy is to keep as much of our infrastructure as possible in-house, and which also refuses to pony up something that feels like a ransom, even if it would be more expedient. I personally don't have $500/hr. to pay a lawyer, and it would be a much worse uphill battle for a sole proprietor or small company these days trying to manage their own email server than it was ten years ago, or is now even with the backing of a larger company. But it's important to us that we don't get pushed around by big providers, and retain as much autonomy as possible in our business dealings with customers. That's part of the philosophy I brought to the company, it's why we run dedicated hardware, and I'm the one who pushed to spend the money to have the lawyers get involved. If more mid-sized companies took this approach, and if other IT leads pushed for running their own metal and their own mailservers (granted, yeah, it's an endless pain in the ass), then big email providers would be forced to actually take things on a case by case basis the way they did ten years ago, rather than block blacklisting whole /24s with the assumption that every company and individual will eventually cave and be forced into their monopoly. So to whoever says that decline is inevitable, save your money, lawyers suck, etc, without offering any positive solutions: you're welcome.
Classic example of regulatory burden in action. Any firm small enough to not have a legal team can't compete.
Edit; to be clear I'm not saying this results from some legislation, although you could make that case. Just that scale has many benefits and the principle of regulatory burden obtains!
Edit; to be clear I'm not saying this results from some legislation, although you could make that case. Just that scale has many benefits and the principle of regulatory burden obtains!
Yes, but - if more small-ish companies did this, it could have an outsized impact. The company I work for is to AT&T what like an amoeba is to a whale; not much bigger than a single phytoplankton (e.g. an end user).
While I am happy for you that you have found a solution, the solution you found is symptomatic of a very dangerous situation: it is increasingly impossible for individuals or SMEs to use essential online facilities like sending messages or transferring money reliably unless they use a broker service as an intermediary. We are allowing small numbers of tech firms to take control of vital functionality that should be using open, standardised protocols in a decentralised way. This leaves everyone who isn't big enough to run their own implementation that others can't afford to ignore beholden to those brokers and subject to arbitrary charges and/or denial of service.
This is just the internet moving to match the real world. In the real world, reputation matters and some people don't want to talk to you unless someone can vouch for you. For areas where the general public needs to interact, third party intermediary services spring up to fill this need.
This is why for any store over the size of a mom and pop operation in a neighborhood, you can't just tell the owner who you know by name to put it on your account, and instead for credit purchases you use a credit card company which acts as an intermediary and smooths problems over on both sides, and refuses to work with stores and people that are untrustworthy.
This is why there are mailing list (mass email) services and why mail servers allow them. They keep their customers working within the accepted bounds (they ensure removal works and fire clients that abuse), and this allows mass email for accepted reasons while still being able to come down hard in random exploited servers/accounts.
This is why big email services are very selective about what servers they talk to. I work at an ISP where our main outbound mail servers are on IPs that we try not to change because they've got decades of reputation attached. Even so, we recently brought up two new servers for email forwards, and shifted a small percentage of our mail queue traffic to them and ramped it up over a couple weeks, and that seemed to work "warming them up" to the likes of Gmail and yahoo, etc. It used to be there were lists of mail operators you could be part of and you could use reputation within that to get them to be lenient with you when you started. These days it's all so centralized in a few very large players that they really likely just talk to each other.
This is why for any store over the size of a mom and pop operation in a neighborhood, you can't just tell the owner who you know by name to put it on your account, and instead for credit purchases you use a credit card company which acts as an intermediary and smooths problems over on both sides, and refuses to work with stores and people that are untrustworthy.
This is why there are mailing list (mass email) services and why mail servers allow them. They keep their customers working within the accepted bounds (they ensure removal works and fire clients that abuse), and this allows mass email for accepted reasons while still being able to come down hard in random exploited servers/accounts.
This is why big email services are very selective about what servers they talk to. I work at an ISP where our main outbound mail servers are on IPs that we try not to change because they've got decades of reputation attached. Even so, we recently brought up two new servers for email forwards, and shifted a small percentage of our mail queue traffic to them and ramped it up over a couple weeks, and that seemed to work "warming them up" to the likes of Gmail and yahoo, etc. It used to be there were lists of mail operators you could be part of and you could use reputation within that to get them to be lenient with you when you started. These days it's all so centralized in a few very large players that they really likely just talk to each other.
In the real world, reputation matters and some people don't want to talk to you unless someone can vouch for you.
This has absolutely nothing to do with someone not wanting to talk to someone else. It has everything to do with some third party having the power to decide whether the other two may communicate.
These days it's all so centralized in a few very large players that they really likely just talk to each other.
And thus the single most important method of remote communication in the world today, the method that is frequently akin to root access to our online lives, became subject to arbitrary monitoring and interference by huge, powerful organisations with their own interests and negligible regulatory oversight, legal safeguards or accountability to anyone but their shareholders.
Do you really not see why this is a problem? You talk about the internet matching the real world, but in the real world we've had laws against monitoring and interference with things like postal mail and telephone calls for a very long time almost everywhere.
This has absolutely nothing to do with someone not wanting to talk to someone else. It has everything to do with some third party having the power to decide whether the other two may communicate.
These days it's all so centralized in a few very large players that they really likely just talk to each other.
And thus the single most important method of remote communication in the world today, the method that is frequently akin to root access to our online lives, became subject to arbitrary monitoring and interference by huge, powerful organisations with their own interests and negligible regulatory oversight, legal safeguards or accountability to anyone but their shareholders.
Do you really not see why this is a problem? You talk about the internet matching the real world, but in the real world we've had laws against monitoring and interference with things like postal mail and telephone calls for a very long time almost everywhere.
> This has absolutely nothing to do with someone not wanting to talk to someone else. It has everything to do with some third party having the power to decide whether the other two may communicate.
They have that power specifically people people outsource vetting of whether someone's worth dealing with. I'll repeat, this is all about reputation, and how without reputation you're subject to every anonymous person's abuse. That's not an issue when the total number of people you're dealing with is manageable. It is when the number of people you expect to have to deal with is the population of a small country or the entire world.
> And thus the single most important method of remote communication in the world today, the method that is frequently akin to root access to our online lives, became subject to arbitrary monitoring and interference by huge, powerful organisations with their own interests and negligible regulatory oversight, legal safeguards or accountability to anyone but their shareholders.
Only for those that opted into that system. Gmail does not dictate what mail servers accept your mail, they dictate whether they themselves accept your mail. They just happen to have a userbase in hundreds of millions billions, so a large percent of the people you might want to contact use it. You can blame Gmail all you want, but they're just doing what their users want, which is reliable email without much spam, and that's how they've achieved it. You're not going to get far telling people when you send stuff to them they're required to accept it. The system only worked the way you wanted it when it was small and offenders weren't as anonymous and social punishments worked against them. When everyone's mostly anonymous, that no longer works.
> Do you really not see why this is a problem?
It is a problem, but it's a problem of people choosing to use them. And there's been ways to keep people from reading your emails for decades at this point with GPG, and if you can't trust the other side to buy into that, or to not use a web based email client, then there's nothing you can do about communication with those people anyway.
Should they be monitoring email? No. Would we be better with laws preventing that? Yes. Is that really the same issue as them being so large that by dictating who they will talk with and how they are effectively dictating rules for running a public mail server? No. Separate issues.
They have that power specifically people people outsource vetting of whether someone's worth dealing with. I'll repeat, this is all about reputation, and how without reputation you're subject to every anonymous person's abuse. That's not an issue when the total number of people you're dealing with is manageable. It is when the number of people you expect to have to deal with is the population of a small country or the entire world.
> And thus the single most important method of remote communication in the world today, the method that is frequently akin to root access to our online lives, became subject to arbitrary monitoring and interference by huge, powerful organisations with their own interests and negligible regulatory oversight, legal safeguards or accountability to anyone but their shareholders.
Only for those that opted into that system. Gmail does not dictate what mail servers accept your mail, they dictate whether they themselves accept your mail. They just happen to have a userbase in hundreds of millions billions, so a large percent of the people you might want to contact use it. You can blame Gmail all you want, but they're just doing what their users want, which is reliable email without much spam, and that's how they've achieved it. You're not going to get far telling people when you send stuff to them they're required to accept it. The system only worked the way you wanted it when it was small and offenders weren't as anonymous and social punishments worked against them. When everyone's mostly anonymous, that no longer works.
> Do you really not see why this is a problem?
It is a problem, but it's a problem of people choosing to use them. And there's been ways to keep people from reading your emails for decades at this point with GPG, and if you can't trust the other side to buy into that, or to not use a web based email client, then there's nothing you can do about communication with those people anyway.
Should they be monitoring email? No. Would we be better with laws preventing that? Yes. Is that really the same issue as them being so large that by dictating who they will talk with and how they are effectively dictating rules for running a public mail server? No. Separate issues.
> Only for those that opted into that system. (...) You can blame Gmail all you want, but they're just doing what their users want, which is reliable email without much spam
That's not true. When you use Gmail you don't opt-in to a setting to "automatically send smaller non-commercial providers to the SPAM folder". Gmail is pretty bad at SPAM handling you still receive lots of commercial advertisement from "reputable" sources, you just don't see the non-scammy non-phishy email you were expecting.
In this specific instance, Gmail is not the worst out there. They're very protective but from what i heard you can get allowlisted in a matter of weeks of harassing their tech support. Meanwhile, Microsoft is a lost cause if you want to get your mail through to an Outlook mailbox...
That's not true. When you use Gmail you don't opt-in to a setting to "automatically send smaller non-commercial providers to the SPAM folder". Gmail is pretty bad at SPAM handling you still receive lots of commercial advertisement from "reputable" sources, you just don't see the non-scammy non-phishy email you were expecting.
In this specific instance, Gmail is not the worst out there. They're very protective but from what i heard you can get allowlisted in a matter of weeks of harassing their tech support. Meanwhile, Microsoft is a lost cause if you want to get your mail through to an Outlook mailbox...
They have that power specifically people people outsource vetting of whether someone's worth dealing with.
People outsource that vetting? What choice did those people have?
If there is a small cabal of tech giants dominating email distribution, and a user's choices are between some of those giants that are essentially as bad as each other and some smaller competitors that might be better in terms of delivering incoming mail reliably but are frozen out by the cabal in terms of delivering outgoing mail reliably instead, there is no meaningful choice or consent about having this filtering done to their incoming mail. It is a situation imposed upon them by the tech firms.
People outsource that vetting? What choice did those people have?
If there is a small cabal of tech giants dominating email distribution, and a user's choices are between some of those giants that are essentially as bad as each other and some smaller competitors that might be better in terms of delivering incoming mail reliably but are frozen out by the cabal in terms of delivering outgoing mail reliably instead, there is no meaningful choice or consent about having this filtering done to their incoming mail. It is a situation imposed upon them by the tech firms.
>This is why for any store over the size of a mom and pop operation in a neighborhood, you can't just tell the owner who you know by name to put it on your account, and instead for credit purchases you use a credit card company which acts as an intermediary and smooths problems over on both sides, and refuses to work with stores and people that are untrustworthy.
Yet, if that store knows you, and wants to extend credit to you, they can. A difficulty with email is that it seems to be increasingly difficult for the recipients to have much control over the email they receive. I know of numerous instances where people are unable to get specific emails they want or need to receive to not be spam-foldered, or to be received at all. They may have no understanding of why this is the case, and even if they do, may have no ability to change it. The systems are often opaque and unchangeable, and involve organization- or provider-level choices before emails ever reach their accounts.
Yet, if that store knows you, and wants to extend credit to you, they can. A difficulty with email is that it seems to be increasingly difficult for the recipients to have much control over the email they receive. I know of numerous instances where people are unable to get specific emails they want or need to receive to not be spam-foldered, or to be received at all. They may have no understanding of why this is the case, and even if they do, may have no ability to change it. The systems are often opaque and unchangeable, and involve organization- or provider-level choices before emails ever reach their accounts.
> and that seemed to work "warming them up" to the likes of Gmail and yahoo, etc
Yes, warming up a "fresh" IP definely works. If you suddenly send thousands of mails from a new server - sure, you'll be labeled as Spam. If you slowly increase the volume over time, have a good domain and recipients that interact with the email, things should be fine. GMail has quite helpful guidelines [1].
1: https://support.google.com/mail/answer/81126?hl=en
Yes, warming up a "fresh" IP definely works. If you suddenly send thousands of mails from a new server - sure, you'll be labeled as Spam. If you slowly increase the volume over time, have a good domain and recipients that interact with the email, things should be fine. GMail has quite helpful guidelines [1].
1: https://support.google.com/mail/answer/81126?hl=en
[deleted]
I agree completely with this, but the one problem is that you haven’t addressed how we control spam without these “trusted” intermediaries. “Trusted” here meaning that they aren’t spammers.
Spam has largely been a solved problem for decades IME. You don't need some big-data-crunching mega-mail-host to block it successfully. For my personal mail, I use a small provider that isn't configured to block anything automatically and the built-in tools in my mail software. For my businesses, we have a pretty standard SpamAssassin-style setup. Either way, I see hardly any spam in my inbox despite receiving mail to multiple published contact addresses for those businesses, and I also can't remember the last time a false positive resulted in missing a legitimate mail.
Meanwhile I've seen people miss events because $BIG_MAIL_PROVIDER decided the invitation was spam, I've seen recruitment go wrong because a CV from an excellent candidate was blocked on its way to the designated email address for applications, and countless other examples where bad spam blocking was throwing baby out with bathwater.
Meanwhile I've seen people miss events because $BIG_MAIL_PROVIDER decided the invitation was spam, I've seen recruitment go wrong because a CV from an excellent candidate was blocked on its way to the designated email address for applications, and countless other examples where bad spam blocking was throwing baby out with bathwater.
You must be living in a different world than me. I regularly get spam even from large corporations, that I know I never signed up for anything for. I know it is actually them, because it's DKIM signed with their domain certificates. Walmart (which doesn't exist in my country), Unilever and tons or their brands.
Even a national division of Microsoft got hands on my email and decided to sign me up for invites to developer events (I know I hadn't signed up for it, because they didn't even use my name in the mails, which they certainly would have known if they had gotten it through any legitimate source).
I assume it's bullshit KPIs, or signup schemes. Some poor seller at Microsoft or Walmart who's ranked according to how many people sign up to the opt-out company promotions, and buys harvested addresses to juice their numbers.
I even got some very obvious fraud spam ("Your [expensive product] is waiting for you!", trying to get you to sign up as a "product tester" only paying $$$ per month) with full DKIM signatures from a site hosted at a Danish hosting provider, and Danish business info. You'd think this sort of thing gets shut down quickly, but nope. The only times I've tried to address this sort of thing through the proper channels (domain registrar's abuse accounts, etc.), I've only gotten markedly more spam.
At least DKIM signatures makes it easy to autodelete mail. No need to even go into the spam folder, I'm happy to forbid Walmart from communicating with me by mail forever.
Even a national division of Microsoft got hands on my email and decided to sign me up for invites to developer events (I know I hadn't signed up for it, because they didn't even use my name in the mails, which they certainly would have known if they had gotten it through any legitimate source).
I assume it's bullshit KPIs, or signup schemes. Some poor seller at Microsoft or Walmart who's ranked according to how many people sign up to the opt-out company promotions, and buys harvested addresses to juice their numbers.
I even got some very obvious fraud spam ("Your [expensive product] is waiting for you!", trying to get you to sign up as a "product tester" only paying $$$ per month) with full DKIM signatures from a site hosted at a Danish hosting provider, and Danish business info. You'd think this sort of thing gets shut down quickly, but nope. The only times I've tried to address this sort of thing through the proper channels (domain registrar's abuse accounts, etc.), I've only gotten markedly more spam.
At least DKIM signatures makes it easy to autodelete mail. No need to even go into the spam folder, I'm happy to forbid Walmart from communicating with me by mail forever.
Apparently we do live in different worlds. May I ask what sort of email infrastructure you use? I don't think anything I use, either personally or professionally, is particularly complicated or unusual but apparently our recent experiences have been very different.
Probably 95% of the spam I receive is automatically filtered to a spam folder, with a negligible false positive rate. Every now and then some new pattern does make it through but once I've flagged a few instances the rest start getting classified as spam automatically.
Probably 95% of the spam I receive is automatically filtered to a spam folder, with a negligible false positive rate. Every now and then some new pattern does make it through but once I've flagged a few instances the rest start getting classified as spam automatically.
I have a gmail account and a RoundCube thing at my own domain (managed by the domain name provider).
Gmail's spam filtering is very generous to DKIM-signed mail - it has a tendency to let it trough even if I've flagged exactly that sender before (as I did with walmart). The one on my own domain has received too little spam to tell how good the filter is.
Gmail's spam filtering is very generous to DKIM-signed mail - it has a tendency to let it trough even if I've flagged exactly that sender before (as I did with walmart). The one on my own domain has received too little spam to tell how good the filter is.
Sure for you, but all of these other mailservers still obviously find value in applying spam filters or they wouldn't keep filtering.
The problem is that it's much easier to send email than it is to receive it. This puts the onus of spam filtering on the recipient. I'm sad that HashCash or some other PoW scheme was never adopted as a way to force rate limiting of mailers.
The problem is that it's much easier to send email than it is to receive it. This puts the onus of spam filtering on the recipient. I'm sad that HashCash or some other PoW scheme was never adopted as a way to force rate limiting of mailers.
Sure for you, but all of these other mailservers still obviously find value in applying spam filters or they wouldn't keep filtering.
But since they have no accountability, their incentive is to eliminate anything potentially hostile regardless of the collateral damage from false positives. What are their users going to do? Take their business to another big provider that will probably be doing the same, and in the process lose access to a bunch of online services they like because their accounts on those services are tied to the old address? Move to a small provider, which is being forced out of the network by the giants so can't deliver mail reliably?
We are rapidly heading for a world where email is no longer a reliable communications medium for genuine messages between friends, family, work colleagues, etc. The giants are killing email through death by 1,000 cuts instead of a shotgun to the face but the end result is the same.
But since they have no accountability, their incentive is to eliminate anything potentially hostile regardless of the collateral damage from false positives. What are their users going to do? Take their business to another big provider that will probably be doing the same, and in the process lose access to a bunch of online services they like because their accounts on those services are tied to the old address? Move to a small provider, which is being forced out of the network by the giants so can't deliver mail reliably?
We are rapidly heading for a world where email is no longer a reliable communications medium for genuine messages between friends, family, work colleagues, etc. The giants are killing email through death by 1,000 cuts instead of a shotgun to the face but the end result is the same.
The open internet is dead. Spammers, fraudsters, and abusers have killed it. I can see how you might hate me, but I only speak the truth. Once upon a time the Internet consisted of a bunch of anonymous university students and u.s. government, Bell, IBM, etc. employees doing unfettered research. Today the Internet consists of anonymous grifters trying to squeeze a buck out of you any way they can.
The open internet is dead. Spammers, fraudsters, and abusers have killed it.
I don't accept that. Nothing about defending against those threats requires the kind of guilt-by-association spam blocking we have been discussing.
There will always be bad people in the world but that doesn't mean we allow some random big business to tell us where we may go, what we may do or who we may talk to because it is in that business's interests and (coincidentally or otherwise) it might make us safer.
I don't accept that. Nothing about defending against those threats requires the kind of guilt-by-association spam blocking we have been discussing.
There will always be bad people in the world but that doesn't mean we allow some random big business to tell us where we may go, what we may do or who we may talk to because it is in that business's interests and (coincidentally or otherwise) it might make us safer.
[deleted]
I hope to see bitmessage supplant email for many use cases.
It is decentralized and has built-in mechanisms to fight spam. Also all messages are end-to-end encrypted.
Despite the name it has nothing to do with bitcoin or cryptocurrency.
https://wiki.bitmessage.org/
It is decentralized and has built-in mechanisms to fight spam. Also all messages are end-to-end encrypted.
Despite the name it has nothing to do with bitcoin or cryptocurrency.
https://wiki.bitmessage.org/
The flip side is that before, when the rules were lax and everybody could send email, everyone would get at least 10 spam mails every day, more if your address was online somewhere. Where now most people get one a week.
This is the real value of cryptocurrencies. Yes, I know HN doesn't like them, yes there's a bunch of get-rich-quick bros and scammers out there, please try and separate the grift from the tech and consider how vital it is that people are able to control their finances without a third party having the ultimate say as to whether a transaction takes place or not.
The only currently realistic way to acquire cryptocurrency or for non-tech people to use it is through a 3rd party broker. It's about as difficult as running your own SMTP server I'd say.
edit see this current front-page submission about how Bitcoin fails to provide this despite being centralized in exactly the same way as the decentralized SMTP: https://news.ycombinator.com/item?id=30224637
edit see this current front-page submission about how Bitcoin fails to provide this despite being centralized in exactly the same way as the decentralized SMTP: https://news.ycombinator.com/item?id=30224637
> for non-tech people to use it
Then it should be only the SMTP server operators who have to handle the crypocurrency side of things, so email users never have to worry about it.
As you say, acquiring and using the cryptocurrency would be about as technically difficult as what the SMTP server operators are already doing, and there are a variety of 3rd party brokers they can choose if they want to simplify things and not run a node themselves.
The system I'm imagining is one where each newly registered domain has to put up a cryptocurrency bond if the registrant wants to send email from it. Existing domains would be grandfathered in (having already built up their reputation) and new domains would have their bonds burned if some N-of-M stakeholders agreed that they were sending (DKIM-signed) spam.
Choosing those stakeholders would be controversial, but hopefully no less controversial than the system we have today where Google can use the threat of a Gmail blacklist to make every SMTP server in the world follow its wishes. Ideally some of the stakeholders would be non-profits like the ISRG and Mozilla Foundation.
Then it should be only the SMTP server operators who have to handle the crypocurrency side of things, so email users never have to worry about it.
As you say, acquiring and using the cryptocurrency would be about as technically difficult as what the SMTP server operators are already doing, and there are a variety of 3rd party brokers they can choose if they want to simplify things and not run a node themselves.
The system I'm imagining is one where each newly registered domain has to put up a cryptocurrency bond if the registrant wants to send email from it. Existing domains would be grandfathered in (having already built up their reputation) and new domains would have their bonds burned if some N-of-M stakeholders agreed that they were sending (DKIM-signed) spam.
Choosing those stakeholders would be controversial, but hopefully no less controversial than the system we have today where Google can use the threat of a Gmail blacklist to make every SMTP server in the world follow its wishes. Ideally some of the stakeholders would be non-profits like the ISRG and Mozilla Foundation.
> The only currently realistic way to acquire cryptocurrency or for non-tech people to use it is through a 3rd party broker.
They could provide goods or services for it the same way people acquire dollars without going through a 3rd party broker.
They could provide goods or services for it the same way people acquire dollars without going through a 3rd party broker.
cryptocurrencies and not screwing up global implementation of SMTP are completely different things.
cryptocurrencies are not in any way going to solve the problem of people centralizing their MX all onto office365 and gsuite.
cryptocurrencies are not in any way going to solve the problem of people centralizing their MX all onto office365 and gsuite.
How do you know whether your mail is going through? I can understand for messages that bounce, but how about mails that are silently dropped or end up in spam folders?
In our case it's quickly obvious since our server sends verification codes when people sign up for online accounts or change their passwords. We'll see a pattern of customer complaints from certain mail services within a day of being on any major blacklist.
Honestly, we usually find out after communication has failed and some other form of communication is used.
It's usually worthwhile to remind businesses that when they use "free" services like privacy-paid outlook.com and Gmail, they'll get what they pay for, and if their communications really matter, they should find proper email providers.
It's usually worthwhile to remind businesses that when they use "free" services like privacy-paid outlook.com and Gmail, they'll get what they pay for, and if their communications really matter, they should find proper email providers.
I can never know for sure if email is avoiding the spam folder, but everything tells me if it does happen, it's quite rare now.
First, I will send emails to test accounts I have set up with major email providers (gmail, yahoo, outlook, etc). I used to do this often, but they have all been going to the inbox on my last few tests, so I haven't done this recently.
Second, is feedback I get from recipients. When I send an email and ask for a reply, I'll get a reply. I also used to hear a few times a month, "sorry for the delay. I found your email in my spam folder!". This has gone away.
First, I will send emails to test accounts I have set up with major email providers (gmail, yahoo, outlook, etc). I used to do this often, but they have all been going to the inbox on my last few tests, so I haven't done this recently.
Second, is feedback I get from recipients. When I send an email and ask for a reply, I'll get a reply. I also used to hear a few times a month, "sorry for the delay. I found your email in my spam folder!". This has gone away.
I've had a few issues with Mailgun getting blocked over the years. Especially with Hotmail/Outlook.
I did not even know services like Postmark and Mailgun solved this problem. Thank you!
To be clear, Postfix is the Open Source Linux software for sending email. In a perfect world, Postfix would be all you need for sending.
Providers like Postmark and Mailgun let you forward and send email through them. Essentially, you are paying these companies to deal with IP address reputation management and ensure high deliverability.
Providers like Postmark and Mailgun let you forward and send email through them. Essentially, you are paying these companies to deal with IP address reputation management and ensure high deliverability.
[deleted]
> and control my email data
Are you though? Aren't you giving your outbound emails to Postmark or Mailgun in that case?
Are you though? Aren't you giving your outbound emails to Postmark or Mailgun in that case?
Yes, Postmark keeps a 45 day history of emails for user reporting. After that it's deleted:
https://postmarkapp.com/eu-privacy#security-and-privacy
It's a decision I am fine with. Others may not be.
https://postmarkapp.com/eu-privacy#security-and-privacy
It's a decision I am fine with. Others may not be.
I was flabbergasted that relaying their mail through a service that does high volume, and enforces abuse policies, wasn't one of his listed options. As the Internet has become, due to rampant spamming, it's become best to just outsource mail relaying.
> through a transactional SMTP sender, like Postmark or Mailgun.
... cheapest is $10/month. Not paying that for my meager email traffic. Are there other options for personal domains?
... cheapest is $10/month. Not paying that for my meager email traffic. Are there other options for personal domains?
I've been using the free tier at MailJet to relay mail for my server for several years and have had few problems. Several others offer free tiers too (SendinBlue, etc.)
Both of those seem to insert their logo in the emails for the free tier.
I'd rather pay a little and get unmodified emails. Guess they all cater to marketers not personal use.
I'd rather pay a little and get unmodified emails. Guess they all cater to marketers not personal use.
AWS SES
My Dad still sends all emails to my main address and my gmail address, due to the occasional spat between netzero and godaddy that would block emails to my main address.
The funny thing about that work-around is, what stops spammers from doing it either?
A spammer is going to get blocked and then have to send another letter ad infinitum. It's not financially feasible to do that. The respondent should also have evidence available to prove that the spammer is acting in bad faith (if they don't, then maybe they should get some before initiating a block).
I'm dealing with this right now.
Both my personal domain and rsync.net are on a distinct subnet, but that subnet is smaller than a /24 and someone on a different subnet has, apparently, behaved badly.
Enter "abusix" ...
One of my engineers had an enlightening webchat with one of their engineers where we were shown the "offending" IP and it was explained that they have no ability to distinguish subnets (and no interest in doing so). So if you're not wasting an entire /24 (we only need ~10 IPs at this location) you're in danger of this misclassification.
We were also informed that our normal, business communications with paying customers should have unsubscribe notices appended to them. Which is to say, you're a paying customer of a service and we send you some kind of alert or critical announcement ... and it should have an unsubscribe link.
Unbelievable.
Both my personal domain and rsync.net are on a distinct subnet, but that subnet is smaller than a /24 and someone on a different subnet has, apparently, behaved badly.
Enter "abusix" ...
One of my engineers had an enlightening webchat with one of their engineers where we were shown the "offending" IP and it was explained that they have no ability to distinguish subnets (and no interest in doing so). So if you're not wasting an entire /24 (we only need ~10 IPs at this location) you're in danger of this misclassification.
We were also informed that our normal, business communications with paying customers should have unsubscribe notices appended to them. Which is to say, you're a paying customer of a service and we send you some kind of alert or critical announcement ... and it should have an unsubscribe link.
Unbelievable.
Those unsubscribe links should be there, for several reasons.
- The service-based economy means that entities (individuals and businesses) have numerous relationships. For the typical individual the number of password-based accounts crossed the 100 threshold years ago, at a doubling rate of every 2--3 years.
- Responsibilities can be transferred. The person who signed up for your service 5 years ago may no longer be at the company.
- List purging is a Real Thing. A few years back I'd worked for an organisation that had ... numerous relationships ... with individuals and corporations. These received regular email messages. Nominally, requested. Included amongst these was a major Wall Street financial firm whose implosion years earlier hit lead news and headlines worldwide. Despite not existing for years, there remained hundreds if not thousands of addresses being sent email on a regular ongoing basis.
- Mail can be forwarded. It's quite possible that you're sending mail to one address that is is being forwarded, manually or automatically, to others. This raises issues in unsubscribe requests, but might at the least be an opportunity to reach out to your customer to clarify the situation.
- I don't know if revisiting email contact approval on a regular basis (say once every year or two) is yet a recommended practice, but I'd strongly suggest that it be so.
Your hat may be less blisteringly white than you presume.
- The service-based economy means that entities (individuals and businesses) have numerous relationships. For the typical individual the number of password-based accounts crossed the 100 threshold years ago, at a doubling rate of every 2--3 years.
- Responsibilities can be transferred. The person who signed up for your service 5 years ago may no longer be at the company.
- List purging is a Real Thing. A few years back I'd worked for an organisation that had ... numerous relationships ... with individuals and corporations. These received regular email messages. Nominally, requested. Included amongst these was a major Wall Street financial firm whose implosion years earlier hit lead news and headlines worldwide. Despite not existing for years, there remained hundreds if not thousands of addresses being sent email on a regular ongoing basis.
- Mail can be forwarded. It's quite possible that you're sending mail to one address that is is being forwarded, manually or automatically, to others. This raises issues in unsubscribe requests, but might at the least be an opportunity to reach out to your customer to clarify the situation.
- I don't know if revisiting email contact approval on a regular basis (say once every year or two) is yet a recommended practice, but I'd strongly suggest that it be so.
Your hat may be less blisteringly white than you presume.
Everything you've said makes perfect sense - for a contact management function.
We have that. You can change contact info, set owner/technical/emergency contacts, alert thresholds, etc.
But unsubscribe means something totally different:
When I click on unsubscribe I want it to be the end of all communications. Period.
In this case, that makes no sense. Ceasing communications for all purposes implies service cancellation and, in our case, service cancellation implies a human interaction confirming data destruction.
How would we confirm data destruction for your implied cancellation if no further contact is permitted ?
You, and the blacklist operators, have become so jaded by the abuse you've suffered that you've forgotten that legitimate, paid services exist. I'm sorry.
We have that. You can change contact info, set owner/technical/emergency contacts, alert thresholds, etc.
But unsubscribe means something totally different:
When I click on unsubscribe I want it to be the end of all communications. Period.
In this case, that makes no sense. Ceasing communications for all purposes implies service cancellation and, in our case, service cancellation implies a human interaction confirming data destruction.
How would we confirm data destruction for your implied cancellation if no further contact is permitted ?
You, and the blacklist operators, have become so jaded by the abuse you've suffered that you've forgotten that legitimate, paid services exist. I'm sorry.
From a comment below from the other side of the equation it sounds like the email in question WAS indeed marketing for a lifetime promotion.
People should have every right to unsubscribe themselves from that, and thus should have some sort of feedback loop attached to the email being sent (to the detriment of your bottom line I fully understand and sympathize with).
If this was indeed a marketing email, then I don't think it matters if it comes from a "legitimate, paid service" - the receiver still should be able to choose if they want to receive sales related emails wouldn't you agree?
People should have every right to unsubscribe themselves from that, and thus should have some sort of feedback loop attached to the email being sent (to the detriment of your bottom line I fully understand and sympathize with).
If this was indeed a marketing email, then I don't think it matters if it comes from a "legitimate, paid service" - the receiver still should be able to choose if they want to receive sales related emails wouldn't you agree?
"People should have every right to unsubscribe themselves from that, and thus should have some sort of feedback loop attached to the email being sent (to the detriment of your bottom line I fully understand and sympathize with)."
I agree.
We have a flag for such a thing and set that flag when people ask us to. They ask us in a nice email exchange between human beings. We're very responsive to this since they are our paying customers.
That's the big disconnect here: it's inconceivable to many people (including abusix, et. al) that healthy, straightforward interactions like this occur in 2022.
In their mind there are nothing but robots and newsletter subscribers forever locked in an arms race.
I'm sorry that is the case.
I agree.
We have a flag for such a thing and set that flag when people ask us to. They ask us in a nice email exchange between human beings. We're very responsive to this since they are our paying customers.
That's the big disconnect here: it's inconceivable to many people (including abusix, et. al) that healthy, straightforward interactions like this occur in 2022.
In their mind there are nothing but robots and newsletter subscribers forever locked in an arms race.
I'm sorry that is the case.
Did they request this marketing message through a "nice email exchange between humans"? Or did you automatically sign up this person for marketing then expect them to manually contact you?
You are wayyy to smart not too see the abusive asymmetrical theater of that scheme.
You are wayyy to smart not too see the abusive asymmetrical theater of that scheme.
Atleast in my country, to send legitimate marketing e-mails, a user has to be given a choice to opt-out *before* the first e-mail.
So it's not enough to allow them to opt-out with an e-mail exchange or an unsubscribe link, you must allow the user to opt out when you initially gather the contact information.
If that wasn't done, it's illegal (and unethical) to send marketing e-mails, even from a paid service.
So it's not enough to allow them to opt-out with an e-mail exchange or an unsubscribe link, you must allow the user to opt out when you initially gather the contact information.
If that wasn't done, it's illegal (and unethical) to send marketing e-mails, even from a paid service.
Your earlier argument about unsubscribe being effectively an informal termination only works if you properly separate that side from all promotional/marketing/new features/etc material, otherwise it seems you are avoiding the main point and purpose of the unsubscribe button
The flag for marketing etc. should be automated, I understand account-related emails like billing not having "unsubscribe" links but promotions definitely need one. Clicking that would then toggle the flag (and maybe have one in their account panel).
[deleted]
An unsubscribe link doesn't have to immediately cancel all service and communication. It can simply lead to account settings or even to a page explaining how to cancel the account.
You may have missed that bit in my earlier comment about working for a paid service provider.
I strongly disagree. There is absolutely no need to put an unsubscribe link into a transactional email.
All emails should of course contain enough information to make it clear who the message is from, why the message is being sent, and who it was sent to.
But there is no point in adding unsubscribe links to messages and notifications that are essential to the service.
I mean, what are you going to do if the user accidentally clicks "unsubscribe", and then a payment doesn't go through? Should you just cancel their account without informing them? That's absurd.
I'd be really pissed if eg. my backups were deleted because I accidentally unsubscribed from emails from a cloud service provider.
All emails should of course contain enough information to make it clear who the message is from, why the message is being sent, and who it was sent to.
But there is no point in adding unsubscribe links to messages and notifications that are essential to the service.
I mean, what are you going to do if the user accidentally clicks "unsubscribe", and then a payment doesn't go through? Should you just cancel their account without informing them? That's absurd.
I'd be really pissed if eg. my backups were deleted because I accidentally unsubscribed from emails from a cloud service provider.
> There is absolutely no need to put an unsubscribe link into a transactional email.
Agreed. rsync alluded to it below as well.
'unsubscribe'... from what? If I just bought something from service ABC, and I get an email from ABC saying "you just bought foo from us"... what would an 'unsubscribe' even mean? "Do not ever email me about this purchase again?" "Do not ever email me about future purchases?"
Agreed. rsync alluded to it below as well.
'unsubscribe'... from what? If I just bought something from service ABC, and I get an email from ABC saying "you just bought foo from us"... what would an 'unsubscribe' even mean? "Do not ever email me about this purchase again?" "Do not ever email me about future purchases?"
> Do not ever email me about this purchase again
Please send me the order, just don't send me the PDF invoice :)
Please send me the order, just don't send me the PDF invoice :)
This actually does cause problems with blocked password resets and things
If I request a password reset, that constitutes an approval to send me that email.
Some systems might just block all outgoing emails to anyone who has clicked unsubscribe which would block password resets as well
And if I didn't request a password reset ... I'd like to hear about it.
how do you know it's 'you' that requested it? if I requested a password reset to your email... is it spam?
What about situations where your email somehow (mistype) gets set up for someone else's account? I have 2-3 people with similar emails to my previous email address that would mistype and I'd receive their emails. These weren't spam but the companies wouldn't offer _any_ way to fix this.
My recourse is to just flag them as spam in gmail.
My recourse is to just flag them as spam in gmail.
That's why I said the email should contain info about the sender -- there should of course be a way to contact them. Ideally you should just be able to reply to the message and tell them about the error. If there's no way to contact a company, that's a whole different problem, and not really one that would be fixed with unsubscribe links in important email messages.
> That's why I said the email should contain info about the sender -- there should of course be a way to contact them.
That implies a level of manual effort on the part of the recipient that's unreasonable. I have no relationship with these companies. They did not verify the email address before starting to send a stream of supposedly transactional messages to it. They should be happy that I'm willing to click unsubscribe when available, because the alternative is to set up a mark-as-spam filtering rule that'll hopefully tank their sender reputation.
Writing them via a contact form begging to be removed is not an option.
That implies a level of manual effort on the part of the recipient that's unreasonable. I have no relationship with these companies. They did not verify the email address before starting to send a stream of supposedly transactional messages to it. They should be happy that I'm willing to click unsubscribe when available, because the alternative is to set up a mark-as-spam filtering rule that'll hopefully tank their sender reputation.
Writing them via a contact form begging to be removed is not an option.
Yup. And is this situation the recipient will just mark it as spam. Because to them that's what it is.
In 99% of the cases - there is no recourse. In one instance, I tried replying and they asked me to prove my identity as the customer to cancel the emails.
This, I've had people receiving bank alerts for an account they don't own and they can't be stopped. What these companies lack are customer-centric processes that they've thought through.
Wtf is wrong with putting contact information in the unsubscribe link, or reach out productively on request? Why would you presume somebody clicks it by accident vs. the much more likely case of it being a legitimate request? Are you afraid they really want to cancel your service? Or are you afraid you can't send spam under the guise of transactional messages? Or worse, listen to customers about how best to alert them? Truly ridiculous!
Wtf is wrong with putting contact information in the unsubscribe link, or reach out productively on request? Why would you presume somebody clicks it by accident vs. the much more likely case of it being a legitimate request? Are you afraid they really want to cancel your service? Or are you afraid you can't send spam under the guise of transactional messages? Or worse, listen to customers about how best to alert them? Truly ridiculous!
> Wtf is wrong with putting contact information in the unsubscribe link
By law, depending where you are, a unsubscribe link has to be instant. So there can not be an intermediate screen asking for confirmation or showing contact information. Well, you could show contact information, but then the unsubscribe (of in this case service critical mails, thus the service itself) had already happened.
By law, depending where you are, a unsubscribe link has to be instant. So there can not be an intermediate screen asking for confirmation or showing contact information. Well, you could show contact information, but then the unsubscribe (of in this case service critical mails, thus the service itself) had already happened.
I'm assuming you're referring to the CAN-SPAM Act [1] or equivalent in other jurisdiction [2], but nothing of the sort is implied. The requirement is to make available and process opt-out without charge and promptly (typically in so many days).
It is neither dire nor prevents clarification, nor prescribes a specific experience related to unsubscribe links. It's odd to hear the only options are between receiving messages and having service terminated. That sounds pretty user-hostile tbh.
[1] https://www.ftc.gov/tips-advice/business-center/guidance/can...
[2] https://www.lsoft.com/resources/optinlaws.asp
It is neither dire nor prevents clarification, nor prescribes a specific experience related to unsubscribe links. It's odd to hear the only options are between receiving messages and having service terminated. That sounds pretty user-hostile tbh.
[1] https://www.ftc.gov/tips-advice/business-center/guidance/can...
[2] https://www.lsoft.com/resources/optinlaws.asp
Oh, interesting! https://www.rapidmail.de/blog/welche-vorgaben-gelten-bei-der... claims the same for Germany. That makes it a bit easier to find a solution.
I have a few hundred dollars worth of gift cards for an Australian store received as gifts over several years.
The company won’t talk to me, and the sender sends a lovely message, but no contact information.
The company won’t talk to me, and the sender sends a lovely message, but no contact information.
You're expecting that a company incompetent enough to attach unverified email addresses to an account to "correctly" deal with unsubscribing from transactional email? This seems entirely futile and counter-productive to me. (Correctly in scare quotes because I can't fathom what a correct automated unsubscribe would look like in this situation.)
Emails should be confirmed before being used for ongoing communication. Simple as that. It’s easier to get right up front than it is to clutter and confuse in the cases already illustrated.
Sure - but these companies aren't doing that. They're sending emails without any means of preventing it, e.g. spam.
You can create a Gmail filter to delete or archive them automatically and avoid poisoning the spam filter.
If I get continuously get emails sent to me that I did not request and I can't unsubscribe to, then it's spam. Maybe companies should make sure they're not sending emails to the wrong person, because I'm just going to keep marking it as spam when it comes my way.
Exactly. "Poisoning" the spam filter is nothing more than sending an alert back to upstream to deal with it when and where it starts to hurt business. Typically it's the only feedback mechanism taken seriously and thus a service to the community.
But...it is spam? If they don't give the tools necessary to stop the spam (unsubscribe or a link to "received this by mistake") then it's spam - intentional or not.
It's not on me to do their job for them. It costs time and hence money.
Yeah, I run my own email server and have the same issues with the same services as the person who wrote the piece this links to.
In my case users are sending estimates and invoices and monthly statements to their clients and while fake invoices may be a spammer thing those clients know who's sending them an invoice, and why and what for, so an 'Unsubscribe" link would be completely out of context because they are not subscribed to any email list.
I've had the same domain name for over 20 years now and none of my users have ever used my apps to send spam. And as spam and email volume go my server isn't even close to sending out a lot of email.
When I set up a new email server last year, with a new IP address, I had to go through the process of getting white listed. All the big email service providers have ways to do that. Google made it very easy. They gave you a unique string to add it to your DNS records and that's it. Microsoft is so convoluted I've still not gotten anywhere with them. Comcast and others had a few hoops and ladders but nothing that got me stuck.
Personally, while it's a bit of a PITA to setup and manage an email server, it's been worth it.
I used "Mail-in-a-Box". It's pretty easy to set one up with that. It has a built-in DNS server and that's a really great thing to have for managing several domain names and as many email addresses as you want. I've setup email accounts for family and friends as well as throwaways for my wife, who signs up for everything she sees on the internet.
I can move the IP address of my email server to the top of the list in my Mac's System Preferences for DNS and start testing new domain names and changes to the DNS immediately. I don't have to wait for those to propagate to whatever my access provider is using.
So I have 3 servers. An Email/DNS server, a database server, and a website/webapp server running on DigitalOcean's "Droplets". It's a bit of work for a small shop but it's much easier to manage once it's setup and I don't have to worry about any 3rd party service selling out or going under or changing their API to something entirely different. All of which has happened to me in the past.
In my case users are sending estimates and invoices and monthly statements to their clients and while fake invoices may be a spammer thing those clients know who's sending them an invoice, and why and what for, so an 'Unsubscribe" link would be completely out of context because they are not subscribed to any email list.
I've had the same domain name for over 20 years now and none of my users have ever used my apps to send spam. And as spam and email volume go my server isn't even close to sending out a lot of email.
When I set up a new email server last year, with a new IP address, I had to go through the process of getting white listed. All the big email service providers have ways to do that. Google made it very easy. They gave you a unique string to add it to your DNS records and that's it. Microsoft is so convoluted I've still not gotten anywhere with them. Comcast and others had a few hoops and ladders but nothing that got me stuck.
Personally, while it's a bit of a PITA to setup and manage an email server, it's been worth it.
I used "Mail-in-a-Box". It's pretty easy to set one up with that. It has a built-in DNS server and that's a really great thing to have for managing several domain names and as many email addresses as you want. I've setup email accounts for family and friends as well as throwaways for my wife, who signs up for everything she sees on the internet.
I can move the IP address of my email server to the top of the list in my Mac's System Preferences for DNS and start testing new domain names and changes to the DNS immediately. I don't have to wait for those to propagate to whatever my access provider is using.
So I have 3 servers. An Email/DNS server, a database server, and a website/webapp server running on DigitalOcean's "Droplets". It's a bit of work for a small shop but it's much easier to manage once it's setup and I don't have to worry about any 3rd party service selling out or going under or changing their API to something entirely different. All of which has happened to me in the past.
We add link to accounts page to manage notifications - but we clearly say that certain notifications can be disable only by deleting the account.
But even that does not help: some people just do not understand how to manage their email: they will consider as “spam” any email they receive and they cannot act on it immediately. Like storage quota reached, cc expired, or similar. There is some strange feeling that if they ignore the email and mark it as spam that the problem will magically disappear.
We have so many cases like that: and they then contact us with questions like “why my service is not working” etc.
I bet Netflix has same issues…
But even that does not help: some people just do not understand how to manage their email: they will consider as “spam” any email they receive and they cannot act on it immediately. Like storage quota reached, cc expired, or similar. There is some strange feeling that if they ignore the email and mark it as spam that the problem will magically disappear.
We have so many cases like that: and they then contact us with questions like “why my service is not working” etc.
I bet Netflix has same issues…
You guys are thinking way too literally and rigidly about this. Unsubscribe in a transactional email does not need to be an automated stop. Just have it open a support ticket and then follow up to find out why they clicked unsubscribe.
If they are an established customer, it is legal to do that in all jurisdictions, even under GDPR.
It’s much better than having a customer mark it as spam, as some people will definitely do if they can’t see how to unsubscribe.
If they are an established customer, it is legal to do that in all jurisdictions, even under GDPR.
It’s much better than having a customer mark it as spam, as some people will definitely do if they can’t see how to unsubscribe.
I've also had email from the wrong person delivered to me some times. One company in particular kept sending updates for a service I had no way of using. An unsubscribe link would have been handy, though confirming email addresses before linking them to an account would also be a good idea, probably.
Those unsubscribe links should be there, for several reasons.
In some jurisdictions there is information that businesses are legally required to provide to their customers in a permanent form and email is the conventional (and potentially the only) way of satisfying that requirement.
IMHO, it is not helpful for anyone to have a system where recipients may not understand this and may treat that mail as spam, yet businesses are compelled to send it anyway.
In some jurisdictions there is information that businesses are legally required to provide to their customers in a permanent form and email is the conventional (and potentially the only) way of satisfying that requirement.
IMHO, it is not helpful for anyone to have a system where recipients may not understand this and may treat that mail as spam, yet businesses are compelled to send it anyway.
Mail may also be simply going to the completely wrong address, for some reason, and there needs to be some way to ask that this be corrected without requiring credentials that the recipient might not have. If not an unsubscribe link, then at least a reply-to address that actually works, or some way to handle a mistake.
I had one email address that for some reason often ended up being mistakenly used for other people, and ended up enormously frustrated by the usual combinations of no unsubscribe links, no-reply senders, and notes that any changes should be made by logging in. In the worst case, the US Department of Education, for several years, sent the address private personal and financial details about a complete stranger at a university with student loans. There was no way for me to unsubscribe, and no monitored address to reply to: in their view, they were important emails for the recipient, and they were, but I was not the intended recipient at all.
I had one email address that for some reason often ended up being mistakenly used for other people, and ended up enormously frustrated by the usual combinations of no unsubscribe links, no-reply senders, and notes that any changes should be made by logging in. In the worst case, the US Department of Education, for several years, sent the address private personal and financial details about a complete stranger at a university with student loans. There was no way for me to unsubscribe, and no monitored address to reply to: in their view, they were important emails for the recipient, and they were, but I was not the intended recipient at all.
I treat every unsubscribe link as though it read "click here to confirm your email address is live and being used by a human being so it will command a higher value when we resell it".
If I don't like an email I never unsub. I add it to the spam filter.
If I don't like an email I never unsub. I add it to the spam filter.
[deleted]
> We were also informed that our normal, business communications with paying customers should have unsubscribe notices appended to them.
You should have an unsubscribe link. You should also have your business address and identify yourself.
Even if it's not required by the letter of the law, you should add it.
As an example: Amazon automatically opted me into an "alert" when a wishlist I viewed had a new viewer. Since it's an "alert" and a "business communication" it has no unsubscribe. This is spam - this is an ad hidden as a notification.
You should have an unsubscribe link. You should also have your business address and identify yourself.
Even if it's not required by the letter of the law, you should add it.
As an example: Amazon automatically opted me into an "alert" when a wishlist I viewed had a new viewer. Since it's an "alert" and a "business communication" it has no unsubscribe. This is spam - this is an ad hidden as a notification.
"You should have an unsubscribe link. You should also have your business address and identify yourself."
What would that even look like ?
You're a paying customer of a service - they charge you every month - and you use that service ~daily ... and then you unsubscribe to emails ...
So then what ?
We just keep taking your money and when the service fails or there is an outage or critical notification we ... just don't send it ?
What would that even look like ?
You're a paying customer of a service - they charge you every month - and you use that service ~daily ... and then you unsubscribe to emails ...
So then what ?
We just keep taking your money and when the service fails or there is an outage or critical notification we ... just don't send it ?
Yes, just because I use your service doesn't mean I want to see every outage notification status update as an email. Preferably email subscription status would be granular so I can select what I want to get not what some idealized average user would want to get.
I've been a paying customer of rsync's service for more than a decade. The only mail I get is the monthly invoice, and roughly once-per-year notice of infrastructure changes that may temporarily affect availability.
Oh I have no doubts whatsoever the volume is low and the messages sent intended to be genuinely important to the vast majority of customers, rsync seems very reputable based on what I've heard over the years on HN.
It's still nice to have granular subscription though even for rare things you think 95% of users may like to hear about e.g. I've been using a similar service since 2015 and I have 0 interest in receiving their downtime or scheduled maintenance notifications as I don't care enough to take a special action for a failed sync or two in the first place so... I don't opt to receive them and I appreciate that option. I don't get the invoices emailed so I haven't had to think about it one way or the other there.
It's still nice to have granular subscription though even for rare things you think 95% of users may like to hear about e.g. I've been using a similar service since 2015 and I have 0 interest in receiving their downtime or scheduled maintenance notifications as I don't care enough to take a special action for a failed sync or two in the first place so... I don't opt to receive them and I appreciate that option. I don't get the invoices emailed so I haven't had to think about it one way or the other there.
Yes. I explicitly told you I didn’t want any emails from you.
That is gonna lead to all kinds of misunderstandings and complaints.
If I unsubscribed that already told you I didn’t want you sending me emails. Your attitude is the very reason I use “Hide My Email”.
Surely 99.9% of your users have a backup system that could recover from a day-long outage on your end without any manual intervention?
What are they supposed to do in response to an outage notification, apart from wait for things to recover on their own?
What are they supposed to do in response to an outage notification, apart from wait for things to recover on their own?
~rsync runs a storage service for offsite backups. You think they should add a one-click "unsubscribe" link to service alerts?
if the service is managed: customers should be able to manage notification preferences tailored to the severity of the issue, methinks.
That's not how unsubscribe links are supposed to work.
Once the unsubscribe is activated -- and it's supposed to be very easy to activate -- then it's permanent. There's no "un-unsubscribe", "oops I clicked it again", "some other service glitched and clicked it for me".
Further, there's a distinction made between "commercial" and "transactional" messages in both law and etiquette. The unsubscribe link is expected in commercial messages, not transactional ones.
Abusix didn't know what they were talking about.
Once the unsubscribe is activated -- and it's supposed to be very easy to activate -- then it's permanent. There's no "un-unsubscribe", "oops I clicked it again", "some other service glitched and clicked it for me".
Further, there's a distinction made between "commercial" and "transactional" messages in both law and etiquette. The unsubscribe link is expected in commercial messages, not transactional ones.
Abusix didn't know what they were talking about.
> Further, there's a distinction made between "commercial" and "transactional" messages in both law and etiquette. The unsubscribe link is expected in commercial messages, not transactional ones.
Most of the junk that gets through my spam filter are transactional or other "mandatory" messages intended for someone who fat fingered their email address. If those senders don't want to be marked as spam, they need to provide a way for me to make the messages stop.
Most of the junk that gets through my spam filter are transactional or other "mandatory" messages intended for someone who fat fingered their email address. If those senders don't want to be marked as spam, they need to provide a way for me to make the messages stop.
Email confirmations should be standard but that's not what we're talking about here (and I'd expect that ~rsync is handling that properly).
Unsubscribing from transactional emails eventually causes the following support conversation: "Hi, uhh, rsync? Yeah, so, I'm having trouble logging in to my account and we really really need our backups, our intern just nuked a database. Yeah, it's uhh... [email protected]. What do you mean my account's not active? ... ... Why didn't you just tell me my card expired? Well yeah, of course I unsubscribed, but I still wanted to know my account was being shut down!"
There's a scale of headaches happening here. At one end of the scale we have "nuisance", as in, "I'm getting too much email, or I have a stupid email address, or I don't know how to filter messages from reputable senders", and at the other end we have "job-ending cockup", as in, "I'm just now finding out that a critical part of our disaster recovery plan hasn't been working for a long time because somebody somewhere was inconvenienced by a notification, and I'm finding this out now because today happens to be the day we really need that disaster recovery plan".
Pushing the needle away from the nuisance end moves it closer to the disaster end.
Unsubscribing from transactional emails eventually causes the following support conversation: "Hi, uhh, rsync? Yeah, so, I'm having trouble logging in to my account and we really really need our backups, our intern just nuked a database. Yeah, it's uhh... [email protected]. What do you mean my account's not active? ... ... Why didn't you just tell me my card expired? Well yeah, of course I unsubscribed, but I still wanted to know my account was being shut down!"
There's a scale of headaches happening here. At one end of the scale we have "nuisance", as in, "I'm getting too much email, or I have a stupid email address, or I don't know how to filter messages from reputable senders", and at the other end we have "job-ending cockup", as in, "I'm just now finding out that a critical part of our disaster recovery plan hasn't been working for a long time because somebody somewhere was inconvenienced by a notification, and I'm finding this out now because today happens to be the day we really need that disaster recovery plan".
Pushing the needle away from the nuisance end moves it closer to the disaster end.
The service is not meant to cater to the lowest common denominator. If you unsubscribe from critical notifications and get screwed over.. that is on you.
It is not fair to the rest of us to be inundated with endless spam just so some screwup can be kept from doing something stupid.
It is not fair to the rest of us to be inundated with endless spam just so some screwup can be kept from doing something stupid.
Transactional email from a backup service you deliberately signed up to isn’t spam, so congratulations you’ve got what you’re after.
Now someone will likely reply shifting the definition of what “spam” is to include Rsync’s critical service emails, and now the term spam is so wide as to be meaningless.
At that point it’s on you to manage your own spam filter if you truly feel “your critical backup service is down” is spam. I haven’t been inundated with endless spam for about a decade.
Abusix don’t know what they are talking about, and basically all services that let you manage your email notifications still send through critical “your service is about to be turned off because your card details failed” emails regardless of how many checkboxes you disable — and for good reason.
Now someone will likely reply shifting the definition of what “spam” is to include Rsync’s critical service emails, and now the term spam is so wide as to be meaningless.
At that point it’s on you to manage your own spam filter if you truly feel “your critical backup service is down” is spam. I haven’t been inundated with endless spam for about a decade.
Abusix don’t know what they are talking about, and basically all services that let you manage your email notifications still send through critical “your service is about to be turned off because your card details failed” emails regardless of how many checkboxes you disable — and for good reason.
I got dogpiled on here a couple weeks ago for the temerity to suggest that "spam" is, by definition, unsolicited. Unreasonable people like this put companies in no-win situations.
I did not solicit emails from that service.
I solicited them to store my data.
Did you solicit every nag and advert Amazon sends you?
That you bought something from someone does not mean your email inbox is now free game.
I solicited them to store my data.
Did you solicit every nag and advert Amazon sends you?
That you bought something from someone does not mean your email inbox is now free game.
Fine then. Filter them client side. That’s your choice, don’t make the choice for others.
[deleted]
You can always login and re-enable an email on the service. The service is allowed to request information needed to process the unsubscribe.
I get emails for some dude's Chevy when it needs servicing. I can't unsubscribe. I am stuck getting emails about a car I have never owned from some dealer in Pittsburg. I need an opt out that lets me communocate "hey, some dumbass fatfingered his email, stop spamming me."
I get emails for some dude's Chevy when it needs servicing. I can't unsubscribe. I am stuck getting emails about a car I have never owned from some dealer in Pittsburg. I need an opt out that lets me communocate "hey, some dumbass fatfingered his email, stop spamming me."
Genuine question: your comment and a bunch of others make me wonder why people seem unable to filter email by sender. That used to be a pretty standard part of having an inbox. Are you using a mail client or service that doesn't have filtering built in? Do you find it difficult to set up a filter rule? Are you unfamiliar with filter rules? Do you use filters but just ideologically object to any unwanted email?
I'm honestly curious.
I'm honestly curious.
It's a chore, and it is a never ending one. I have several thousand senders blocked. However,
1. Some senders intentionally send ads and actually useful notifications from the same address, making filtering more difficult
2. Better yet, some senders constantly shift their address so they are almost unblockable without going to arbitrary keyword blocking
I would like to live in a world where instead of me doing stupid amounts of work to not get a flood of spam.. people just don't spam.
I am lucky - I can literally throw money and tech at this problem. Most people are not lucky. I would much rather spam be elimited as a whole.
1. Some senders intentionally send ads and actually useful notifications from the same address, making filtering more difficult
2. Better yet, some senders constantly shift their address so they are almost unblockable without going to arbitrary keyword blocking
I would like to live in a world where instead of me doing stupid amounts of work to not get a flood of spam.. people just don't spam.
I am lucky - I can literally throw money and tech at this problem. Most people are not lucky. I would much rather spam be elimited as a whole.
The conversations about spam are usually incredibly nebulous, as there's different perceptions and perspectives.
I think what you're picking up on, is that some folks don't differentiate between commercial email filtering services, and personal spam filters.
There's conflation of email 'I don't care about and don't want', bulk UCE, shifty list operators with shifty 40 page terms, etc.
I think what you're picking up on, is that some folks don't differentiate between commercial email filtering services, and personal spam filters.
There's conflation of email 'I don't care about and don't want', bulk UCE, shifty list operators with shifty 40 page terms, etc.
Gmail filters are incapable of marking things as spam. Worse, if something matches a filter rule, it's whitelisted.
Well, that's horrible and explains a lot. Thanks.
For rsync however, it seems more likely that it's instead things like disc quota or expiring service.
At least based on my understanding of rsync-the-company and rsync-the-hn-commenter.
At least based on my understanding of rsync-the-company and rsync-the-hn-commenter.
Though that may be a nice opinion, your ISP has no business dictating that to you.
Abusix isn't their ISP, they're an email blocklist provider. Telling people what they need to do to not get blocked for being abusive is literally their job.
yea, in an ideal world, the blocklist provider would educate others in how to avoid beeing blocklisted. yet, if this course is met with success then the blocklist provider is out of business.
there seems to be some conflict of interest here.
there seems to be some conflict of interest here.
Please see my comment below: https://news.ycombinator.com/item?id=30227886 and the subsequent threads below.
Not all blocklist providers are the same.
Not all blocklist providers are the same.
terribly sorry that you felt personally implicated, that was not my intent.
thou every security-for-profit scheme suffers from afromentioned conflict of interest.
thou every security-for-profit scheme suffers from afromentioned conflict of interest.
> We were also informed that our normal, business communications with paying customers should have unsubscribe notices appended to them. Which is to say, you're a paying customer of a service and we send you some kind of alert or critical announcement ... and it should have an unsubscribe link.
You absolutely should. The amount of junk I get because someone else signed up for something and fat fingered their email address is ridiculous. "Mandatory communication" with a company I've never dealt with gets flagged as spam.
You absolutely should. The amount of junk I get because someone else signed up for something and fat fingered their email address is ridiculous. "Mandatory communication" with a company I've never dealt with gets flagged as spam.
That's a (very legitimate and important) reason to do double opt-in unilaterally for all email communications. Companies should make 100% sure that the person who signed up, and the person receiving the email, are the same person, before they associate the email with the account. Otherwise, malicious people can sign up arbitrary third parties for tons of random crap.
But it's not a good reason for adding unsubscribe links unilaterally to all email communications.
Remember, unsub links are machine-automatable; Gmail at least offers to follow any embedded unsubscribe links for you if you mark a message as spam. (Which, with hotkeys enabled, is one accidental keypress away.)
So consider the extreme case: what if the user fat-fingers an unsubscribe (without realizing) to their local electric company's e-invoices, which is what they've been relying on to prod them to log onto the site and pay the bill?
If it's clear that "bills you need to react to or your power will be shut off" shouldn't have an unsubscribe link, then clearly there's some sort of line that must be drawn somewhere.
(Note, I'm not arguing against the use of "Manage your Mail Preferences" links in these cases — the kind that act as magic sign-in links and take you directly to a page on which you can un-check a "mail me about X" checkbox. It makes sense to include those. I'm just arguing specifically against unilaterally including "Unsubscribe" links — the kind where following the link unsubscribes you with no further confirmation needed.)
But it's not a good reason for adding unsubscribe links unilaterally to all email communications.
Remember, unsub links are machine-automatable; Gmail at least offers to follow any embedded unsubscribe links for you if you mark a message as spam. (Which, with hotkeys enabled, is one accidental keypress away.)
So consider the extreme case: what if the user fat-fingers an unsubscribe (without realizing) to their local electric company's e-invoices, which is what they've been relying on to prod them to log onto the site and pay the bill?
If it's clear that "bills you need to react to or your power will be shut off" shouldn't have an unsubscribe link, then clearly there's some sort of line that must be drawn somewhere.
(Note, I'm not arguing against the use of "Manage your Mail Preferences" links in these cases — the kind that act as magic sign-in links and take you directly to a page on which you can un-check a "mail me about X" checkbox. It makes sense to include those. I'm just arguing specifically against unilaterally including "Unsubscribe" links — the kind where following the link unsubscribes you with no further confirmation needed.)
> So consider the extreme case: what if the user fat-fingers an unsubscribe (without realizing) to their local electric company's e-invoices, which is what been relying on to prod them to log onto the site and pay the bill?
To name a specific example of this problem, I want Gulf Power of Florida to stop sending exactly the kind of email you speak of. Bills. Nastygrams when the person falls behind on the bills. Unwanted power saving tips. Calling the company and sending them postal mail has not helped. It all gets marked as spam these days. If they had an unsubscribe button, it wouldn't.
If the email is so damn important, they can go back to sending postal mail to the service address when someone unsubscribes.
To name a specific example of this problem, I want Gulf Power of Florida to stop sending exactly the kind of email you speak of. Bills. Nastygrams when the person falls behind on the bills. Unwanted power saving tips. Calling the company and sending them postal mail has not helped. It all gets marked as spam these days. If they had an unsubscribe button, it wouldn't.
If the email is so damn important, they can go back to sending postal mail to the service address when someone unsubscribes.
I think you're trying to use two wrongs to make a right. If we're talking about things Gulf Power of Florida should do differently, then rather than add unsubscribe buttons to bills which is a bad idea, they should confirm people's email addresses before sending them email.
What's wrong with giving the user the ability to remove themselves from any automated emails? The alternative is being hit with the spam button.
They should have confirmed their user controlled the email address, too, but why not go with both?
And this is hardly confined to Gulf Power. Verizon, Spectrum, countless banks...
They should have confirmed their user controlled the email address, too, but why not go with both?
And this is hardly confined to Gulf Power. Verizon, Spectrum, countless banks...
Well, I think letting people get silently charged monthly without sending them any message about it is a bad idea, even if they say they want it. Similar with not telling them about an outage to their service.
Well, then that mail is going to get legitimately marked as spam when I didn't request it, I have no business relationship with the sender, and there's no other way to make it stop. And the user is still getting silently charged because it's going to my inbox, not theirs.
Unsubscribe is the alternative to the spam button. If you don't provide it, you are asking to be marked as spam, no matter how important you think the email is.
Unsubscribe is the alternative to the spam button. If you don't provide it, you are asking to be marked as spam, no matter how important you think the email is.
If you have no business relationship with the sender, this is fixed by not sending email to unconfirmed addresses, as we discussed above.
the email w/unsub link could be forwarded also, it's often a portal to change notification settings w/o auth and leaks personal preference info - and when there is auth it's impossible to unsub when if were signed up maliciously.
it happened to me - someone charged a bunch of stuff to my cc and then registered my email at thousands of sites to bury the email receipts (it didn't work since I have simple filters for that sort of thing) but it has been impossible to unsubscribe from all the junk. livemail's bulk optout was roughly 50% effective. the dark patterns around optout are outrageous and it's worse when you have to use google translate just to find it.
it happened to me - someone charged a bunch of stuff to my cc and then registered my email at thousands of sites to bury the email receipts (it didn't work since I have simple filters for that sort of thing) but it has been impossible to unsubscribe from all the junk. livemail's bulk optout was roughly 50% effective. the dark patterns around optout are outrageous and it's worse when you have to use google translate just to find it.
Ugh that sucks.
But in the cases where there is authentication, isn't it enough (in most cases) to reset the password and change the email to something disposable?
Of course that's not really practical for the case where you get subscribe-bombed, but maybe for the general case it is, no?
But in the cases where there is authentication, isn't it enough (in most cases) to reset the password and change the email to something disposable?
Of course that's not really practical for the case where you get subscribe-bombed, but maybe for the general case it is, no?
a bunch of my remaining junk accounts have broken pw reset, or even after taking control there's no way to delete the account and/or the "optout of everything" option doesn't actually stop them from sending me junk regularly (ie they just dont honor their own optouts)
> So consider the extreme case: what if the user fat-fingers an unsubscribe (without realizing) to their local electric company's e-invoices, which is what they've been relying on to prod them to log onto the site and pay the bill?
I actually unsubscribed from my provider's invoices. That's because I have activated direct debit from my bank account so they're always paid, and I can view my past invoices on the website.
However you make a good point. I'd say the one thing where it doesn't make sense to have an "unsubscribe" at all is on "bill unpaid" emails.
I actually unsubscribed from my provider's invoices. That's because I have activated direct debit from my bank account so they're always paid, and I can view my past invoices on the website.
However you make a good point. I'd say the one thing where it doesn't make sense to have an "unsubscribe" at all is on "bill unpaid" emails.
For my power to be cut off, I would have to…
1. Forget I had a monthly power bill for a couple of months.
2. Ignore the e-bill that gets sent to my bank bill payment service - ebills have been a thing for almost two decades. I worked on some of the early implementations.
3. Ignore the physical snail mail warnings for a couple of months.
1. Forget I had a monthly power bill for a couple of months.
2. Ignore the e-bill that gets sent to my bank bill payment service - ebills have been a thing for almost two decades. I worked on some of the early implementations.
3. Ignore the physical snail mail warnings for a couple of months.
[deleted]
> Otherwise, malicious people can sign up arbitrary third parties for tons of random crap.
In the early days of the internet I was a teenager in a prank war with someone so I signed them up for all kinds of spam and also free trial magazine subscriptions.
In the early days of the internet I was a teenager in a prank war with someone so I signed them up for all kinds of spam and also free trial magazine subscriptions.
That's a different problem. They should have first sent a confirmation email, then paused all communications until it was confirmed.
But once the email is confirmed, I think it's totally fair for a company like rsync to say 'if you're a paying customer of this service, then we need to send you certain information to fulfill our obligations in the contract, if you truly don't like it cancel your account and take your business elsewhere.'
But once the email is confirmed, I think it's totally fair for a company like rsync to say 'if you're a paying customer of this service, then we need to send you certain information to fulfill our obligations in the contract, if you truly don't like it cancel your account and take your business elsewhere.'
Yep. I had to file a GDPR complaint to get an airline to stop sending me "letters from the CEO" and other COVID-related reports that never mattered to me.
I never flew with them but somehow they still sent ads disguised as reassuring messages every other week.
Support constantly denied help, since I was never a customer. Only a GDPR complaint solved it.
I never flew with them but somehow they still sent ads disguised as reassuring messages every other week.
Support constantly denied help, since I was never a customer. Only a GDPR complaint solved it.
The inability to generate IP reputation smaller than a /24 is inherent to the way internet routing works. Nothing smaller than a /24 can be publicly assigned or advertised to prevent the route table from becoming too bloated. On IPv6 the smallest advertise able block is a /48 for the same reason. Privately managed assignments in shared or further split subnets aren't publicly visible, verifiable, or accountable to anything but the organization owning and advertising the /24 (or larger).
As such the reputation score of a subnet is the reputation of the entity advertising itself as publicly controlling and maintaining that network, not the reputation of individual sub entities inside that subnet (which is known only to the controlling entity). If that entity is constantly allowing bad actors onto their block then that block is considered poor reputation.
As such the reputation score of a subnet is the reputation of the entity advertising itself as publicly controlling and maintaining that network, not the reputation of individual sub entities inside that subnet (which is known only to the controlling entity). If that entity is constantly allowing bad actors onto their block then that block is considered poor reputation.
> The inability to generate IP reputation smaller than a /24 is inherent to the way internet routing works.
Nonsense. SMTP is not internet routing, and there is no reason at all why some "reputation" system should be constrained by the same /24 limit that the global routing table ended up with.
I find it telling that these "reputation" outfits tend to serve up this sort of poor excuse to justify their businesses; whereas they're often plain old protection rackets. Like the one described in the article clearly is. They take money to solve the problem they created!
Nonsense. SMTP is not internet routing, and there is no reason at all why some "reputation" system should be constrained by the same /24 limit that the global routing table ended up with.
I find it telling that these "reputation" outfits tend to serve up this sort of poor excuse to justify their businesses; whereas they're often plain old protection rackets. Like the one described in the article clearly is. They take money to solve the problem they created!
I think you're thinking domain based email reputation lists but commenting on the IP reputation conversation while also conflating what Abusix is doing in this comment chain with what UCEPROTECT is doing in the post and forming a conclusion from hand picked pieces of the conglomerate. For the Abusix portion related to this thread check out their response here as it goes into much better detail with what I originally described https://news.ycombinator.com/item?id=30227886 (note: I'm unaffiliated, I work for a generic network VAR and we don't even sell their service for email).
These domain based lists are also a thing but you need to have a pretty sizable volume to become trusted in those lists as anyone can register a domain in seconds, none of the above will have gained trust on them.
The whitelist payment option for arbitrary hosts (which Abusix doesn't offer) is a workaround to create a publicly registered identity with some stake to tie the reputation to. I mean it can be other things to, scammy rackets exist too, but you can't just claim it's a poor excuse without offering a better answer that solves the same problem.
Speaking of spammers are the ones that created the problem because people started using individual network block lists before companies started aggregating and mediating them. You're welcome to make your own community replacement solution but the problem is the replacement has to be reliable enough other mail administrators want to use it not just nice for the occasional legitimate guy that wants to run email out of his house and be trusted by default. Also it's a lot of work to mediate.
These domain based lists are also a thing but you need to have a pretty sizable volume to become trusted in those lists as anyone can register a domain in seconds, none of the above will have gained trust on them.
The whitelist payment option for arbitrary hosts (which Abusix doesn't offer) is a workaround to create a publicly registered identity with some stake to tie the reputation to. I mean it can be other things to, scammy rackets exist too, but you can't just claim it's a poor excuse without offering a better answer that solves the same problem.
Speaking of spammers are the ones that created the problem because people started using individual network block lists before companies started aggregating and mediating them. You're welcome to make your own community replacement solution but the problem is the replacement has to be reliable enough other mail administrators want to use it not just nice for the occasional legitimate guy that wants to run email out of his house and be trusted by default. Also it's a lot of work to mediate.
I'm the Abusix engineer in question (and actually the architect of the system in question), and you're being somewhat "economical" with what actually happened here.
Here's the actual chain of events in question:
- You recently switched ISPs and that meant you moved to a new IP block.
- The IP block in question is owned by Hurricane Internet and unfortunately contains a host which persistently sends out a lot of junk (https://lookup.abusix.com/search?q=216.218.240.46)
- Without going into massive details on how our infra works, but when we have the most serious level of listing (e.g. hitting our most secure traps as in this case), we treat the same /24 more aggressively than we normally would (because of things like snoeshoe spam that would normally spread traffic across a wide range of IPs).
We use /24s only where we cannot determine different ownership of the IPs by looking at the abuse contact registered at the RIR e.g. if the IPs are different contacts then we don't bundle them into the same bucket. In this case Hurricane Internet owns the entire range, so the /24 is used.
We do this because there is no other way to do it that isn't completely abuseable by a bad actor e.g. rDNS is completely trivial to forge and to claim multiple fake entities. If someone has a fool-proof way to do this, then I'm all ears.
Then we get to your failings in this case:
- You don't have proper, working bounce handling.
You were repeatedly sending mail to an old customer and we were rejecting 100% of the email you sent to that address - stating that you should stop sending to it in the rejection message. We always reject traffic on traps we are building so that bounce handling removes them automatically over time.
- You decided to send a marketing message to a bunch of users, including to the address described above "We (rsync.net) are experimenting with a lifetime prepayment option". This message provided no List-Unsubscribe option at all, so I could not unsubscribe it without exposing the trap to your engineer (which is my primary concern as it takes years to build traps properly).
- Your engineer said to me "we took down 600 old accounts and are reviewing our contact policies going forward" and "there are still plenty of other customers who are listed generically as bouncing so we will have work to do here".
That tells me that you knew that you were sending to old accounts and that your bounce handling was either not great or non-existent.
I take our role very seriously and I and my team go out of our way to help anyone who finds themselves listed by providing evidence and advice and we always try to find a good resolution for all legitimate senders.
That is exactly what I did here with your engineer, the problem is resolved as the specific account was removed and you know now what you were doing wrong and how to fix it so that it never happens again.
Your engineer was appreciative and I said if there are any further issues in the meantime whilst you fix things on your end, then we would help by exempting your traffic whilst you did those changes.
I can't see how we could have done anything more in this case.
On our website, we provide blog posts and videos, we take part in conferences and workshops to give advice on how to do things properly so you never have blocklisting issues.
I can easily summarise here what everyone should do to avoid issues:
- Have proper rDNS for your sending IPs that is part of your administrative domain (e.g. don't use your providers generic rDNS that contains the entire IP in the hostname). If someone visits the domain name, make sure there is a website present that has contact details as a minimum.
- Make sure your abuse@domain and postmaster@domain accounts actually go to a responsible person.
- If you send any marketing mail at all, make sure it has a working List-Unsubscribe header (preferably HTTP that allows someone to unsubscribe without having to contact you). Important note: If you use a mailto: unsubscribe, then I cannot unsubscribe a trap, even if I wanted to.
- Make sure you have working bounce handling. If you repeatedly send to an account and it either bounces or is hard rejected, then you need to stop sending to it and mark it as bad. No excuses.
- Don't send to addresses where you have not contacted them or had any interaction at all for > 1 year. If you haven't kept in touch with your customers, then that is on you.
- If you have a web form that when submitted, sends a message to an external user, then you MUST a) validate all of the input fields and disallow URLs unless the field required it and b) prevent automated submission via the use of a CAPTCHA.
- When collecting email addresses for mailing lists, always use confirm opt-in e.g. send a message containing a link that they have to click to activate the subscription. Do not send any further messages to them until this has been completed.
- Make sure you separate IP addresses being used for outbound mail from those used for outbound NAT pool. Block outbound port 25 from the NAT pools and make your firewall notify you of any port 25 activity from any hosts as this could indicate they are infected.
- When provisioning a new mail server IP, don't send more than 30 messages per minute (e.g. 0.5 messages/sec) for the first day and then increase the volume over the following week.
I'm sure some will disagree with some of these, but I guarantee that if you follow all of these, then you'll never have an issue with a blocklist like ours.
I hope this helps.
Here's the actual chain of events in question:
- You recently switched ISPs and that meant you moved to a new IP block.
- The IP block in question is owned by Hurricane Internet and unfortunately contains a host which persistently sends out a lot of junk (https://lookup.abusix.com/search?q=216.218.240.46)
- Without going into massive details on how our infra works, but when we have the most serious level of listing (e.g. hitting our most secure traps as in this case), we treat the same /24 more aggressively than we normally would (because of things like snoeshoe spam that would normally spread traffic across a wide range of IPs).
We use /24s only where we cannot determine different ownership of the IPs by looking at the abuse contact registered at the RIR e.g. if the IPs are different contacts then we don't bundle them into the same bucket. In this case Hurricane Internet owns the entire range, so the /24 is used.
We do this because there is no other way to do it that isn't completely abuseable by a bad actor e.g. rDNS is completely trivial to forge and to claim multiple fake entities. If someone has a fool-proof way to do this, then I'm all ears.
Then we get to your failings in this case:
- You don't have proper, working bounce handling.
You were repeatedly sending mail to an old customer and we were rejecting 100% of the email you sent to that address - stating that you should stop sending to it in the rejection message. We always reject traffic on traps we are building so that bounce handling removes them automatically over time.
- You decided to send a marketing message to a bunch of users, including to the address described above "We (rsync.net) are experimenting with a lifetime prepayment option". This message provided no List-Unsubscribe option at all, so I could not unsubscribe it without exposing the trap to your engineer (which is my primary concern as it takes years to build traps properly).
- Your engineer said to me "we took down 600 old accounts and are reviewing our contact policies going forward" and "there are still plenty of other customers who are listed generically as bouncing so we will have work to do here".
That tells me that you knew that you were sending to old accounts and that your bounce handling was either not great or non-existent.
I take our role very seriously and I and my team go out of our way to help anyone who finds themselves listed by providing evidence and advice and we always try to find a good resolution for all legitimate senders.
That is exactly what I did here with your engineer, the problem is resolved as the specific account was removed and you know now what you were doing wrong and how to fix it so that it never happens again.
Your engineer was appreciative and I said if there are any further issues in the meantime whilst you fix things on your end, then we would help by exempting your traffic whilst you did those changes.
I can't see how we could have done anything more in this case.
On our website, we provide blog posts and videos, we take part in conferences and workshops to give advice on how to do things properly so you never have blocklisting issues.
I can easily summarise here what everyone should do to avoid issues:
- Have proper rDNS for your sending IPs that is part of your administrative domain (e.g. don't use your providers generic rDNS that contains the entire IP in the hostname). If someone visits the domain name, make sure there is a website present that has contact details as a minimum.
- Make sure your abuse@domain and postmaster@domain accounts actually go to a responsible person.
- If you send any marketing mail at all, make sure it has a working List-Unsubscribe header (preferably HTTP that allows someone to unsubscribe without having to contact you). Important note: If you use a mailto: unsubscribe, then I cannot unsubscribe a trap, even if I wanted to.
- Make sure you have working bounce handling. If you repeatedly send to an account and it either bounces or is hard rejected, then you need to stop sending to it and mark it as bad. No excuses.
- Don't send to addresses where you have not contacted them or had any interaction at all for > 1 year. If you haven't kept in touch with your customers, then that is on you.
- If you have a web form that when submitted, sends a message to an external user, then you MUST a) validate all of the input fields and disallow URLs unless the field required it and b) prevent automated submission via the use of a CAPTCHA.
- When collecting email addresses for mailing lists, always use confirm opt-in e.g. send a message containing a link that they have to click to activate the subscription. Do not send any further messages to them until this has been completed.
- Make sure you separate IP addresses being used for outbound mail from those used for outbound NAT pool. Block outbound port 25 from the NAT pools and make your firewall notify you of any port 25 activity from any hosts as this could indicate they are infected.
- When provisioning a new mail server IP, don't send more than 30 messages per minute (e.g. 0.5 messages/sec) for the first day and then increase the volume over the following week.
I'm sure some will disagree with some of these, but I guarantee that if you follow all of these, then you'll never have an issue with a blocklist like ours.
I hope this helps.
Incredibly insightful, thank you for taking the time to post this!
> - Don't send to addresses where you have not contacted them or had any interaction at all for > 1 year. If you haven't kept in touch with your customers, then that is on you.
That seems odd though. "Keeping in touch" in regular intervals is exactly the kind of thing customers may want to unsubscribe from, so a working unsubscribe function means there are customers where I cannot keep in touch. How is that on me? That means I can't send e.g. a password reset email to them if they try to log in two years later?
> - Don't send to addresses where you have not contacted them or had any interaction at all for > 1 year. If you haven't kept in touch with your customers, then that is on you.
That seems odd though. "Keeping in touch" in regular intervals is exactly the kind of thing customers may want to unsubscribe from, so a working unsubscribe function means there are customers where I cannot keep in touch. How is that on me? That means I can't send e.g. a password reset email to them if they try to log in two years later?
Apologies for not being clearer on this, I'm generally talking about marketing messages when I say "not keeping in touch".
My point here is "consent to send", e.g. I think most people would find that someone importing a years old list of email addresses and then suddenly sending marketing messages is unacceptable and constitutes spam.
If you're "doing it right", then "touching" your customers with a once a year "Hey, we haven't seen you in a while" message is absolutely fine and generally a good thing to do as it maintains consent, because they can unsubscribe if they're no longer interested (as they might have forgotten they signed-up or had an account).
If someone hasn't logged into your "service" for >1 year and now wants to do a password reset several years later, then I would say that is entirely different.
We generally try our hardest to handle this case without it causing a listing if you hit our traps with something like this.
My point here is "consent to send", e.g. I think most people would find that someone importing a years old list of email addresses and then suddenly sending marketing messages is unacceptable and constitutes spam.
If you're "doing it right", then "touching" your customers with a once a year "Hey, we haven't seen you in a while" message is absolutely fine and generally a good thing to do as it maintains consent, because they can unsubscribe if they're no longer interested (as they might have forgotten they signed-up or had an account).
If someone hasn't logged into your "service" for >1 year and now wants to do a password reset several years later, then I would say that is entirely different.
We generally try our hardest to handle this case without it causing a listing if you hit our traps with something like this.
"- Your engineer said to me "we took down 600 old accounts and are reviewing our contact policies going forward" and "there are still plenty of other customers who are listed generically as bouncing so we will have work to do here"."
(snip)
"That tells me that you knew that you were sending to old accounts and that your bounce handling was either not great or non-existent."
Yes, that is exactly right.
I have instructed everyone to be as liberal and forgiving of non-payment and failed contact as possible. These people, who were paying customers, have data stored here for safekeeping and we're not going to trash it because we haven't heard from them in 3 months or their email bounced.
I can give you hours of stories of customers who came back from military deployment, came back from depression, came back from prison, came back from financial ruin ... that contacted us, beyond all hope, to see if we still had their account. And we did.
After 21 years of doing this are there a few hundred accounts that we are giving extreme benefit of the doubt to ? There certainly are.
Bottom line: our duty of safeguarding customer data trumps this weeks fashionable spam heuristics.
(snip)
"That tells me that you knew that you were sending to old accounts and that your bounce handling was either not great or non-existent."
Yes, that is exactly right.
I have instructed everyone to be as liberal and forgiving of non-payment and failed contact as possible. These people, who were paying customers, have data stored here for safekeeping and we're not going to trash it because we haven't heard from them in 3 months or their email bounced.
I can give you hours of stories of customers who came back from military deployment, came back from depression, came back from prison, came back from financial ruin ... that contacted us, beyond all hope, to see if we still had their account. And we did.
After 21 years of doing this are there a few hundred accounts that we are giving extreme benefit of the doubt to ? There certainly are.
Bottom line: our duty of safeguarding customer data trumps this weeks fashionable spam heuristics.
> Bottom line: our duty of safeguarding customer data trumps this weeks fashionable spam heuristics.
I haven't suggested that you do anything differently to that.
Keep the data, keep the accounts, do whatever you feel is best for you and your business.
All I've said is to be smarter when it comes to sending email to old accounts that are repeatedly bouncing. I've outlined what was wrong and I worked with your engineers to resolve it.
I haven't suggested that you do anything differently to that.
Keep the data, keep the accounts, do whatever you feel is best for you and your business.
All I've said is to be smarter when it comes to sending email to old accounts that are repeatedly bouncing. I've outlined what was wrong and I worked with your engineers to resolve it.
An aside:
The screenshot I got of your conversation with (Dave) was, indeed very well done and was both informative and actionable. You responded to us very quickly and with a high level of professionalism.
Further, participating here in this HN discussion is noteworthy and impressive.
FWIW, we immediately complained to he.net about the bad actor elsewhere in that /24 ... we'll see.
The screenshot I got of your conversation with (Dave) was, indeed very well done and was both informative and actionable. You responded to us very quickly and with a high level of professionalism.
Further, participating here in this HN discussion is noteworthy and impressive.
FWIW, we immediately complained to he.net about the bad actor elsewhere in that /24 ... we'll see.
> The screenshot I got of your conversation with (Dave) was, indeed very well done and was both informative and actionable. You responded to us very quickly and with a high level of professionalism.
> Further, participating here in this HN discussion is noteworthy and impressive.
Thank you - much appreciated.
The quality, morals and perception of our product and how we treat people matters to me greatly which is why I'm always happy to get involved in discussions like this.
As is evidenced from other post on this topic, there are a number of competitors that don't behave in the same way and that leads people to think we are all the same and I absolutely want to challenge that perception when it comes to us.
> FWIW, we immediately complained to he.net about the bad actor elsewhere in that /24 ... we'll see.
Thanks, feel free to refer them to me and I'm happy to provide as much evidence as they need.
> Further, participating here in this HN discussion is noteworthy and impressive.
Thank you - much appreciated.
The quality, morals and perception of our product and how we treat people matters to me greatly which is why I'm always happy to get involved in discussions like this.
As is evidenced from other post on this topic, there are a number of competitors that don't behave in the same way and that leads people to think we are all the same and I absolutely want to challenge that perception when it comes to us.
> FWIW, we immediately complained to he.net about the bad actor elsewhere in that /24 ... we'll see.
Thanks, feel free to refer them to me and I'm happy to provide as much evidence as they need.
As a former admin of an isp & running saas apps I applaud you greatly for your approach. Some people may find these approaches are "heavy handed" but the reality this is what life is like in 2022 with email. Should we need to do this? Definitely not, but it's an arms race and each company sending email has to up their game - to both be compliant in getting email accepted, and to foster a healthy relationship with customers.
The comment about "lifetime prepayment option" is exactly something that MUST have an unsubscribe option to it. I understand everyone is trying to increase business, but this is something @rsync missed mentioning, insisting you don't need one.
Bollocks.
The comment about "lifetime prepayment option" is exactly something that MUST have an unsubscribe option to it. I understand everyone is trying to increase business, but this is something @rsync missed mentioning, insisting you don't need one.
Bollocks.
Great response, and I was just wondering what went into sending email.
One question though, what do you mean by 'traps'? I feel like I'm missing some interesting context
One question though, what do you mean by 'traps'? I feel like I'm missing some interesting context
Traps = Spam Traps.
One of the ways most blocklists work is by employing spam traps which can either be individual email addresses or entire domains.
It's quite a large topic, but I'll try and summarise for you.
You can't just take a spam trap and use it to block anything that hits it - that would be incredibly unfair as you might not know the history of the email address or domain and you'd generate considerable false positives if you did.
If you buy a domain or register an email address and you know for certain that it's never been used before, then this is typically called a "pristine trap" and there are then various ways to then "seed" this so it starts to receive spam and malicious traffic. This is the exception where it could be used immediately. However this process usually takes a very long time (usually years) to receive enough traffic to be useful.
Any other address or domain where you don't know the history would be called a "recycled" trap. There are various opinions on how these should be managed, but generally it's accepted that you need to reject ALL traffic on these for at least 2 years to allow genuine senders to work out they are no longer valid (which is why bounce handling is important!).
The other type of trap is a "typo" trap, which is typically an entire domain (or an email address deliberately close to another) which could easily be typo'd. I have a rule here that says we never ever use these for any blocklisting (other lists will not have the same rules!), however they can be useful to see trends and detect compromised hosts (as these typos frequently end up in compromised databases that get sold around the dark web). They can receive considerable volumes as well.
Typo traps are especially why you should always use confirmed opt-in and employ CAPTCHA on your sign-up forms (although this also helps with all traps in general).
We carefully monitor all traps to see how they are performing e.g. detections .vs. traffic from whitelisted sources and false positive reports and will immediately remove a trap if it begins to perform poorly or has been made public in any way.
It takes years to build a trap network and we have to discard all of that work should they become discovered (as they could be used maliciously), which is why a blocklist operator will never reveal the addresses to you as they are a closely guarded secret. In fact on our own network, even we don't see them - we monitor them via a hashed name! (only I and a few other select people can convert the hashes to the actual domains or addresses).
I think that covers the basics - I hope that explains it sufficiently for you
One of the ways most blocklists work is by employing spam traps which can either be individual email addresses or entire domains.
It's quite a large topic, but I'll try and summarise for you.
You can't just take a spam trap and use it to block anything that hits it - that would be incredibly unfair as you might not know the history of the email address or domain and you'd generate considerable false positives if you did.
If you buy a domain or register an email address and you know for certain that it's never been used before, then this is typically called a "pristine trap" and there are then various ways to then "seed" this so it starts to receive spam and malicious traffic. This is the exception where it could be used immediately. However this process usually takes a very long time (usually years) to receive enough traffic to be useful.
Any other address or domain where you don't know the history would be called a "recycled" trap. There are various opinions on how these should be managed, but generally it's accepted that you need to reject ALL traffic on these for at least 2 years to allow genuine senders to work out they are no longer valid (which is why bounce handling is important!).
The other type of trap is a "typo" trap, which is typically an entire domain (or an email address deliberately close to another) which could easily be typo'd. I have a rule here that says we never ever use these for any blocklisting (other lists will not have the same rules!), however they can be useful to see trends and detect compromised hosts (as these typos frequently end up in compromised databases that get sold around the dark web). They can receive considerable volumes as well.
Typo traps are especially why you should always use confirmed opt-in and employ CAPTCHA on your sign-up forms (although this also helps with all traps in general).
We carefully monitor all traps to see how they are performing e.g. detections .vs. traffic from whitelisted sources and false positive reports and will immediately remove a trap if it begins to perform poorly or has been made public in any way.
It takes years to build a trap network and we have to discard all of that work should they become discovered (as they could be used maliciously), which is why a blocklist operator will never reveal the addresses to you as they are a closely guarded secret. In fact on our own network, even we don't see them - we monitor them via a hashed name! (only I and a few other select people can convert the hashes to the actual domains or addresses).
I think that covers the basics - I hope that explains it sufficiently for you
That was phenomenal, thank you!
> make sure it has a working List-Unsubscribe header (preferably HTTP that allows someone to unsubscribe without having to contact you). Important note: If you use a mailto: unsubscribe, then I cannot unsubscribe a trap, even if I wanted to.
This is very interesting to me. Could you elaborate further?
- Some providers (IIRC outlook being the main one) only support mailto: and most prefer it. I have never seen advice that you "must" provide HTTP before.
- What do you mean by unsubscribe without contacting me? Isn't HTTP or SMTP just as much contacting me?
- Why can't you unsubscribe mailto: even if you wanted to? Earlier you hinted that it may have to do with revealing the trap, but while a HTTP url may not directly include the trap address it would be linked anyways. (Otherwise how would the unsubscribe work?)
Thanks for all the valuable insights. It does sound like you are taking a responsible approach here.
This is very interesting to me. Could you elaborate further?
- Some providers (IIRC outlook being the main one) only support mailto: and most prefer it. I have never seen advice that you "must" provide HTTP before.
- What do you mean by unsubscribe without contacting me? Isn't HTTP or SMTP just as much contacting me?
- Why can't you unsubscribe mailto: even if you wanted to? Earlier you hinted that it may have to do with revealing the trap, but while a HTTP url may not directly include the trap address it would be linked anyways. (Otherwise how would the unsubscribe work?)
Thanks for all the valuable insights. It does sound like you are taking a responsible approach here.
Sure - happy to elaborate here, re-reading my response I understand the confusion here as my choice language wasn't great ;-)
I didn't say anywhere that you MUST support HTTP unsubscribes, but I just said it was preferable (to me at least).
HTTP typically means there is some sort of form which will typically auto-fill the email address from the database (because of the HTTP GET parameters included in the link) and all the user has to do is generally click 'Unsubscribe' and the address is automatically removed from the database.
The reason I prefer this is that if I or one of my colleagues want to unsubscribe a spam trap, then we can - but only if the above is true. (Disclaimer: we only do this very occasionally, usually it's only if we've spoken to someone and are convinced they are legitimate, have fixed any issues we've identified, but are hitting traps because of a single address - if we didn't do this then we'd either have to force the sender to reconfirm their entire database by asking everyone to opt-in again, or we'd have to whitelist the sender, neither are great options - so if we ever do this, then it's done hours/days after the event so as to not tip off anyone on what the trap was).
Because spam traps have to stay secret, we only reference them by a hash value internally (and only I and a select few can convert hash to trap value to do the unsubscribe) and as the messages are received on our trap network the traps themselves are obfuscated to a know value before they are stored in our evidence system.
mailto: means we can't do this at all - a trap can never originate traffic, that's cardinal rule of them, it would be grossly unfair if they did.
> - What do you mean by unsubscribe without contacting me? Isn't HTTP or SMTP just as much contacting me?
Poor use of words from me.
What I meant is that I don't trust messages that have an unsubscribe that does not encode the recipient in it in some way e.g. HTTP method without GET parameters (as this means I would have to do considerably more work to fill out the form) or a mailto:[email protected]?subject=unsubscribe - both to me scream our that there is no automatic connection to a database and that someone on the receiving end has to manually remove the address - which could take weeks, or be never.
I didn't say anywhere that you MUST support HTTP unsubscribes, but I just said it was preferable (to me at least).
HTTP typically means there is some sort of form which will typically auto-fill the email address from the database (because of the HTTP GET parameters included in the link) and all the user has to do is generally click 'Unsubscribe' and the address is automatically removed from the database.
The reason I prefer this is that if I or one of my colleagues want to unsubscribe a spam trap, then we can - but only if the above is true. (Disclaimer: we only do this very occasionally, usually it's only if we've spoken to someone and are convinced they are legitimate, have fixed any issues we've identified, but are hitting traps because of a single address - if we didn't do this then we'd either have to force the sender to reconfirm their entire database by asking everyone to opt-in again, or we'd have to whitelist the sender, neither are great options - so if we ever do this, then it's done hours/days after the event so as to not tip off anyone on what the trap was).
Because spam traps have to stay secret, we only reference them by a hash value internally (and only I and a select few can convert hash to trap value to do the unsubscribe) and as the messages are received on our trap network the traps themselves are obfuscated to a know value before they are stored in our evidence system.
mailto: means we can't do this at all - a trap can never originate traffic, that's cardinal rule of them, it would be grossly unfair if they did.
> - What do you mean by unsubscribe without contacting me? Isn't HTTP or SMTP just as much contacting me?
Poor use of words from me.
What I meant is that I don't trust messages that have an unsubscribe that does not encode the recipient in it in some way e.g. HTTP method without GET parameters (as this means I would have to do considerably more work to fill out the form) or a mailto:[email protected]?subject=unsubscribe - both to me scream our that there is no automatic connection to a database and that someone on the receiving end has to manually remove the address - which could take weeks, or be never.
One other question. How do you manage domain based reputation vs IP reputation. I'm trying to get by running a service off of Cloud provider IPs and they may rotate from time to time as VPSs rotate. Part of me feels that once my domain has enough reputation it should be fine. It does seem like some of the big providers work like this (Google seems to trust me now from any IP) but it seems that a lot of smaller providers rely on blacklists more as they don't see as much traffic to build reputation. How do blacklists handle domain reputation?
From our perspective, we don't link IP and domains at all at the moment, they are totally independent.
Because spam traps get very different traffic to a real mail server, you can make a lot of assumptions that would be impossible to do on a real mail stream.
So generally speaking, if your domain name isn't in the top million domains (and there are still some exceptions to this - but I'm not going to list them all here) and you hit one of our pristine traps (see my post about the different trap types) with a message containing your domain, that will usually cause the domain to be listed.
Other lists and spam filters will have their own methods for this.
My belief is that Google and Microsoft only use AI for this (which is a bit of a nightmare if it gets it wrong).
Because spam traps get very different traffic to a real mail server, you can make a lot of assumptions that would be impossible to do on a real mail stream.
So generally speaking, if your domain name isn't in the top million domains (and there are still some exceptions to this - but I'm not going to list them all here) and you hit one of our pristine traps (see my post about the different trap types) with a message containing your domain, that will usually cause the domain to be listed.
Other lists and spam filters will have their own methods for this.
My belief is that Google and Microsoft only use AI for this (which is a bit of a nightmare if it gets it wrong).
[deleted]
FWIW, Mailgun has worked pretty well for me in the past as an alternative to handling delivery myself.
You might not be able to go this route if your customers have some expectations about how your email is handled, in which case this recommendation is here for anyone else that might read this and need another thing to try.
You might not be able to go this route if your customers have some expectations about how your email is handled, in which case this recommendation is here for anyone else that might read this and need another thing to try.
Spam blocklists are run by an unaccountable cowboy cult that somehow has managed to consolidate a ton of power simply for the fact that most people who run email inbox services didn't want to deal with the problem of spam, so they were more than willing to just hand over anti-spam "enforcement" to anyone who was allegedly doing "what was best for the internet." There's no check on these people who run these blacklists, and the system they've built is entirely a black box, antithetical to the principles of the open internet. And if you're not a huge corporation that can afford professional management of your email deliverability, good luck – the individuals and small organizations are just out of luck. It's a miserable racket and for what?
If you want to know why there's a new thread each week on HN about why it's impossible to host your own email service, this is why.
If you want to know why there's a new thread each week on HN about why it's impossible to host your own email service, this is why.
Let's not classify all spam blacklists as the same. UCEPROTECT is in a special class of extortionist cowboy, because it's basically just an inaccurate protection racket throwing a wide net across cloud providers who won't play their game. Some other blacklists are updated regularly and only contain IP addresses that have actually sent spam. By contrast, UCEPROTECT3 just lumps ISPs into the list even though an address has never sent spam.
I run a mail server on AWS, and we use some blacklists to drop mail. It's quite effective and that's why people keep using them. A properly curated blacklist is a powerful tool, and more accurate than the machine learning mush that people have come to rely upon.
I run a mail server on AWS, and we use some blacklists to drop mail. It's quite effective and that's why people keep using them. A properly curated blacklist is a powerful tool, and more accurate than the machine learning mush that people have come to rely upon.
UCEPROTECT is just horrible, full stop. Stuff like [1] read like it's written by a 13 year old. Lots more where that came from; as near as I can tell UCEPROTECT is run by a single person who is consistently vitriolic like this to anyone offering even the mildest of criticisms (supposedly multiple people have involved over the years, but their, ehm, distinct colourful personalities are so alike that I'm inclined to think it's just the same person using different aliases).
People have been complaining about this for a very long time; as long as I can remember. In the past they've also just permanently added people's IPs after complaining their IPs were wrongfully listed (not sure if they still do that).
The UCEPROTECT blacklist should be blacklisted by everyone. Yes, we need a blacklist of blacklists.
[1]: http://www.uceprotect.org/cart00neys/2021-001.html
People have been complaining about this for a very long time; as long as I can remember. In the past they've also just permanently added people's IPs after complaining their IPs were wrongfully listed (not sure if they still do that).
The UCEPROTECT blacklist should be blacklisted by everyone. Yes, we need a blacklist of blacklists.
[1]: http://www.uceprotect.org/cart00neys/2021-001.html
I think I understand a lot more about UCEPROTECT now that I've read the message you linked. This is pretty extraordinarily sexist and unprofessional, and you can really get a sense of how much the author despises anyone who criticizes him.
> CEPROTECT3 just lumps ISPs into the list even though an address has never sent spam.
But this is by design[0]
> This blacklist has been created for HARDLINERS. It can, and probably will cause collateral damage to innocent users when used to block email.
And it makes for a perfectly usable blocklist. If you use postfix, the postscreen_dnsbl_threshold and postscreen_dnsbl_sites parameters let you create a simple scoring system:
But it isn't really a problem with uceprotect, it's about how DNSBLs are used.
[0] https://www.uceprotect.net/en/index.php?m=3&s=5 [1] http://www.postfix.org/postconf.5.html#postscreen_dnsbl_site...
But this is by design[0]
> This blacklist has been created for HARDLINERS. It can, and probably will cause collateral damage to innocent users when used to block email.
And it makes for a perfectly usable blocklist. If you use postfix, the postscreen_dnsbl_threshold and postscreen_dnsbl_sites parameters let you create a simple scoring system:
postscreen_dnsbl_threshold = 10
postscreen_dnsbl_sites =
zen.spamhaus.org*5,
bl.spameatingmonkey.net*5,
dnsbl.sorbs.net*4,
bl.spamcop.net*4,
dnsbl-3.uceprotect.net*3
But it isn't really a problem with uceprotect, it's about how DNSBLs are used.
[0] https://www.uceprotect.net/en/index.php?m=3&s=5 [1] http://www.postfix.org/postconf.5.html#postscreen_dnsbl_site...
I also run an email server. I just used https://mxtoolbox.com/ to check my status and yep, UCEPROTECTL3 is the ONLY one to have my IP range listed.
The article sums it up nicely: IP blacklists have their place, however, please don’t use the overarching neighbouring blacklists such as UCEPROTECTL2 and UCEPROTECTL3.
The article sums it up nicely: IP blacklists have their place, however, please don’t use the overarching neighbouring blacklists such as UCEPROTECTL2 and UCEPROTECTL3.
Spamhaus, which (as far as I can tell) is seen as the "most legitimate" blacklist is just as bad, if not worse. UCEPROTECT might be extorting you, but at least you know the name of the game. With Spamhaus, any email sender can get blacklisted for whatever nonsense rules Spamhaus comes up with – many times a set of entrapments they themselves have devised – and they have zero urgency or clarity around resolving it. "Fix the problem" they say. Okay, what's the problem? "Be a better sender." Okay, how? It's like a conversation with Yoda.
To be clear: I DON'T think blacklists are the problem. I think the problem is that a half-dozen major blacklists are controlled by unaccountable organizations who make up rules willy nilly, and privilege major email senders while punishing smaller senders / home hobbyists.
To be clear: I DON'T think blacklists are the problem. I think the problem is that a half-dozen major blacklists are controlled by unaccountable organizations who make up rules willy nilly, and privilege major email senders while punishing smaller senders / home hobbyists.
> antithetical to the principles of the open internet.
No. These blocklists are employed by the actor receiving the email. They have a perfect right, even on "the open internet", to decide that they want to limit who can send them messages.
There are tons of checks on the people who provide those blacklists, in the form of their users complaining about lack of mail delivery and ultimately not using their list anymore. We vote with our wallets, and as a group we have decided that these blocklists are useful.
> and for what?
To make email usable. Full stop.
No. These blocklists are employed by the actor receiving the email. They have a perfect right, even on "the open internet", to decide that they want to limit who can send them messages.
There are tons of checks on the people who provide those blacklists, in the form of their users complaining about lack of mail delivery and ultimately not using their list anymore. We vote with our wallets, and as a group we have decided that these blocklists are useful.
> and for what?
To make email usable. Full stop.
Absolutely agree. This doesn't fall under the principles of the open internet nor anything in the 'net neutrality' arena. You have no right to expect anyone to receive traffic from your server if they choose not to. It's a major pitfall of running your own relay, but it's not unethical.
Unfortunately 50% of the world's mailboxes are hosted by 2 companies. That means the open internet is far as email goes is already dead in practical terms for most people. When they block another host that host no longer has email network access.
> These blocklists are employed by the actor receiving the email.
If I send an email to [email protected], Alice is the recipient, not Bob the sysadmin for example.com.
If I send an email to [email protected], Alice is the recipient, not Bob the sysadmin for example.com.
This is incorrect. The service you are sending the mail to (example.com in this case) is receiving that mail. They are then delivering it to the user, or not, depending on the agreement they have with the user. If example.com and a user of example.com have agreed that example.com is responsible for providing spam filtering, then they would be remiss in delivering the email to their customer.
This is the way that email is designed to work and the fact that users and service providers have choice is a feature, not a bug. If you don't want your service provider to do this type of filtering, that is your choice. But you have absolutely no right to say that Alice can't get the service she wants from example.com because it's inconvenient for you.
Edit: and just to head off the inevitable "but what about if it's at work and I can't change providers", the domain and email accounts belong to your employer, not you. You are welcome to work somewhere else if you don't like how they configure their systems.
This is the way that email is designed to work and the fact that users and service providers have choice is a feature, not a bug. If you don't want your service provider to do this type of filtering, that is your choice. But you have absolutely no right to say that Alice can't get the service she wants from example.com because it's inconvenient for you.
Edit: and just to head off the inevitable "but what about if it's at work and I can't change providers", the domain and email accounts belong to your employer, not you. You are welcome to work somewhere else if you don't like how they configure their systems.
I'd hardly say this is a feature. It's an artifact of the fact that once upon a time computers were expensive, so many people had to share a server with multiple email accounts, and a netop had to run this server. The fact that Alice is beholden to example.com helps nobody but example.com. In an ideal world, Alice would not have to couple her infrastructure with a blocklist and Alice would have the choice of blocklist herself.
Spam is inevitable. Opining about ideal worlds is fun, but it's also irrelevant because that world is gone and it will never come back because it can't.
Alice is not beholden to example.com, she can use any hosting provider she wants. Moving providers isn't even that difficult. It's also totally irrelevant, because again spam is inevitable.
Alice is not beholden to example.com, she can use any hosting provider she wants. Moving providers isn't even that difficult. It's also totally irrelevant, because again spam is inevitable.
no. its because of spam. just because these people are doing a less than perfect job of keeping the screaming hordes out of my inbox doesn't remove any of the blame from the hordes.
It's been years since I last hosted my own mail server, and even then the HAM to SPAM ratio was well over 1:100. The torrent of absolute crap faced at port 25 is unbelievable.
Still better than the alternative
> Or I can leave the current hosting company
Yep, that's the one.
If your hoster doesn't care about spam spreading from their IP space, you should take your mail server elsewhere. There's literally nothing to think about.
And if they do care about this issue, they are likely to be taking steps to remove any of their IP space from the blacklists, without being nudged.
PS.
I've been running a mail server for close to 20 years now and I do blacklist by /24 netblock on the second offense. This doesn't bounce emails though, just tags them as spam. So I had a quick look in the logs and Hetzner, Digital Ocean, OVH and LeaseWeb are all spamming a lot. LayerHost, Colo Crossing, Liquid Web, Host Winds and Servion are also close to the top. Anecdotal data, obviously, so caveat emptor and all that.
Yep, that's the one.
If your hoster doesn't care about spam spreading from their IP space, you should take your mail server elsewhere. There's literally nothing to think about.
And if they do care about this issue, they are likely to be taking steps to remove any of their IP space from the blacklists, without being nudged.
PS.
I've been running a mail server for close to 20 years now and I do blacklist by /24 netblock on the second offense. This doesn't bounce emails though, just tags them as spam. So I had a quick look in the logs and Hetzner, Digital Ocean, OVH and LeaseWeb are all spamming a lot. LayerHost, Colo Crossing, Liquid Web, Host Winds and Servion are also close to the top. Anecdotal data, obviously, so caveat emptor and all that.
I have run my own mail server about as long, run an RBL, and a transactional mail service too. This is a hard line approach and blacklisting a /24 on a second offense that never expires just doesn't work long term but at least you are not completely blocking it and accepting it but as spam.
Lets be real there is spam coming from gmail and hotmail/outlook as well and places like abuseix specifically state they don't block these ranges. So the large providers get excused for clean up because they are too big. Sure blocking colo crossing probably won't get any one to complain, but Digital Ocean is probably going to get some collateral damage. For your own mail server fine, don't accept it, send to spam - but there is a reason real RBL lists are very careful to skip the big providers or make sure they expire. Spamcop always had the best method - expire when the spam stops. Does it keep getting listing? Keep it longer. Rspamd also has a good method where an RBL increases the score. The hard line approach gives gmail and microsoft a large share of the email market and hurts smaller providers when they are not held to the same standards as everyone else. If gmail emails start bouncing when they have a heavy spam hit, then maybe gmail users will change isps and help gmail clean up. These are two trillion dollar companies that also have spam problems.
As far as UCEprotect. Their level1 is actually reasonable, especially for spam traps. The timestamps easily allow for you to find exactly what the spam is from with the smtp response and time frame. Their scanning methods are less so. Dos prevention measures can get you listed there and are not valid. The level2/3 lists are utter shit.
Lets be real there is spam coming from gmail and hotmail/outlook as well and places like abuseix specifically state they don't block these ranges. So the large providers get excused for clean up because they are too big. Sure blocking colo crossing probably won't get any one to complain, but Digital Ocean is probably going to get some collateral damage. For your own mail server fine, don't accept it, send to spam - but there is a reason real RBL lists are very careful to skip the big providers or make sure they expire. Spamcop always had the best method - expire when the spam stops. Does it keep getting listing? Keep it longer. Rspamd also has a good method where an RBL increases the score. The hard line approach gives gmail and microsoft a large share of the email market and hurts smaller providers when they are not held to the same standards as everyone else. If gmail emails start bouncing when they have a heavy spam hit, then maybe gmail users will change isps and help gmail clean up. These are two trillion dollar companies that also have spam problems.
As far as UCEprotect. Their level1 is actually reasonable, especially for spam traps. The timestamps easily allow for you to find exactly what the spam is from with the smtp response and time frame. Their scanning methods are less so. Dos prevention measures can get you listed there and are not valid. The level2/3 lists are utter shit.
I think the issue is that, short of customers running their own BGP routers and hosting their own subnet, anything smaller than a /24 assigned by the ISP is completely transparent to BGP databases and thus spam filters / IP reputation databases.
Any hosting company where some random person can buy a VM for $5 on a credit card has this problem. Mostly I feel sorry for the support and staff at the hosting companies who do not have the time/manpower/resources to deal with it properly, and this is an intentional business decision by the people who own and run the companies.
It's a race to scraping the bottom of the barrel on per customer profit margin and pricing.
It's a race to scraping the bottom of the barrel on per customer profit margin and pricing.
When I first started monitoring spam connections years ago, it was almost always home cable+internet providers. They seem to have gotten the message and cleaned up thier acts.
As of last night, the number one source of spam for both the personal and company servers I maintain is Digital Ocean.
Which is a shame, because otherwise I'm a happy Digital Ocean customer. But because of this, I would never move any of my commercial projects there.
As of last night, the number one source of spam for both the personal and company servers I maintain is Digital Ocean.
Which is a shame, because otherwise I'm a happy Digital Ocean customer. But because of this, I would never move any of my commercial projects there.
> home cable+internet providers. They seem to have gotten the message and cleaned up thier acts.
By blocking outbound connection to port 25 outright. I don't want my VPS hosting company to "clean up" this way.
By blocking outbound connection to port 25 outright. I don't want my VPS hosting company to "clean up" this way.
> I don't want my VPS hosting company to "clean up" this way.
Most ISPs submit their domestic ranges voluntarily to certain blocklists, e.g. the Spamhaus PBL.
I used to have a bunch of blocklist services in my MTA configuration, with different lists having different weights and so on. Eventually it became clear to me that nearly all true spam that hit any one list would also hit Spamhaus Zen. So I scrapped my list of lists, and relied entirely on Zen. My life became easier.
Note: Zen includes the PBL; so if you block using Zen, then you will block mail from domestic ranges. Zen is pretty safe (unless you are expecting mail from domestic ranges). It's safest to use Zen as part of a scoring system, with something like Spamassassin to add heuristic content scoring.
Most ISPs submit their domestic ranges voluntarily to certain blocklists, e.g. the Spamhaus PBL.
I used to have a bunch of blocklist services in my MTA configuration, with different lists having different weights and so on. Eventually it became clear to me that nearly all true spam that hit any one list would also hit Spamhaus Zen. So I scrapped my list of lists, and relied entirely on Zen. My life became easier.
Note: Zen includes the PBL; so if you block using Zen, then you will block mail from domestic ranges. Zen is pretty safe (unless you are expecting mail from domestic ranges). It's safest to use Zen as part of a scoring system, with something like Spamassassin to add heuristic content scoring.
I'm not sure that home IPs have cleaned up their act, I think it is that no large email host accepts their email any longer so spammers have stopped sending from there.
I have a similar experience: Some hosting providers / AS host shady stuff and I understand that VPS ranges end up on block lists quite easily.
I only block AS 4134 and AS 4837, some AS that host services like shodan, and aggressive crawlers like semrush.
Anything that sends packets to my server get ratelimited quickly. Still barely noticeable for occasional human interaction. I also started with /24, but I am now up to /12.
PS. By the way, has anyone ever seen [email protected] in the logs? It shows up almost on a weekly basis as RCPT TO address from literally all over the world.
I only block AS 4134 and AS 4837, some AS that host services like shodan, and aggressive crawlers like semrush.
Anything that sends packets to my server get ratelimited quickly. Still barely noticeable for occasional human interaction. I also started with /24, but I am now up to /12.
PS. By the way, has anyone ever seen [email protected] in the logs? It shows up almost on a weekly basis as RCPT TO address from literally all over the world.
> By the way, has anyone ever seen [email protected] in the logs? It shows up almost on a weekly basis as RCPT TO address from literally all over the world.
Yes, I see it all the time in my logs.
Yes, I see it all the time in my logs.
>/24 netblock on the second offense.
That seems excessive and abusive, I am not aware of any commercial ISP that is giving out /24 anymore. /29 is most common, I had to practically beg to get a /28 so what is the justification for banning an /24???
That seems excessive and abusive, I am not aware of any commercial ISP that is giving out /24 anymore. /29 is most common, I had to practically beg to get a /28 so what is the justification for banning an /24???
Block registers don't deal with "justification".
If they did, what would be Google's justification for blocking senders that send too few emails?
If they did, what would be Google's justification for blocking senders that send too few emails?
I wish I knew. I have a user google keeps blocking for spam (not even marking, outright blocking) while at the same time their reporting tool say the user doesn't send enough e-mail to google for any data to be displayed (the user's domain generates less than ten e-mails per day, many/most of which not to google).
> That seems excessive and abusive
Good thing it's my own mail server then, isn't it?
The practical reason is that virtually all Whois lookups of offending IPs return blocks of /24 or larger, so that's a reasonable default. Besides, as I said, this doesn't result in a "ban", just tags emails as spam and passes them through. At my scale an occasional false positive is not a big deal.
Good thing it's my own mail server then, isn't it?
The practical reason is that virtually all Whois lookups of offending IPs return blocks of /24 or larger, so that's a reasonable default. Besides, as I said, this doesn't result in a "ban", just tags emails as spam and passes them through. At my scale an occasional false positive is not a big deal.
If you start a new mail server in 2022 (or migrate from a previous address), you have to apply to be whitelisted by outlook.com and the many domains owned by Microsoft. They no longer accept mail from servers they haven't seen before. Companies such as google, microsoft, and facebook would rather that email died, and are actively working to destroy it through neglect, so that people will shift their messaging to proprietary networks they happen to own. Email has problems - spam being one of them, but it's as potentially important as IP routing itself, and we should work to preserve it.
Another fascinating fact is that forwarding Outlook spam to [email protected] will get you shitcanned ... as a spammer!
You can't make this up. They pass all abuse reports throuh their spam filter and this affects your server's reputation. Then at some point they start rejecting emails from you with a 5xx code that stands for "your IP is on our blacklist". Contacting "Hotmail sender support" will yield "We see no problems on our end, kindly fuck off". You have to then go through a ritual of summoning a manager, taking anywhere from 2 to 5 days, at which point they will say "we put a mitigation in place, will take up to 48 hours to propagate", but it works right away. If it doesn't, it means that they mistyped your server address and the next chance to ask them to fix this will be, you guessed it, in 48 hours. So a week of bounces in total.
Ran into this 3 times in a course of a year, then took a closer look at the logs and - yep - in all three cases it was triggered by sending a report to [email protected].
It's just such a magnifecent clusterfuck it's jaw-dropping.
You can't make this up. They pass all abuse reports throuh their spam filter and this affects your server's reputation. Then at some point they start rejecting emails from you with a 5xx code that stands for "your IP is on our blacklist". Contacting "Hotmail sender support" will yield "We see no problems on our end, kindly fuck off". You have to then go through a ritual of summoning a manager, taking anywhere from 2 to 5 days, at which point they will say "we put a mitigation in place, will take up to 48 hours to propagate", but it works right away. If it doesn't, it means that they mistyped your server address and the next chance to ask them to fix this will be, you guessed it, in 48 hours. So a week of bounces in total.
Ran into this 3 times in a course of a year, then took a closer look at the logs and - yep - in all three cases it was triggered by sending a report to [email protected].
It's just such a magnifecent clusterfuck it's jaw-dropping.
> They no longer accept mail from servers they haven't seen before.
Interesting. This tracks with my experiences too, but is there a good source for this that I can reference in the future?
I got out of hosting email last year entirely because of intractable deliverability problems with the big three: Google, Microsoft, and Comcast. Comcast's issues mostly appeared intermittent and the result of incompetence. Google and Microsoft have clearly been competing to see who can kill small email service providers the fastest.
Interesting. This tracks with my experiences too, but is there a good source for this that I can reference in the future?
I got out of hosting email last year entirely because of intractable deliverability problems with the big three: Google, Microsoft, and Comcast. Comcast's issues mostly appeared intermittent and the result of incompetence. Google and Microsoft have clearly been competing to see who can kill small email service providers the fastest.
I contacted Microsoft, actually received an answer from them and got an IP unblocked.
Google can be tough though. If their Algorithms don't like you, good look.
I was responsible for a new IP/Email setup and while all other relevant providers liked our mail and Google Postmaster tools didn't show anything bad, all our mail went to Spam in GMail. I found out why when even mails from completely different senders would usually go to Inbox but hit Spam once our Website or email was included:
Despite showing it as "high reputation", GMail must have considered it as utter trash. Changed to a different, older domain and all has been fine since.
Google can be tough though. If their Algorithms don't like you, good look.
I was responsible for a new IP/Email setup and while all other relevant providers liked our mail and Google Postmaster tools didn't show anything bad, all our mail went to Spam in GMail. I found out why when even mails from completely different senders would usually go to Inbox but hit Spam once our Website or email was included:
Despite showing it as "high reputation", GMail must have considered it as utter trash. Changed to a different, older domain and all has been fine since.
How did you contact them? I had problems getting mail to MS years ago and at the time wasn't able to get anyone to do anything or even respond. At some point later the problem went away, though.
I contacted them via https://support.microsoft.com/en-US/supportrequestform/8ad56...
It's actually not that big of a problem. I moved to a new server with a different IP last year, and also the year before, and as long as you have SPF, DMARC, DKIM and reverse DNS set up, generally everything works. I think I just had to got whitelisted from one or two.
A few years ago I did have major problems with hotmail and icloud, but recently it doesn't seem to be an issue. I think I didn't have DKIM set up when I was having problems. I have noticed that google will auto-junk your email if you don't have SPF, DMARC and DKIM set up, and you must set DMARC to "reject".
A few years ago I did have major problems with hotmail and icloud, but recently it doesn't seem to be an issue. I think I didn't have DKIM set up when I was having problems. I have noticed that google will auto-junk your email if you don't have SPF, DMARC and DKIM set up, and you must set DMARC to "reject".
Fortunately for the author, I haven’t noticed any email servers that use the UCEPROTECTL3 RBLs to reject mail—which is to say, I’ve noticed some servers I administer end up on UCEPROTECTL3 incidentally and it has never caused a delivery problem.
On the other hand, some VPS providers are still allocating multiple customers to the same IPv6 /64 using SLAAC by default, and this will make it impossible to deliver mail on IPv6 since reputable RBLs always blacklist the whole /64.
As far as the argument about spamming being immoral but not illegal, I’ve never seen a reputable ISP that didn’t prohibit unsolicited bulk email in their terms of use, so the grounds for reporting it is that a customer is violating the terms that they agreed to follow when they signed up.
And to answer the question of whether or not RBLs are useful: in my experience, yes, they are quite useful. The biggest problem I’ve noticed with them is not typically false positives on small providers, but false negatives on giant companies like Google who cannot ever end up on an RBL because they process so much mail but don’t do a good enough job of preventing their servers from being used to send spam.
On the other hand, some VPS providers are still allocating multiple customers to the same IPv6 /64 using SLAAC by default, and this will make it impossible to deliver mail on IPv6 since reputable RBLs always blacklist the whole /64.
As far as the argument about spamming being immoral but not illegal, I’ve never seen a reputable ISP that didn’t prohibit unsolicited bulk email in their terms of use, so the grounds for reporting it is that a customer is violating the terms that they agreed to follow when they signed up.
And to answer the question of whether or not RBLs are useful: in my experience, yes, they are quite useful. The biggest problem I’ve noticed with them is not typically false positives on small providers, but false negatives on giant companies like Google who cannot ever end up on an RBL because they process so much mail but don’t do a good enough job of preventing their servers from being used to send spam.
Outlook properties seemingly use uceprotect lists. Outlook support even has a page for getting off their block list, but I could never get a reply.
And my hosting provider basically said "we're warning against using our servers for outbound email", which amounts to "we're not gonna do anything about your bad neighbors that landed you on the uceprotect lvl2&3 lists"
Been running my own email for 18 years, and outlook is the only place that i have problems delivering to.
And my hosting provider basically said "we're warning against using our servers for outbound email", which amounts to "we're not gonna do anything about your bad neighbors that landed you on the uceprotect lvl2&3 lists"
Been running my own email for 18 years, and outlook is the only place that i have problems delivering to.
Outlook definitely engage in aggressive blocking of netranges, but I’m pretty sure they don’t use UCEPROTECT to do this since I’ve seen servers caught in Outlook netrange blocks despite all the public RBLs (including UCEPROTECTL3) showing no problem.
Just to be sure, you are using the Outlook form[0] and not the Office365 form? Have you signed up for SNDS?[1] All these hoops are pretty dumb, and I don’t think they have updated any of these systems since the mid-2000s so they are also pretty terrible to use, but here we are.
My limited personal experience with Outlook’s removal process is that it’s just slow and confusing (sometimes 12 hours for a response, sometimes the response emails are empty). The last time I had to do this, their TLS certificates had expired (and they use HSTS), so I could not even get into SNDS for half the day until someone there finally noticed and deployed new certificates.
[0] https://support.microsoft.com/supportrequestform/8ad563e3-28...
[1] https://sendersupport.olc.protection.outlook.com/snds/
Just to be sure, you are using the Outlook form[0] and not the Office365 form? Have you signed up for SNDS?[1] All these hoops are pretty dumb, and I don’t think they have updated any of these systems since the mid-2000s so they are also pretty terrible to use, but here we are.
My limited personal experience with Outlook’s removal process is that it’s just slow and confusing (sometimes 12 hours for a response, sometimes the response emails are empty). The last time I had to do this, their TLS certificates had expired (and they use HSTS), so I could not even get into SNDS for half the day until someone there finally noticed and deployed new certificates.
[0] https://support.microsoft.com/supportrequestform/8ad563e3-28...
[1] https://sendersupport.olc.protection.outlook.com/snds/
I've signed up for SNDS, but there's no mitigation to be done from there, it's purely informative. And that form looks like one I've filled out to no avail.
My email server is very small, so it's not a big deal... But thanks for the links.
My email server is very small, so it's not a big deal... But thanks for the links.
UCEPROTECT is a scam. Blocking innocent people and asking them for money has nothing to do with security.
The good thing is that most email servers do not use it because it's just bad.
The bad thing is that Hotmail uses it (or at least was at until recently). It does mean that, as a Hotmail user, there's legitimate email that you won't receive.
I do wonder if the guy behinf this scam is randomly blocking whole ip ranges to make a living, having enough people agreeing to the racket.
I do wonder if the guy behinf this scam is randomly blocking whole ip ranges to make a living, having enough people agreeing to the racket.
This has literally just recently (re-)become an issue for me because after a good year or so of not being blocked, several emails from my server started getting filtered as spam by both Google and then just being outright blocked by both Google and Microsoft. When I got through Microsoft's steps to unblock, after several days and steps, they send me the message that actually my IP is not blocked.
Google's recent message was more helpful - apparently forwarding emails from the accounts I setup for my kids to my wife's Gmail account triggered some obscure rule that ruined my server's reputation with Google, and I think Google & Microsoft collaborate because the issues cropped up within a week of each other.
The interesting thing was I discovered the Outlook issue by trying to reply to an email sent from an Outlook customer. Yes, my reply to an Outlook customer's email to me was blocked because of my server's reputation.
To be clear, I run no mailing lists nor solicit any business with my email server. I use it for personal use only and my consulting work which involves know contacts. The forwarding I spoke of before is solely to our own personal accounts.
I use Mail-In-A-Box, for what it's worth, on a Linode VPS.
Google's recent message was more helpful - apparently forwarding emails from the accounts I setup for my kids to my wife's Gmail account triggered some obscure rule that ruined my server's reputation with Google, and I think Google & Microsoft collaborate because the issues cropped up within a week of each other.
The interesting thing was I discovered the Outlook issue by trying to reply to an email sent from an Outlook customer. Yes, my reply to an Outlook customer's email to me was blocked because of my server's reputation.
To be clear, I run no mailing lists nor solicit any business with my email server. I use it for personal use only and my consulting work which involves know contacts. The forwarding I spoke of before is solely to our own personal accounts.
I use Mail-In-A-Box, for what it's worth, on a Linode VPS.
How did you contact Google and Microsoft regarding deliverability issues?
Microsoft:
1. Set up on https://sendersupport.olc.protection.outlook.com/snds/
2. Read everything at https://sendersupport.olc.protection.outlook.com/pm/
3. Make sure everything is fixed, then use the link to the form hidden under Troubleshooting > "Sender services, tools, and issue submission" (the link's label is "here") to contact support. Make sure all fields are provided, including a website. It may take a few days to get a response. You may have to try multiple times until someone actually helps you.
Google:
Pray
1. Set up on https://sendersupport.olc.protection.outlook.com/snds/
2. Read everything at https://sendersupport.olc.protection.outlook.com/pm/
3. Make sure everything is fixed, then use the link to the form hidden under Troubleshooting > "Sender services, tools, and issue submission" (the link's label is "here") to contact support. Make sure all fields are provided, including a website. It may take a few days to get a response. You may have to try multiple times until someone actually helps you.
Google:
Pray
You failed to mention that if you haven't managed to complete this successfully(google also has a similar form: https://postmaster.google.com), google will just mark it as SPAM, but outlook will *silently* drop your mails.
Both google and outlook can drop your mails without delivering them to the recipient, but you can find the rejection error code in your MTA logs. Though it's true that microsoft just rejects everybody and google might not reject you at all, at least outlook has a procedure of sorts these days. I do not see any form at postmaster.google.com that I'm allowed to access.
This is a fair reflection of my experience.
RBLs are useful, but there are a few that are not what they appear to be. The particular one in question, UCEPROTECT is
a) not worth paying
b) should never be used by a production mailserver to block messages.
From the beginning there have been enterprising RBLs that are clearly overbroad, and offer to accept money. The money is not for getting off the list, it is always for something else so as to appear legitimate and a side effect is getting your domain off the list. This model is unethical at best, and is right up there with companies that snail mail over-priced domain renewal notices.
a) not worth paying
b) should never be used by a production mailserver to block messages.
From the beginning there have been enterprising RBLs that are clearly overbroad, and offer to accept money. The money is not for getting off the list, it is always for something else so as to appear legitimate and a side effect is getting your domain off the list. This model is unethical at best, and is right up there with companies that snail mail over-priced domain renewal notices.
Actually no RBL alone should be used for decisive actions such as banning.
It's not UCEPROTECT either, big vendors like Cisco with their IronPorts are just as bad.
It's not UCEPROTECT either, big vendors like Cisco with their IronPorts are just as bad.
My personal experience with UCEPROTECT was that they had blacklisted 2 or 3 IPs in my /24 that were not routed to anything, nor had they ever been routed to anything, a fresh new block from RIPE NCC too.
Of course they offered to unblacklist them in exchange for payment, or wait.
Waited 2 weeks and they dropped off. I've yet to hear about anyone using their DNSBL for anything serious in 2020/2021/2022
I only knew about the listings because a monitoring service emailed me about it.
Of course they offered to unblacklist them in exchange for payment, or wait.
Waited 2 weeks and they dropped off. I've yet to hear about anyone using their DNSBL for anything serious in 2020/2021/2022
I only knew about the listings because a monitoring service emailed me about it.
> a fresh new block from RIPE NCC too
While I personally don't use UCE (and personally think that they're not good at what they're doing), unless you've get that IP range before 2012, I doubt it's a new one. Many spammers do often exploit RIPE's unallocated IPs for their spamming operations (either using BGP hijacking or just asking RIPE nicely for a range), which unfortunately is a perennial problem.
While I personally don't use UCE (and personally think that they're not good at what they're doing), unless you've get that IP range before 2012, I doubt it's a new one. Many spammers do often exploit RIPE's unallocated IPs for their spamming operations (either using BGP hijacking or just asking RIPE nicely for a range), which unfortunately is a perennial problem.
> I can complain to my hosting company and hope they evict the bad user from the network. But then why should my hosting company do so?
The answer to this is the other option, leaving the hosting company. In this manner, every hosting company gets a choice - either they will kick out legal-but-immoral things like spammers, or they will not and rightly lose their above-board customers.
This is essentially how the global e-mail community self-polices by establishing a norm that a host either has to work to exclude bad actors or will get boycotted/excluded for allowing them.
The answer to this is the other option, leaving the hosting company. In this manner, every hosting company gets a choice - either they will kick out legal-but-immoral things like spammers, or they will not and rightly lose their above-board customers.
This is essentially how the global e-mail community self-polices by establishing a norm that a host either has to work to exclude bad actors or will get boycotted/excluded for allowing them.
> The answer to this is the other option, leaving the hosting company. In this manner, every hosting company gets a choice - either they will kick out legal-but-immoral things like spammers, or they will not and rightly lose their above-board customers.
This assumes that the RBL in question is a good actor. It is as bad as the spammers.
> This is essentially how the global e-mail community self-polices by establishing a norm that a host either has to work to exclude bad actors or will get boycotted/excluded for allowing them.
Most of us have ignored UCEPROTECTCTL lists for a long long time as it appears to be a money grab.
This assumes that the RBL in question is a good actor. It is as bad as the spammers.
> This is essentially how the global e-mail community self-polices by establishing a norm that a host either has to work to exclude bad actors or will get boycotted/excluded for allowing them.
Most of us have ignored UCEPROTECTCTL lists for a long long time as it appears to be a money grab.
His whole defense of the hosting company's hypothetical failure to help is bizarre. He considers a lot of things that aren't his concern or even really his business. A real analysis of whether to complain would only consider whether the hosting company will be responsive and whether they are competent to fix the problem.
Why is he considering—with incomplete information—the hosting company's legal options and the severity of the spamming?
Edited to add: He could also consider whether it is his hosting company with the problem or the orgs using UCEPROTECT; although as someone who needs their emails to go through "no matter what" that can be more difficult.
Why is he considering—with incomplete information—the hosting company's legal options and the severity of the spamming?
Edited to add: He could also consider whether it is his hosting company with the problem or the orgs using UCEPROTECT; although as someone who needs their emails to go through "no matter what" that can be more difficult.
This is a complaint about "UCEPROTECT Blacklist Policy LEVEL 3" [0]
It's description is not subtle:
> This blacklist has been created for HARDLINERS. It can, and probably will cause collateral damage to innocent users when used to block email.
So if the mailsystem you are trying to reach employs it, is either experiencing spam levels that justify it's use - OR they made a mistake in using it, if this is the sole reason you are being banned.
The first order of business is of course to complain to your hosting provider. Nobody wants spammers on their networks - but if they do: then this is kind of exactly the reason for this list. The policy describes in detail what made it possible for this netblock to end up on the list, that should be enough for them to take action either pre-emptively or by notifying their offending customer, and if neccessary kicking them off the network.
The next thing you can do, instead of paying for whitelisting, ist to contact the mailserver-admin at system you are trying to deliver mail to. This can be a bit of a hassle - seeing that your mailserver just got blocked - but it usually works. The same way systems don't want to receive SPAM they also don't want to overblock, after all they want their users to receive emails as well. If you are the mail admin of a sending system and you're reaching out to the receiving system this is usually a pretty good indicator that you don't want to spam them.
I have had success doing this even at some larger ISPs, where you would expect this to be more difficult.
I very much enjoy these blocklists - simple, transparent. Loads better than the kafkaesk black holes that are the major mail providers who barely care, and who do not give you easy recourse if you are mistakenly blocked.
[0] https://www.uceprotect.net/en/index.php?m=3&s=5
It's description is not subtle:
> This blacklist has been created for HARDLINERS. It can, and probably will cause collateral damage to innocent users when used to block email.
So if the mailsystem you are trying to reach employs it, is either experiencing spam levels that justify it's use - OR they made a mistake in using it, if this is the sole reason you are being banned.
The first order of business is of course to complain to your hosting provider. Nobody wants spammers on their networks - but if they do: then this is kind of exactly the reason for this list. The policy describes in detail what made it possible for this netblock to end up on the list, that should be enough for them to take action either pre-emptively or by notifying their offending customer, and if neccessary kicking them off the network.
The next thing you can do, instead of paying for whitelisting, ist to contact the mailserver-admin at system you are trying to deliver mail to. This can be a bit of a hassle - seeing that your mailserver just got blocked - but it usually works. The same way systems don't want to receive SPAM they also don't want to overblock, after all they want their users to receive emails as well. If you are the mail admin of a sending system and you're reaching out to the receiving system this is usually a pretty good indicator that you don't want to spam them.
I have had success doing this even at some larger ISPs, where you would expect this to be more difficult.
I very much enjoy these blocklists - simple, transparent. Loads better than the kafkaesk black holes that are the major mail providers who barely care, and who do not give you easy recourse if you are mistakenly blocked.
[0] https://www.uceprotect.net/en/index.php?m=3&s=5
> The next thing you can do, instead of paying for whitelisting, ist to contact the mailserver-admin at system you are trying to deliver mail to. This can be a bit of a hassle - seeing that your mailserver just got blocked - but it usually works. The same way systems don't want to receive SPAM they also don't want to overblock, after all they want their users to receive emails as well. If you are the mail admin of a sending system and you're reaching out to the receiving system this is usually a pretty good indicator that you don't want to spam them.
That's nice in theory. In practice, UCEPROTECT level 3 is used by, for example, all Microsoft properties (including hotmail, etc). And UCEPROTECT level 3 lists a lot of netblocks.
So, if you want to send e-mail from, say, Digital Ocean, to any Microsoft managed e-mail domain, you can just forget about it. DO doesn't care, Microsoft very much doesn't care. Good luck trying to contact a postmaster there!
That's nice in theory. In practice, UCEPROTECT level 3 is used by, for example, all Microsoft properties (including hotmail, etc). And UCEPROTECT level 3 lists a lot of netblocks.
So, if you want to send e-mail from, say, Digital Ocean, to any Microsoft managed e-mail domain, you can just forget about it. DO doesn't care, Microsoft very much doesn't care. Good luck trying to contact a postmaster there!
> My hosting company is competitively priced, is fast, and has served me well for many years.
... attributes which they achieve by (1) selling services to anyone and anyone and (2) not dedicating any resources to fighting spam.
So you got what you pay for.
... attributes which they achieve by (1) selling services to anyone and anyone and (2) not dedicating any resources to fighting spam.
So you got what you pay for.
I don't want my hosting company to dictate what I can do and what I can't do. I don't spam, but I won't hold my moral standard to everyone else.
Your hosting company is free to allow all the spam they want to leave their network and every other ISP on the planet is free to drop all traffic from your irresponsible hosting company's IP space. freedom sure is nice.
I and I alone is responsible for my act. It is a slippery downward slope once you start outsourcing responsibility.
When your act is choosing and giving money to a host that supports and enables spammers and attackers yes, your are responsible for that. Blacklists can be outsourced, but many ISPs maintain their own internal blacklists as well. ISPs can also subscribe to a 3rd party blacklist, but still whitelist specific blacklisted IPs they know aren't going to cause them problems. There are plenty of blacklists that ISPs choose not to use because they block too much to be useful. ISPs can also use blacklists not to block senders, but to greylist them or give them higher levels of scrutiny before deciding what do accept or block.
In the end, even with 3rd party blacklists networks still maintain control over what incoming traffic/mail they block or allow.
In the end, even with 3rd party blacklists networks still maintain control over what incoming traffic/mail they block or allow.
You missed my point. I don't want my network to censor traffic. It is email traffic today, it could be web traffic tomorrow. If this trend holds, we would all live in walled gardens, and cede our power to the gate keepers.
Preventing spammers and hackers from using your network isn't censorship though. If you want people to let you into their homes, you have to make sure you don't keep lighting fires and smashing up their furniture.
Internet censorship is a real issue, but blocking hackers and spammers are not an example of internet censorship gone wrong. ISPs black spam over email. They block malicious web traffic too. Every network gets to decide what to accept or reject. It's about as fair a system as you could ask for.
Internet censorship is a real issue, but blocking hackers and spammers are not an example of internet censorship gone wrong. ISPs black spam over email. They block malicious web traffic too. Every network gets to decide what to accept or reject. It's about as fair a system as you could ask for.
> If you want people to let you into their homes, you have to make sure you don't keep lighting fires and smashing up their furniture.
You analogy is flawed. In the case of IP blacklisting, it is an intermediary preventing legitimate email delivery on shaky ground. My correspondent want the mail; yet she never got to see it. This is censorship.
You analogy is flawed. In the case of IP blacklisting, it is an intermediary preventing legitimate email delivery on shaky ground. My correspondent want the mail; yet she never got to see it. This is censorship.
> it is an intermediary preventing legitimate email delivery on shaky ground.
The 3rd party hosting the blacklist is an intermediary, but the network operator on the receiving end isn't. They still have total control over what lists to subscribe to, how they'll be used, and what to block or allow. That's not censorship. As a network operator it is there right to block or accept whatever traffic they want. If you, as a customer/end user don't like how they run their network you're free to operate your own or choose to give your money to someone else.
The 3rd party hosting the blacklist is an intermediary, but the network operator on the receiving end isn't. They still have total control over what lists to subscribe to, how they'll be used, and what to block or allow. That's not censorship. As a network operator it is there right to block or accept whatever traffic they want. If you, as a customer/end user don't like how they run their network you're free to operate your own or choose to give your money to someone else.
So you think as long as the user can choose to leave, the platform is no an intermediary. I strongly disagree. Even if the user theoretically can choose, we have to consider the asymmetric information, high switching cost, and monopoly power that are so prevalent nowadays. I choose not to use the big email providers, but most people can't do the same.
> So you think as long as the user can choose to leave, the platform is no an intermediary. I strongly disagree. Even if the user theoretically can choose, we have to consider the asymmetric information, high switching cost, and monopoly power that are so prevalent nowadays.
Some platforms rise to that level I think. Youtube is one example. Email doesn't really have that problem. Anyone can install an email server, and there are a huge amount of options (many of them "free") for those who don't want to run their own mail servers. Email is still a competitive space with different companies offering specialized services targeting people who want privacy, low cost, mass mailing capability, tracking features, etc. Email itself has a ton of competing communication protocols. There are plenty of folks who prefer to communicate over chat or by text. You can even keep an old email address and forward anything sent there to another one or use a separate email address for sending, but get replies at your old address. Switching email addresses is about as easy as it gets.
Some platforms rise to that level I think. Youtube is one example. Email doesn't really have that problem. Anyone can install an email server, and there are a huge amount of options (many of them "free") for those who don't want to run their own mail servers. Email is still a competitive space with different companies offering specialized services targeting people who want privacy, low cost, mass mailing capability, tracking features, etc. Email itself has a ton of competing communication protocols. There are plenty of folks who prefer to communicate over chat or by text. You can even keep an old email address and forward anything sent there to another one or use a separate email address for sending, but get replies at your old address. Switching email addresses is about as easy as it gets.
The author has some oversimplifications which would be worth addressing.
First, there actually does exist a strata of spam which is plainly illegal. All the phishing spam and all the messages that claim to be from networks and/or addresses that they are not, for example, are illegal. The problem is that it's not enforceable.
Second, sending unsolicited messages when you do not have the permission of the sender is unambiguously wrong. Large sources of spam get around this by mixing bad messages in with good ones. This is why we get tons and tons of spam from Gmail, from Outlook.com, from Sendgrid, and so on, even though they really should know better.
The point is that your "bad neighborhood" doesn't just have regular spammers - it's almost certain there are illegal and egregious spammers that your ISP is doing nothing about. How do we know this to be the case? Because many of the blocklists also run honeypot email addresses. If email addresses get harvested and someone sends spam to these addresses, you can be 100% certain that no permission was ever given, so the behavior that leads to this is definitely wrong.
ISPs make too much money to punish anyone but the worst of their clients, and that's definitely a factor that contributes to the affordability of the author's ISP.
However, the author left out one big, simple, obvious option: pay significantly less money than the cost of the blocklist extortion to smarthost through an ISP that has good email reputation. You get what you pay for, so when you save on your ISP, don't be surprised if you have to pay a little more to make up for their shortcomings.
First, there actually does exist a strata of spam which is plainly illegal. All the phishing spam and all the messages that claim to be from networks and/or addresses that they are not, for example, are illegal. The problem is that it's not enforceable.
Second, sending unsolicited messages when you do not have the permission of the sender is unambiguously wrong. Large sources of spam get around this by mixing bad messages in with good ones. This is why we get tons and tons of spam from Gmail, from Outlook.com, from Sendgrid, and so on, even though they really should know better.
The point is that your "bad neighborhood" doesn't just have regular spammers - it's almost certain there are illegal and egregious spammers that your ISP is doing nothing about. How do we know this to be the case? Because many of the blocklists also run honeypot email addresses. If email addresses get harvested and someone sends spam to these addresses, you can be 100% certain that no permission was ever given, so the behavior that leads to this is definitely wrong.
ISPs make too much money to punish anyone but the worst of their clients, and that's definitely a factor that contributes to the affordability of the author's ISP.
However, the author left out one big, simple, obvious option: pay significantly less money than the cost of the blocklist extortion to smarthost through an ISP that has good email reputation. You get what you pay for, so when you save on your ISP, don't be surprised if you have to pay a little more to make up for their shortcomings.
There is a typo in their title. If intentional please instead consider words like block reject and deny for the people that do not speak English as a first language.
I've dealt with real time blocklists as long as they have existed. They are not going away any time soon. I agree that the paid exception lists are a bit shady but I also see the validity of their methods of temporarily punishing everyone on a hosts network to put pressure on the ISP/platform provider to police it's own network and remove spammers. The best one can do today aside from securing ones own server is to research an ISP's IP space ahead of time to see how dirty they are. There are plenty of providers that cleaned up their act some time ago. Linode is a great example of change. New accounts can't even send email unless they open a ticket and prove they made some effort to comply with can-spam. More providers need to follow that example so that we don't run into this problem of dirty networks that real time block-lists like UceProtect have listed. It's an imperfect solution to an old ugly problem.
I've dealt with real time blocklists as long as they have existed. They are not going away any time soon. I agree that the paid exception lists are a bit shady but I also see the validity of their methods of temporarily punishing everyone on a hosts network to put pressure on the ISP/platform provider to police it's own network and remove spammers. The best one can do today aside from securing ones own server is to research an ISP's IP space ahead of time to see how dirty they are. There are plenty of providers that cleaned up their act some time ago. Linode is a great example of change. New accounts can't even send email unless they open a ticket and prove they made some effort to comply with can-spam. More providers need to follow that example so that we don't run into this problem of dirty networks that real time block-lists like UceProtect have listed. It's an imperfect solution to an old ugly problem.
Yeah except we've had this recently with Linode's IP range landing on that exact blacklist and being blocked by Microsoft and other major mail providers, knocking out our ability to send to huge chunks of our customers. I've had to get the mail server moved off Linode to another hosting company as paying the ransom fee did nothing. UceProtect does seem at least as morally dubious if not more so that the spammers it alleges to protect against.
Which other major provider uses this list? Uceprotect is attempting to racket me from time to time. While I'm blocked i try different providers to guess the ones using this, I only found Hotmail blocking me.
Unfortunately we were being blocked by all Microsoft mail services so outlook.com but also anyone using hosted 365, exchange etc. which in our case meant a lot of our enterprise and public sector customers. Also NHS.net mail and a lot of large hospital groups in Europe (APHP etc.)
Ditto here. It is ironic because I actually recommended outlook.com over gmail.com for my non-techie friends, because outlook.com was more lenient, at least 2 years ago. Gmail is using some invisible reputation crap that shovel my emails to the jink folder from time to time. Now outlook.com just did the one-up and start using UCEPROTECTL2/3.
That's good to know. I never thought about exchange/365, that's a really good point.
What hosts are best for running a mail server? I have the same issue.
That's a tough question to answer. This is very much a moving target. I suppose if I were to generalize an answer it would be something to the effect of
- A dedicated server provider that has been around for a while and has a strict AUP and is known to enforce it.
- A hosting provider that is not entry-level in cost. Spammers gravitate towards cheap throw-away ephemeral solutions.
- A hosting provider that verifies identity of its customers. e.g. Dunn and Bradstreet lookup for commercial customers. Video conference meeting for individual customers and commercial customers and that have mutually signed contracts.
Short of that if you just want to use a VPS provider then I would look up their AS number of a prospective provider, get all their CIDR blocks and start validating their IP addresses against the numerous RBL/RSL sites. AFAIK there is not a good database of this. Good RBL/RSL sites will remove listings after a week or two. One could even open a ticket with a VPS provider and state your intentions to run a mail server, explain your process to deal with spam and ask for an IP from a clean subnet.
- A dedicated server provider that has been around for a while and has a strict AUP and is known to enforce it.
- A hosting provider that is not entry-level in cost. Spammers gravitate towards cheap throw-away ephemeral solutions.
- A hosting provider that verifies identity of its customers. e.g. Dunn and Bradstreet lookup for commercial customers. Video conference meeting for individual customers and commercial customers and that have mutually signed contracts.
Short of that if you just want to use a VPS provider then I would look up their AS number of a prospective provider, get all their CIDR blocks and start validating their IP addresses against the numerous RBL/RSL sites. AFAIK there is not a good database of this. Good RBL/RSL sites will remove listings after a week or two. One could even open a ticket with a VPS provider and state your intentions to run a mail server, explain your process to deal with spam and ask for an IP from a clean subnet.
Can you recommend any providers with these verification methods? Apologies but my search turns up no results.
On the very low end of that spectrum of managed/dedicated server providers would be Rackspace but I have heard it may not be as good there any more. On the high end of the spectrum would be IBM but that is very expensive. In between are various managed server providers. Just look for managed hosting and what you would actually want to research is who their customers are. Or look at it the other way around. Find medium sized companies and research who their managed hosting providers are.
Another method would be to look at colocation providers. They can give you blocks of IP space. On the low end would be Hurricane Electric if you are on the west coast. A step up from there would be Sonic. There are a myriad of colocation providers that scale up in price from there. If you wanted a presence in many locations then Equinix may be a good option though they can be expensive when you are first starting out. All of these providers have partnerships with network providers and can help get you peering arrangements, address space and much more.
Another method would be to look at colocation providers. They can give you blocks of IP space. On the low end would be Hurricane Electric if you are on the west coast. A step up from there would be Sonic. There are a myriad of colocation providers that scale up in price from there. If you wanted a presence in many locations then Equinix may be a good option though they can be expensive when you are first starting out. All of these providers have partnerships with network providers and can help get you peering arrangements, address space and much more.
It's "blacklist" in most languages though so the non-standard "block list" would be probably more confusing to non-native speakers. Anyway they clearly meant blacklist.
Thanks, I fixed the typo.
Blocking outgoing port 25 unless going through an extra step is IMHO against the spirit of the internet, and would make operating a personal email server harder than necessary. The world of email is already too centralized as it is.
Blocking outgoing port 25 unless going through an extra step is IMHO against the spirit of the internet, and would make operating a personal email server harder than necessary. The world of email is already too centralized as it is.
against the spirit of the internet
I completely agree. I believe the commonly used phrase is and this is why we can't have nice things. The open and decentralized internet is also open to people with ill intent. We can solve things with technical solutions, monetary solutions and/or legislative solutions. All of these are double edged swords and have varying degrees of effectiveness and unintended side effects in my opinion.
I don't really know what the right solution should have been that would have made everyone happy minus the people with ill intent.
I completely agree. I believe the commonly used phrase is and this is why we can't have nice things. The open and decentralized internet is also open to people with ill intent. We can solve things with technical solutions, monetary solutions and/or legislative solutions. All of these are double edged swords and have varying degrees of effectiveness and unintended side effects in my opinion.
I don't really know what the right solution should have been that would have made everyone happy minus the people with ill intent.
I guess another option is to sue the blacklist operator for inappropriately including him. Or sending some legal sounding letter threatening suit.
I found some old lawsuits [0] that got injunctions and even damages awarded from being included in black lists.
I hate how the big tech customer nonsupport is bleeding into small firms attitude. “Sorry, you did nothing wrong but might one day so get fucked.” is really not something that should happen very much.
[0] https://www.techdirt.com/articles/20051228/1349229.shtml
I found some old lawsuits [0] that got injunctions and even damages awarded from being included in black lists.
I hate how the big tech customer nonsupport is bleeding into small firms attitude. “Sorry, you did nothing wrong but might one day so get fucked.” is really not something that should happen very much.
[0] https://www.techdirt.com/articles/20051228/1349229.shtml
What would be nice is if there could be whitelists as well, or a blacklist that additionally keeps count of positive interactions.
I've had an IP for about a decade that never once sent spam, but has ended up on blacklists from time to time (hosting EICAR on a web server apparently gets your mail server banned, 15-year-old me found out). SPF nicely says that this IP is supposed to be sending email for this domain. I don't think I'm on blacklists anymore, but email still ends up in spam folders nine out of ten times. People are giving off signals that my messages aren't spam all the time (it's my personal email server).
Years of non-spam emails count for nothing whereas a single spam mail from an adjacent IP can get you on such a list. Somehow it's a bit imbalanced.
I've had an IP for about a decade that never once sent spam, but has ended up on blacklists from time to time (hosting EICAR on a web server apparently gets your mail server banned, 15-year-old me found out). SPF nicely says that this IP is supposed to be sending email for this domain. I don't think I'm on blacklists anymore, but email still ends up in spam folders nine out of ten times. People are giving off signals that my messages aren't spam all the time (it's my personal email server).
Years of non-spam emails count for nothing whereas a single spam mail from an adjacent IP can get you on such a list. Somehow it's a bit imbalanced.
You should check dnswl.org
That is the biggest whitelist there is.
Blacklisting an ISP's ASN or entire network range because the blacklist creator set an arbitrary threshold of acceptable number of spam activity from 1 or more IPs. I'm really not sure about the legal aspects here, but there are so many practices that blatantly approach extortion. And it's skillfully done in the name of "online etiquette" or a "safe internet".
Obviously, legitimate use-cases exist for such services, provided that they are operated faithfully by people/entities with some level of credibility.
Whether or not Spam is legal, many (or most) service providers list it as prohibited in their SLA (Service-Level Agreement). Depending on the type of setup they run--the clientele; company focus/prioritization of ROI--they forward complaints to the account owners, with the understanding that the activity must be stopped. Again, this depends on the service provider, especially as you start thinking on a more granular level. E.g. they can have automated systems handling much of this, or they may have employees handle the communications and provide help to their clients in thwarting such activity. Generally, though, the account owners need to balance the security with compatibility--or the uptime and short-term performance of their services.
Ultimately, however, some of these spam block-lists--UCEPROTECTL3 in particular--are blatantly using a sledge-hammer approach for little benefit to any of their users, in the hopes of spurring controversy; or fear that leads to customers; or outrage from customers toward the service providers. In the end, the intention is purely to get paid, either by clients who sign up for their service or by the internet service providers that can't wait multiple days or weeks until their network gets de-listed. Their accusatory and aggressive language conveys to people that they must be associated with either some amateur company or one that uses, and knowingly profits from, nefarious practices and shady characters if they somehow ended up on their list.
Depending on where you stand, it can be seen as deplorable or underhanded behavior maybe even rising to the level extortion, or it can be considered a shrewd business tactic.
Obviously, legitimate use-cases exist for such services, provided that they are operated faithfully by people/entities with some level of credibility.
Whether or not Spam is legal, many (or most) service providers list it as prohibited in their SLA (Service-Level Agreement). Depending on the type of setup they run--the clientele; company focus/prioritization of ROI--they forward complaints to the account owners, with the understanding that the activity must be stopped. Again, this depends on the service provider, especially as you start thinking on a more granular level. E.g. they can have automated systems handling much of this, or they may have employees handle the communications and provide help to their clients in thwarting such activity. Generally, though, the account owners need to balance the security with compatibility--or the uptime and short-term performance of their services.
Ultimately, however, some of these spam block-lists--UCEPROTECTL3 in particular--are blatantly using a sledge-hammer approach for little benefit to any of their users, in the hopes of spurring controversy; or fear that leads to customers; or outrage from customers toward the service providers. In the end, the intention is purely to get paid, either by clients who sign up for their service or by the internet service providers that can't wait multiple days or weeks until their network gets de-listed. Their accusatory and aggressive language conveys to people that they must be associated with either some amateur company or one that uses, and knowingly profits from, nefarious practices and shady characters if they somehow ended up on their list.
Depending on where you stand, it can be seen as deplorable or underhanded behavior maybe even rising to the level extortion, or it can be considered a shrewd business tactic.
I agree with your plea, and view the current blacklists as Mafia style "protection" (that also conveniently helps the big players maintain their monopoly).
Practically though, if you want to get your mail delivered, you should use Amazon SES or the like to send it - setup is really simple, and they have the clout to not be blacklisted EVEN THOUGH they are definitely being used to send spam. At $1 per 1,000 mails, it is unlikely you will even feel the cost.
(I commented on this a few weeks ago at https://news.ycombinator.com/item?id=29713030.)
Practically though, if you want to get your mail delivered, you should use Amazon SES or the like to send it - setup is really simple, and they have the clout to not be blacklisted EVEN THOUGH they are definitely being used to send spam. At $1 per 1,000 mails, it is unlikely you will even feel the cost.
(I commented on this a few weeks ago at https://news.ycombinator.com/item?id=29713030.)
I think many RBLs started out with good intentions. But it does feel like like many of them have shifted to pure grift. Pay for "priority" review definitely has negatively impacted this. UCEPROTECTL3 especially feels like an extortion attempt similar to the threatening domain renewal scams. But I've never notice any outgoing email from my services being blocked by it. So it just goes in the crank file.
Really we probably need some sort of anti-RBL system. To keep good actors honest and force bad actors out of business.
Really we probably need some sort of anti-RBL system. To keep good actors honest and force bad actors out of business.
If I were to design a replacement messanging system to replace email, I would design it with deny by default, where messages that aren't signed by someone on your contact list are rejected. Maybe with a system to request that someone add you to their contact list (though with a limit on how much text can be in that request).
Maybe something like that could be done with email, but without a culture around it, figuring out what addresses you need to add to your contact list when you add new services could be a pain.
Maybe something like that could be done with email, but without a culture around it, figuring out what addresses you need to add to your contact list when you add new services could be a pain.
So basically how social media platforms handle messaging.
F*k UCEPROTECT!
I got listed as well (layer 3 only), because there were more than 32 cases of spam in an ISP that maintains more than 4 /16 networks across the whole country. With the whole AS blocklisted, I fail to see the point of UCEPROTECT and agree with the OP that they are mere thieves.
On topic of protecting people, they have a trivial XSS right on the input field where one checks if their IP address is listed - yes, a GET variable is printed directly to HTML.
</rant>
I got listed as well (layer 3 only), because there were more than 32 cases of spam in an ISP that maintains more than 4 /16 networks across the whole country. With the whole AS blocklisted, I fail to see the point of UCEPROTECT and agree with the OP that they are mere thieves.
On topic of protecting people, they have a trivial XSS right on the input field where one checks if their IP address is listed - yes, a GET variable is printed directly to HTML.
</rant>
Email has basically been taken over by Microsoft and Google. They set the rules and decides who can is blacklisted. Although to me, email has become increasingly less relevant, to a point where I only check it once a week or so.
> If my understanding of the law is correct, spamming is legal, albeit immoral. If I were the hosting company, and I had a paying customer that is conducting legal business on my network, on what grounds can I evict them?
Hosting companies can kick customers off for basically any reason they want to. High-end hosting companies will absolutely kick a customer off if the customer’s activities result in spam blacklisting.
When I first signed with Rackspace, it took almost a week to negotiate their AUP (acceptable use policy). They were basically unwilling to give us any solid assurances about account suspension. They gave themselves enormous room to determine what was acceptable behavior on our part.
If you haven’t carefully read your host’s AUP, it might be enlightening. If your host doesn’t have an AUP, or doesn’t enforce it, then yeah maybe that is not the best neighborhood.
Hosting companies can kick customers off for basically any reason they want to. High-end hosting companies will absolutely kick a customer off if the customer’s activities result in spam blacklisting.
When I first signed with Rackspace, it took almost a week to negotiate their AUP (acceptable use policy). They were basically unwilling to give us any solid assurances about account suspension. They gave themselves enormous room to determine what was acceptable behavior on our part.
If you haven’t carefully read your host’s AUP, it might be enlightening. If your host doesn’t have an AUP, or doesn’t enforce it, then yeah maybe that is not the best neighborhood.
No one sane is using UCEPROTECT.. I would not even care. If someone is using them as blacklist(s) they have more problems than my mail not reaching them. Spamhaus/Spamcop/truncate.gbudb.net/Barracuda with some "premiums" like Abusix is all anyone should need
Any shared hosting provider is extremely susceptible to this type of issue and this type of list just doesn't work. It's a scam. There are lots of good blocklists, but there are also lots of bad ones like this.
When I worked with a company who ran email systems like this we'd always steer customers away from lists like this, in cases where we couldn't deliver mail because of it we blamed the other provider. This actually worked a lot of the time and the customer would wind up contacting the sender another way to let them know they had a problem, and fairly often they'd change lists.
Our standard messaging was something like, "Google and Microsoft are receiving our emails just fine, something is wrong with the receiving service"
When I worked with a company who ran email systems like this we'd always steer customers away from lists like this, in cases where we couldn't deliver mail because of it we blamed the other provider. This actually worked a lot of the time and the customer would wind up contacting the sender another way to let them know they had a problem, and fairly often they'd change lists.
Our standard messaging was something like, "Google and Microsoft are receiving our emails just fine, something is wrong with the receiving service"
Blame your ISP, or hosting provider. As an email admin, I deal every-single-day with phishing and malware that comes through email. There is not a single hosting company that does anything more than pass complaints onto the spammer - thus verifying that I'm a good target and causing even more of a problem.
While I do not use any blacklists, I do make my own. I've found that blocking individual IPs is useless, there is always another one. However, if I block the entire IP range I have much more success.
This is your hosting company's fault for allowing spamming and doing nothing about it.
While I do not use any blacklists, I do make my own. I've found that blocking individual IPs is useless, there is always another one. However, if I block the entire IP range I have much more success.
This is your hosting company's fault for allowing spamming and doing nothing about it.
The US federal government should tackle huge email account providers that effectively (by accident or design) use anti-spam as a pretext to sabotage self-hosted email.
Yes because the government involved in technology always makes things better as I click on “allow cookies” on every damn website.
Punishing anti-competitive measures doesn't have to be always complicated.
"Allow cookies" modals are horrible but GDPR gives people in the EU rights to be forgotten and not contacted in the future. Reminding companies about GDPR regulations works very well.
"Allow cookies" modals are horrible but GDPR gives people in the EU rights to be forgotten and not contacted in the future. Reminding companies about GDPR regulations works very well.
just wait for the new law to outlaw spam filters so anyone can host email.
As soon as they do, email might as well be dead.
The GDPR already caused some sites just to block access to EU countries.
As soon as they do, email might as well be dead.
The GDPR already caused some sites just to block access to EU countries.
Probably a lot of those "allow cookies" banner are also illegal [0], but you can enable a cookie banner filer in uBlock Origin anyway.
[0]: https://noyb.eu/en/noyb-files-422-formal-gdpr-complaints-ner...
[0]: https://noyb.eu/en/noyb-files-422-formal-gdpr-complaints-ner...
I have been running my own mail server for two years on my private ISP and have less problems than expected - even with the dynamic IP address (in practice, it changes once in 6-12 months) and no PTR. I also switched the ISP once. I have SPF, DKIM, DMARC.
Edit: The nice thing about running the mail server personally and without a relay (like mailgun) is that mail is to-my-end encrypted. If the other party is running its own mail server, it could even be E2E encrypted. Considering the vast amount of personal information that going through email, this makes me feel good in terms of privacy.
I have never heard of UCEPROTECT and fortunately, I never had to deal with it. The language on the webpage somehow reminds me of Kryptochef...
A small inconvenience is that I had to unblock the IP on Spamhaus PBL every month. By now, it feels as if they know me, because I now only have to do this once if I get a new IP...
Many mail servers are nice and provide the reason for the block even with hints how to unblock it. I successfully unblocked it on Abusix and Microsoft. Never had an issue with Google.
GMX on the other hand will never accept my email because they require a proper PTR record. They are the only company I have come across and I find that scandalous.
Edit: The nice thing about running the mail server personally and without a relay (like mailgun) is that mail is to-my-end encrypted. If the other party is running its own mail server, it could even be E2E encrypted. Considering the vast amount of personal information that going through email, this makes me feel good in terms of privacy.
I have never heard of UCEPROTECT and fortunately, I never had to deal with it. The language on the webpage somehow reminds me of Kryptochef...
A small inconvenience is that I had to unblock the IP on Spamhaus PBL every month. By now, it feels as if they know me, because I now only have to do this once if I get a new IP...
Many mail servers are nice and provide the reason for the block even with hints how to unblock it. I successfully unblocked it on Abusix and Microsoft. Never had an issue with Google.
GMX on the other hand will never accept my email because they require a proper PTR record. They are the only company I have come across and I find that scandalous.
I like running my own mail server but I don't want to waste time on filtering spam so mismatches with ptr and hostname are an instant rejection from me.
A PTR match is a very strong signal that I am looking at a hosted server or business connection which is configured to send email and not a compromised windows PC on consumer dsl which used to be a huge problem. ISPs supply those dsl ranges to blocklists as well.
When I first started hosting my own mail it was on an old repurposed linux PC on ISDN in the early 2000s and even then my ISP offered configurable PTR records along with static IP in a small business package.
I moved my family email vps to a different region recently and it took me a few seconds to update the ptr records for ipv4 and ipv6. It seems a very low bar for such a strong signal that you are dealing with a mail server and not some random compromised machine.
A PTR match is a very strong signal that I am looking at a hosted server or business connection which is configured to send email and not a compromised windows PC on consumer dsl which used to be a huge problem. ISPs supply those dsl ranges to blocklists as well.
When I first started hosting my own mail it was on an old repurposed linux PC on ISDN in the early 2000s and even then my ISP offered configurable PTR records along with static IP in a small business package.
I moved my family email vps to a different region recently and it took me a few seconds to update the ptr records for ipv4 and ipv6. It seems a very low bar for such a strong signal that you are dealing with a mail server and not some random compromised machine.
Your comment about no issues with no valid PTR surprised me.
Spamhaus PBL is build based on your ISP telling Spamhaus which IPs are dynamic and which IPs should not send email. Your ISPs seemed to be nice enough to allow you delisting from it, which not all ISPs will allow.
Abusix has a similar list, but its completely build based on dynamic looking or no PTRs. You can create an account and delist without any issues.
Never the less there is way more services than GMX that block based on dynamic or no PTR. A lot of smaller solutions have this option checked by default. And it actually has been a best practice for decades to have a proper PTR.
If you can, I'd set one and be done with it.
Spamhaus PBL is build based on your ISP telling Spamhaus which IPs are dynamic and which IPs should not send email. Your ISPs seemed to be nice enough to allow you delisting from it, which not all ISPs will allow.
Abusix has a similar list, but its completely build based on dynamic looking or no PTRs. You can create an account and delist without any issues.
Never the less there is way more services than GMX that block based on dynamic or no PTR. A lot of smaller solutions have this option checked by default. And it actually has been a best practice for decades to have a proper PTR.
If you can, I'd set one and be done with it.
UCEPROTECT-2 and 3 aren't blocklists, they're reputation lists. Anyone straight blocking off those lists has most likely misconfigured their filtering and should expect to be missing mails.
NOTE: By using Level 2 blocking, be prepared to lose a few mails too. DO NOT BLAME US, YOU HAVE BEEN FOREWARNED! > https://www.uceprotect.net/en/index.php?m=3&s=4
Use of Level 3 for blocking is recommended only if you are a HARDLINER and you want to cause service providers and carriers that have spammer / abusive clients to be quickly and effectively blocked and it does not matter to you if regular email is also occasionally rejected. > https://www.uceprotect.net/en/index.php?m=3&s=5
This has been the subject of hot debate on one of the popular mailing list operator's lists, and the providers that find themselves in Level 2/3 reliably are working on strategies to deal with that but it becomes complicated to do anything that affects thousands of customers. You'll find the providers that aren't on these lists are the ones that have an equally strict policy for allowing outbound Port 25 from their customers and a severely punitive abuse desk.
IMO, I run a personal mail server and deliver directly and if my mail is rejected or lost then I use another mechanism to contact that person (usually telephone) and inform them of the failure. I also provide usable feedback to senders for why I may have chosen to reject their message. And when it really matters, I also have a relay I can use for troublesome transports.
NOTE: By using Level 2 blocking, be prepared to lose a few mails too. DO NOT BLAME US, YOU HAVE BEEN FOREWARNED! > https://www.uceprotect.net/en/index.php?m=3&s=4
Use of Level 3 for blocking is recommended only if you are a HARDLINER and you want to cause service providers and carriers that have spammer / abusive clients to be quickly and effectively blocked and it does not matter to you if regular email is also occasionally rejected. > https://www.uceprotect.net/en/index.php?m=3&s=5
This has been the subject of hot debate on one of the popular mailing list operator's lists, and the providers that find themselves in Level 2/3 reliably are working on strategies to deal with that but it becomes complicated to do anything that affects thousands of customers. You'll find the providers that aren't on these lists are the ones that have an equally strict policy for allowing outbound Port 25 from their customers and a severely punitive abuse desk.
IMO, I run a personal mail server and deliver directly and if my mail is rejected or lost then I use another mechanism to contact that person (usually telephone) and inform them of the failure. I also provide usable feedback to senders for why I may have chosen to reject their message. And when it really matters, I also have a relay I can use for troublesome transports.
> I run my own email server, and don’t use any blacklist. Yes, I got some amount of SPAMs. However, the vast majority of trashy emails I receive each day are advertisements (I signed up for random stuff quite liberally).
You don't run a mail server for other people who are not so careful with their e-mail addresses, let alone for a large organization. You don't run a mail server which hosts public mailing lists.
You don't run a mail server for other people who are not so careful with their e-mail addresses, let alone for a large organization. You don't run a mail server which hosts public mailing lists.
I don't now; but when I did run a medium sized organization, I used a very lenient interpretation of spamhaus together with greylisting, and my users were happy.
In email, we have accepted a status quo of unreliable availability (delivery), a lack of confidentiality, and frequent fraud. The post office is far better, other than the latency of moving paper.
We should wake up and raise the standard. We could even make it a public utility, or a utility that contracts to competing companies, or just develop standards for delivery and user rights - nobody can read my mail, nobody can interfere with it, delivery guaranteed within X time, etc. One challenge is to esure that individuals are still able to run their own servers (e.g., develop standards that the FOSS email software developers can meet). But we could have similar laws to regular mail, protecting delivery, confidentiality, and against fraud.
Why do we endure this awful system? Recently I feel like the IT world has shifted from an innovative, creative, we-will-solve-any-problem, build-it-in-our-garage community, to one that is defeatist (nothing we can do), status quo, depends on mega-corps (or VCs) for everything, and which prizes a how-much-can-I-take narcissistic business mentality rather than a creative, change-the-world engineering one.
We should wake up and raise the standard. We could even make it a public utility, or a utility that contracts to competing companies, or just develop standards for delivery and user rights - nobody can read my mail, nobody can interfere with it, delivery guaranteed within X time, etc. One challenge is to esure that individuals are still able to run their own servers (e.g., develop standards that the FOSS email software developers can meet). But we could have similar laws to regular mail, protecting delivery, confidentiality, and against fraud.
Why do we endure this awful system? Recently I feel like the IT world has shifted from an innovative, creative, we-will-solve-any-problem, build-it-in-our-garage community, to one that is defeatist (nothing we can do), status quo, depends on mega-corps (or VCs) for everything, and which prizes a how-much-can-I-take narcissistic business mentality rather than a creative, change-the-world engineering one.
I think the stakes have become too high to treat IT like the wild west. There are too many stakeholders and too much money behind it. Haskell adopted the motto "avoid success at all costs" to prevent falling into the same pattern.
> On the other hand, if a legitimate email from a future friend or a potential business associate were accidentally blocked, the lost opportunity cost is several magnitudes higher.
To some extent, this also applies to legitimate business mails going to Spam. That was a big reason for me to switch away from Outlook to a provider where I could configure the spam filter to the equivalent of "Viagra scams etc".
To some extent, this also applies to legitimate business mails going to Spam. That was a big reason for me to switch away from Outlook to a provider where I could configure the spam filter to the equivalent of "Viagra scams etc".
Managing mail servers is nowadays only possible for million dollars backed companies. It's clearly something we discovered while running Improvmx.com.
Of course it's possible (and quite easy) to set up your own mail server, there are tons of guides out there for that, but this is just the tip of the iceberg! Once you have it up and running, you'll find yourself in a constant battle between those who spam you, and those who consider you spammer, real or not.
Using blacklists to filter out the bad incoming email is one efficient tool, but you need to rely on good, reliable and *honest* blacklist systems. Sorbs and Spamhaus comes to mind, compared to uceprotect and backscaterer. The last two asks you money to delist you regardless of what you did, and this is typically bad player playing on top of bad players. They do a race to the bottom by offering a racketing service.
But the worst is still that popular mail server uses these! Microsoft uses UCEProtect to filter email! So, as a mail provider like OP's, you find yourself in a pickle where you need to decide if you want your email to be delivered to Microsoft's users - that means paying, or not.
That is why managing emails has become a million dollars necessity if you want them to be up and running properly. You need to be close to the big ones, have a seat at the big tables where the decisions occurs and hopefully have your IPs added to whitelists.
To plug one very useful link in all this, I'd share Hetrix! (https://Hetrixtools.com), they monitor your IPs and notify you when they are listed on almost all the existing blacklists, and automatically delist them whenever possible, or provides you all the info to do to delist them. As a mail provider, this is on of the most useful tool there is.
Of course it's possible (and quite easy) to set up your own mail server, there are tons of guides out there for that, but this is just the tip of the iceberg! Once you have it up and running, you'll find yourself in a constant battle between those who spam you, and those who consider you spammer, real or not.
Using blacklists to filter out the bad incoming email is one efficient tool, but you need to rely on good, reliable and *honest* blacklist systems. Sorbs and Spamhaus comes to mind, compared to uceprotect and backscaterer. The last two asks you money to delist you regardless of what you did, and this is typically bad player playing on top of bad players. They do a race to the bottom by offering a racketing service.
But the worst is still that popular mail server uses these! Microsoft uses UCEProtect to filter email! So, as a mail provider like OP's, you find yourself in a pickle where you need to decide if you want your email to be delivered to Microsoft's users - that means paying, or not.
That is why managing emails has become a million dollars necessity if you want them to be up and running properly. You need to be close to the big ones, have a seat at the big tables where the decisions occurs and hopefully have your IPs added to whitelists.
To plug one very useful link in all this, I'd share Hetrix! (https://Hetrixtools.com), they monitor your IPs and notify you when they are listed on almost all the existing blacklists, and automatically delist them whenever possible, or provides you all the info to do to delist them. As a mail provider, this is on of the most useful tool there is.
I too have had issues with that particular blacklist, which, I believe, is gaining notoriety as a scam. In my case I was setting up outgoing email for a customer's WordPress website. In the end, rather than spending the time and money dealing with the blacklist, we routed their mail through Sendgrid. Not an especially happy ending, but their email works now.
Is anybody using "fail2ban" connected to "badIP"?
(as described e.g. here https://www.howtoforge.com/tutorial/protect-your-server-comp... )
(in both modes, download & upload)
My root server hosts as separate VMs at least a website and an email server => both are magnets for all kinds of scans and intrusion attempts => I've got logscans + honeypots etc... set up (fail2ban then closes the source IP's connection for a while) which seem to be working.
I wonder if things could improve (e.g. preemptive FW-drop) by making fail2ban use "badIPs", and to make fail2ban feed back to badIPs the "bad IPs that I identify"? Any personal experience in this area here? How dynamic/reliable are "badIPs" lists?
(as described e.g. here https://www.howtoforge.com/tutorial/protect-your-server-comp... )
(in both modes, download & upload)
My root server hosts as separate VMs at least a website and an email server => both are magnets for all kinds of scans and intrusion attempts => I've got logscans + honeypots etc... set up (fail2ban then closes the source IP's connection for a while) which seem to be working.
I wonder if things could improve (e.g. preemptive FW-drop) by making fail2ban use "badIPs", and to make fail2ban feed back to badIPs the "bad IPs that I identify"? Any personal experience in this area here? How dynamic/reliable are "badIPs" lists?
I tried to use F2B and https://voipbl.org on a small 1 core + 1 GB RAM vm and it did not work out very well. I'm pretty sure iptables was crashing because there were so many IPS to block. Your mileage may vary. Probably needs something a little more powerful than what I had.
Thanks - yeah, I did think that that list might be quite long... .
That you can pay them to get off the blacklist does sound rather like a protection racket. Am I missing anything?
This has become bad enough that operators of personal mail servers are put into no-win situations and generally give up.
Either we can concede defeat and relay personal correspondence email through a commercial service or we can just accept that some recipients will not receive our emails.
I've adopted the latter stance because I don't operate a business that uses my personal mail server and therefore have the luxury of not caring a whole lot. My stance is that if a recipient is using an email service that blocks me for no good reason, they don't want to receive my email. They can complain to their service provider if they choose to. In a few cases, I've asked them to complain to their service provider and have seen some minor corrective action. But that's pretty rare.
Either we can concede defeat and relay personal correspondence email through a commercial service or we can just accept that some recipients will not receive our emails.
I've adopted the latter stance because I don't operate a business that uses my personal mail server and therefore have the luxury of not caring a whole lot. My stance is that if a recipient is using an email service that blocks me for no good reason, they don't want to receive my email. They can complain to their service provider if they choose to. In a few cases, I've asked them to complain to their service provider and have seen some minor corrective action. But that's pretty rare.
This is actually a really big and growing problem IMO.
I gave up trying to run SMTP ourselves years (decades?) ago. Services like Sendgrid et al all made it very easy.
However, in the last couple of years I've noticed even the basic paid plan on Sendgrid, Sparkpost, Mailgun (I've tried them all now I think?) still gets constantly blacklisted. Going to support gives you one answer: upgrade to a dedicated IP, which is close to $100/month, and really quite wasteful on the IPv4 space (if anyone knows a cheaper/better alternative, please let me know!).
The problem is I am often building systems which require very little (but really important) email - the odd password reset request perhaps.
For small webapps you're going to be spending probably more for this dedicated email IP than the rest of your infra combined.
I gave up trying to run SMTP ourselves years (decades?) ago. Services like Sendgrid et al all made it very easy.
However, in the last couple of years I've noticed even the basic paid plan on Sendgrid, Sparkpost, Mailgun (I've tried them all now I think?) still gets constantly blacklisted. Going to support gives you one answer: upgrade to a dedicated IP, which is close to $100/month, and really quite wasteful on the IPv4 space (if anyone knows a cheaper/better alternative, please let me know!).
The problem is I am often building systems which require very little (but really important) email - the odd password reset request perhaps.
For small webapps you're going to be spending probably more for this dedicated email IP than the rest of your infra combined.
This is blatant racketeering - they create a problem for you and demand you pay them for protection.
https://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists
That blacklist is listed as "Suspect RBL provider". That says enough about it...
That blacklist is listed as "Suspect RBL provider". That says enough about it...
From the Wikipedia article:
These RBL providers have demonstrated the potential and willingness to adversely affect vast swaths of internet communications for misguided, reckless or likely fraudulent purposes. Using these RBL providers will likely result in clogging up ISP support channels while negatively affecting legitimate business customers.
Blacklists are fine. If someone wants to make a list based on some criteria then OK, there is no real way to prevent them from doing that.
The people doing the actual blocking based on the list have the responsibility for their actions. If, say, one of the largest email providers in the world is found to be giving preferential treatment to the email of other large email providers then they can't use some list as an excuse. Their actions are still anticompetitive and generally harmful.
We shouldn't forget to go after the entity that runs the email server that is blocking email from our servers. You don't have to care that there is a list. Blame the right people and involve governments if required.
The people doing the actual blocking based on the list have the responsibility for their actions. If, say, one of the largest email providers in the world is found to be giving preferential treatment to the email of other large email providers then they can't use some list as an excuse. Their actions are still anticompetitive and generally harmful.
We shouldn't forget to go after the entity that runs the email server that is blocking email from our servers. You don't have to care that there is a list. Blame the right people and involve governments if required.
The UCEPROTECTL blacklists self-clear after a while and it's not nearly as bad other "real" blacklists. Considering all the other hassles one must engage in for self-hosted mail, this one is not high on my list of worries. My host occasionally pops up on these for a few days and usually clears without incident. But yes it's frustrating that my mailhost can be guilty by association despite being in good standing otherwise.
And of course the side-benefit for Gmail to enforce more anti-spam provisions is that it increases the reliability of the data they can mine from your mail, and my mail too since so many people still use Gmail.
And of course the side-benefit for Gmail to enforce more anti-spam provisions is that it increases the reliability of the data they can mine from your mail, and my mail too since so many people still use Gmail.
This post meanders between complaining about RBLs and vaguely whining about the big email hosts, but I don't see the connection. There is zero useful information contained in the RBLs and the big hosts don't use them.
I believe that outlook.com starts using UCEPROTECTL2/3. However I have no proof so I have to be vague about it.
It is in the interest of big email hosts to shut out small, independent senders to strengthen their position of monopoly. Again, I have no proof.
It is in the interest of big email hosts to shut out small, independent senders to strengthen their position of monopoly. Again, I have no proof.
When Exchange hosting which is just about every business in Australia it seems (and possibly outlook.com and 365) blocked a huge range of IPs, a lot of people went to IP blocklist searches and saw the only entry was UCEPROTECT3 and connected the events. It is very possible they are unrelated or it crept in through another provider's product without anyone there explicitly choosing to use it. Unfortunately the whole system is very opaque.
I signed up for random stuff quite liberally
Wait, what? That's like off-handedly mentioning you eat a pine cone for breakfast every day and glossing over it like we're not going to wonder if you're crazy.
Wait, what? That's like off-handedly mentioning you eat a pine cone for breakfast every day and glossing over it like we're not going to wonder if you're crazy.
For the past five years or so at least 80% of spam that reaches my inbox is from Gmail, Yahoo, or Microsoft.
I've been running my own mail server for 9 years now and I don't use any blacklists.
Blacklists are the red line districts of the web. They are just a large blunt tool used in a very inappropriate way. The system of IP address space ownership assignment where the vast majority of the IP address users will never own an IP address. It's like renting a business suite in a large office building and being accountable for some unscrupulous neighbor.
I've been running my own mail server for 9 years now and I don't use any blacklists.
Blacklists are the red line districts of the web. They are just a large blunt tool used in a very inappropriate way. The system of IP address space ownership assignment where the vast majority of the IP address users will never own an IP address. It's like renting a business suite in a large office building and being accountable for some unscrupulous neighbor.
this is the first time that I've ever heard of a email blacklist provider offering to remove an IP from their database if you pay a monthly fee... it sounds like extortion.
From the article:
> If my understanding of the law is correct, spamming is legal, albeit immoral
But the FTC takes a rather clear stance (at least in my eyes) [0]:
> Despite its name, the CAN-SPAM Act doesn’t apply just to bulk email. It covers all commercial messages, which the law defines as “any electronic mail message the primary purpose of which is the commercial advertisement or promotion of a commercial product or service,” including email that promotes content on commercial websites. The law makes no exception for business-to-business email. That means all email – for example, a message to former customers announcing a new product line – must comply with the law.
> Each separate email in violation of the CAN-SPAM Act is subject to penalties of up to $46,517, so non-compliance can be costly. But following the law isn’t complicated*
On top of that there's GDPR [1]:
> After the GDPR passed, some people said it would be “the end of email marketing” or “the end of spam.” But it will be neither. Spam has always been outlawed or against the terms of use of most email providers. Those who send unsolicited or malicious mass emails will probably continue to send them. Did your spam folder dry up after May 25, 2018, when the GDPR took effect?
So, it seems the laws are in place, it's just that the jungle of unwanted & unsolicited email continues. It sucks when it's your server that get's blocked in a dragnet because you've a noisy neighbour. But like others have stated, move neighbourhoods.
[0] https://www.ftc.gov/tips-advice/business-center/guidance/can...
[1] https://gdpr.eu/email-encryption/
> If my understanding of the law is correct, spamming is legal, albeit immoral
But the FTC takes a rather clear stance (at least in my eyes) [0]:
> Despite its name, the CAN-SPAM Act doesn’t apply just to bulk email. It covers all commercial messages, which the law defines as “any electronic mail message the primary purpose of which is the commercial advertisement or promotion of a commercial product or service,” including email that promotes content on commercial websites. The law makes no exception for business-to-business email. That means all email – for example, a message to former customers announcing a new product line – must comply with the law.
> Each separate email in violation of the CAN-SPAM Act is subject to penalties of up to $46,517, so non-compliance can be costly. But following the law isn’t complicated*
On top of that there's GDPR [1]:
> After the GDPR passed, some people said it would be “the end of email marketing” or “the end of spam.” But it will be neither. Spam has always been outlawed or against the terms of use of most email providers. Those who send unsolicited or malicious mass emails will probably continue to send them. Did your spam folder dry up after May 25, 2018, when the GDPR took effect?
So, it seems the laws are in place, it's just that the jungle of unwanted & unsolicited email continues. It sucks when it's your server that get's blocked in a dragnet because you've a noisy neighbour. But like others have stated, move neighbourhoods.
[0] https://www.ftc.gov/tips-advice/business-center/guidance/can...
[1] https://gdpr.eu/email-encryption/
Level 2 and Level 3 are not blocklists that are suitable for blocking on their own. They are punitive blocklists. They should only be used as part of a scoring system, such as SpamAssassin.
If your outbound mail is being blocked by some server just because your MTA is in one of those lists, it might be worth contacting the postmaster at the other end, and asking them either to drop those two lists, to use them only for spam-scoring, or to whitelist your MTA.
If your outbound mail is being blocked by some server just because your MTA is in one of those lists, it might be worth contacting the postmaster at the other end, and asking them either to drop those two lists, to use them only for spam-scoring, or to whitelist your MTA.
We recently ran into this issue and were forced to change email providers. It isn’t that our host didn’t care, it’s that it is almost impossible to play whack-a-mole with dozens of blacklists.
We have been with the same provider for 15 years, no problems, ever. It seems that email delivery started to become unreliable about six months ago. After repeated attempts to fix it we had no choice but to move elsewhere.
We have been with the same provider for 15 years, no problems, ever. It seems that email delivery started to become unreliable about six months ago. After repeated attempts to fix it we had no choice but to move elsewhere.
Who'd you choose?
Sorry, I've been so busy I haven't checked into HN in a while.
Zoho mail. Very happy with them so far. The webmail client is excellent. Deliver is fast and without problems. We usually setup users with multiple email addresses. Using email aliasing works great. In other words, you could receive email from [email protected], [email protected] and [email protected]. This only consumes one paid user account.
Ultimately, we decided to move email for three businesses to Zoho.
The prior host was GoDaddy. I can imagine someone saying something like: "You deserve the problems you had". My counter is that we've hosted email with GoDaddy for more than ten years. No problems at all. Rarely an issue. In my opinion, most of the GoDaddy hatred is truly unfounded. If you are running a more sophisticated infrastructure you need to go to something like Linode (which we use for exactly those applications) or something else.
In the last 6 to 12 months, something changed. Once reliable email delivery became questionable we had no choice but to make a move. They said they could not do much more than request the servers in question to be removed from the various blacklists. In other words, play whack-a-mole with that entire ecosystem. A loosing proposition. So, we moved our business. Perfectly sensible decision. No hate involved. We are buying a service that, in my opinion, was no-longer being offered. Done deal.
Zoho mail. Very happy with them so far. The webmail client is excellent. Deliver is fast and without problems. We usually setup users with multiple email addresses. Using email aliasing works great. In other words, you could receive email from [email protected], [email protected] and [email protected]. This only consumes one paid user account.
Ultimately, we decided to move email for three businesses to Zoho.
The prior host was GoDaddy. I can imagine someone saying something like: "You deserve the problems you had". My counter is that we've hosted email with GoDaddy for more than ten years. No problems at all. Rarely an issue. In my opinion, most of the GoDaddy hatred is truly unfounded. If you are running a more sophisticated infrastructure you need to go to something like Linode (which we use for exactly those applications) or something else.
In the last 6 to 12 months, something changed. Once reliable email delivery became questionable we had no choice but to make a move. They said they could not do much more than request the servers in question to be removed from the various blacklists. In other words, play whack-a-mole with that entire ecosystem. A loosing proposition. So, we moved our business. Perfectly sensible decision. No hate involved. We are buying a service that, in my opinion, was no-longer being offered. Done deal.
The sad thing is that a solution for protection against spam has existed for about thirty years, yet it is almost unused. It is based on the same idea that later came to underpin Bitcoin: proof of work: https://news.ycombinator.com/item?id=30220979
> If my understanding of the law is correct, spamming is legal, albeit immoral.
It's illegal in the US: https://www.ftc.gov/tips-advice/business-center/guidance/can...
It's illegal in the US: https://www.ftc.gov/tips-advice/business-center/guidance/can...
I think this blog should be retitled to some big US tech firms are out of control.
The US firm providing anti virus/antispam to UK politicians wont even confirm if they have blocked email sent from constituents to UK politicians.
Some websites also block emails from some European email providers, but it always traces back to US tech firms.
The US firm providing anti virus/antispam to UK politicians wont even confirm if they have blocked email sent from constituents to UK politicians.
Some websites also block emails from some European email providers, but it always traces back to US tech firms.
Last time I was an admin of a server that got onto SMTP IP blocklists, it was because some spammer had brute-forced the SMTP password of one of our users and was sending out spam through that account. Passwords are terrible, switch to using certs/keys instead of passwords if you can.
We have the same problem, even our verification email during registration goes to spam sometimes. Should there be a law that a legitimate company register their domain in a registry that spam filters have to ignore? Otherwise, companies controlling spam filters can choke other companies.
I commented recently on this very topic and gave an account as to why the company I worked for got out of email hosting years ago. It was blockhole lists.
https://news.ycombinator.com/item?id=29675365
https://news.ycombinator.com/item?id=29675365
It's not a new thing. I was a sysadmin for a small ISP back in the early 2000s and we had a hate relationship with Spamhaus. They were militant, trigger happy, and very difficult to work with. It was an ongoing headache.
I figure that places like spamhaus are a good part of why gmail took over.
I figure that places like spamhaus are a good part of why gmail took over.
Makes you wonder if email will someday become so unusable we'll go back to using the post.
Anecdotal but I've just had a company write to me by post because they didn't recieve my last email I definitely sent them, and I'm going to have to write back to them by post.
I used to have a credit card company that would send me a letter by post every month stating "You have signed up for paperless billing. Please check your account online for your statement."
Hormel Foods have graciously indicated that they don't mind the use of the term "spam" to refer to UBE/UCE, but have requested that the capitalized version "SPAM" be respected as their trademark.
My main issue is one big US company which has history of engaging in monopolistic practces. They blocked a huge range of Linode IP space in December/January including one of my servers which has a totally clean record, supports every industry standard and is only used by my immediate family. It is guilt by association but blocking email because someone lives in the wrong neighborhood isn't a civil rights violation unfortunately.
When a very big company blocks nearly all email from another company that they compete with in hosting services I think it should be investigated as a possible abuse of market power by the EU and DOJ. I think there are important questions to be asked about the damage this does to competition and consumer choice.
I wouldn't worry about UCEPROTECT listing. They have an option to pay to remove IPs which to me looks like a protection racket. If anyone blocks solely on the basis of those lists instead of taking signals from more respectable block lists they are only hurting their own customers by blocking legitimate email.
I have multiple servers at multiple providers for various things so it is relatively trivial for me to add a rule to a transport map and have my server securely login into another machine to relay the email to a misbehaving receiver. But that requires slightly more than minimal knowledge and many people are pushed towards a small number of gatekeepers who will ultimately be purchased by a cartel of mega companies and nobody will know how to run a mail server anymore. Which is a shame because it really isn't very hard or very technical. What makes it tedious is the behaviour of some companies. The technical workaround to a block is as easy as routing email via another IP and takes a few minutes.
But then there is the process of getting the original IP unblocked, sometimes via multiple processes in businesses with a single owner, seemingly none of them knowing what the others or even their own tech people are doing. And/or swapping out the IP for one in a clean block.
When a very big company blocks nearly all email from another company that they compete with in hosting services I think it should be investigated as a possible abuse of market power by the EU and DOJ. I think there are important questions to be asked about the damage this does to competition and consumer choice.
I wouldn't worry about UCEPROTECT listing. They have an option to pay to remove IPs which to me looks like a protection racket. If anyone blocks solely on the basis of those lists instead of taking signals from more respectable block lists they are only hurting their own customers by blocking legitimate email.
I have multiple servers at multiple providers for various things so it is relatively trivial for me to add a rule to a transport map and have my server securely login into another machine to relay the email to a misbehaving receiver. But that requires slightly more than minimal knowledge and many people are pushed towards a small number of gatekeepers who will ultimately be purchased by a cartel of mega companies and nobody will know how to run a mail server anymore. Which is a shame because it really isn't very hard or very technical. What makes it tedious is the behaviour of some companies. The technical workaround to a block is as easy as routing email via another IP and takes a few minutes.
But then there is the process of getting the original IP unblocked, sometimes via multiple processes in businesses with a single owner, seemingly none of them knowing what the others or even their own tech people are doing. And/or swapping out the IP for one in a clean block.
> Other customers within this range did not care about their security and got hacked, started spamming, or were even attacking others, while your provider has possibly not even noticed that there is a serious problem. We are sorry for you, but you have chosen a provider not acting fast enough on abusers.
What a ridiculously hostile message. Just because a website or mail server or whatever they are talking about got compromised does not mean that the owner "did not care about their security". This sounds like a case of a blocklist operator on a power trip, talking down to people.
What a ridiculously hostile message. Just because a website or mail server or whatever they are talking about got compromised does not mean that the owner "did not care about their security". This sounds like a case of a blocklist operator on a power trip, talking down to people.
Can anyone here (the author?) recommend a one-stop-shop script for checking ones IP address / prefixes against these blacklists, ordered by severity?
I have never spammed in my entire life. But it seems I am also blacklisted. Pikachu :)
"Sorry, Your IP is listed in some other Realtime IP Blacklist."
"Sorry, Your IP is listed in some other Realtime IP Blacklist."
I run my own email server, and I have _really_ a pb with smtp servers using spamhaus block lists. Ofc, to "whitelist" your IP with spamhaus (some shady andoran/swiss mafia), you must use a javascript only web engine based web browsers (I don't and I shoud not have to) in order to contact spamhaus. If this is a "chat" with their ppl, where is their IRC server?
If I have to pay something out of this, this will be a lawyer to deal with the admins of such smtp servers.
We just recently switched from a self hosted email server to one of the services (smtp.com, smtp2go, etc.) We gave up the fight.
Would be nice if ipv6 would fix this. This is also the reason I stopped self-hosting email, despite some nice benefits.
Hello All,
I am looking for somebody that already has a sophisticated mailing operation, and I have the rest. Anybody interested?
I am looking for somebody that already has a sophisticated mailing operation, and I have the rest. Anybody interested?
Uceprotect is a scam. Ignore it.
Using it for inbound email is a misconfiguration.
People on HN seem to be unable to run mail servers.
It's not that hard.
SMTP is just a bad protocol and it needs to go away. Unworkable.
Extortion rackets
The entire email, message, and phone model is upside down for me.
Rather than defaulting to allow-all, it should default to deny-all. Content is let in on a case by case basis.
I found an email service the operates this way, Hey.com. Unfortunately, it’s pretty sparse in mostly every way but this. However, it’s practically impossible for me to receive spam. You can’t get into my inbox unless I’ve approved you.
It doesn’t make any sense (to me) to maintain these hilariously complex deny lists that results in infinite cat and mouse. Just deny everything and only allow in known good actors. Voila.
Rather than defaulting to allow-all, it should default to deny-all. Content is let in on a case by case basis.
I found an email service the operates this way, Hey.com. Unfortunately, it’s pretty sparse in mostly every way but this. However, it’s practically impossible for me to receive spam. You can’t get into my inbox unless I’ve approved you.
It doesn’t make any sense (to me) to maintain these hilariously complex deny lists that results in infinite cat and mouse. Just deny everything and only allow in known good actors. Voila.
SMTP is a failed product.
Not so much failed as just outdated. That's like saying NCP is a failed product. It served it purpose for a time and as things changed, it was replaced.
That being said, I don't know how SMTP hasn't been replaced yet. It shouldn't have lived much longer than the early 2000s, in my opinion.
That being said, I don't know how SMTP hasn't been replaced yet. It shouldn't have lived much longer than the early 2000s, in my opinion.
its pretty telling the grandparent used the word 'product'. that's why we haven't replaced SMTP, because everyone is off building products. and they all want to own the whole world. SMTP isn't a product. its an open protocol standard.
(I certainly agree that we should have a new one, I just can't see how that would happen)
(I certainly agree that we should have a new one, I just can't see how that would happen)
The adblock lists are out of the control. Adblock Plus and Brave will block put your site on a block list to then sell ads against your site to companies like Verizon and Google.
Brave loves Verizon so much they even listed them as a featured advertiser. https://brave.com/brave-ads/
Brave loves Verizon so much they even listed them as a featured advertiser. https://brave.com/brave-ads/
note that there is a huge difference between corrupted for profit things like "adblock plus" and the community sourced, not for profit things like ublock origin.
Email is a fundamentally broken protocol which has become less and less important, I don't really get the point of running your own server except as a technical exercise.
Set up your email wherever is convenient, encrypt stuff that matters, and move as much communication as possible elsewhere.
Set up your email wherever is convenient, encrypt stuff that matters, and move as much communication as possible elsewhere.
All the 'elsewheres' are non-federated, centrally controlled, and/or corporate. No thanks.
But where is elsewhere?
Postal, as a backstop. The cost is a feature.
Network-specific messaging tools are another option. I strongly prefer open protocols.
The concept that anyone anywhere can intrude on anyone anywhere else at no financial or reputational cost is ultimately flawed. It works only so long as those with that access are few in number, generally of mutual interest, and act in a largely principled manner.
As numbers increase, and levels of interest and level of principles fall, the system will collapse.
Usenet was the first such network to fall to this dynamic. Email is well on its way. Telephony is into its first years of general intolerability (any direct-dialed universal access, wired or otherwise). Facebook faces this threat less through its lack of filters than the defection of high-affinity users.
Postal mail has its own issues with quality, but the associated costs do in fact impose a minimum bar to malicious content.
Network-specific messaging tools are another option. I strongly prefer open protocols.
The concept that anyone anywhere can intrude on anyone anywhere else at no financial or reputational cost is ultimately flawed. It works only so long as those with that access are few in number, generally of mutual interest, and act in a largely principled manner.
As numbers increase, and levels of interest and level of principles fall, the system will collapse.
Usenet was the first such network to fall to this dynamic. Email is well on its way. Telephony is into its first years of general intolerability (any direct-dialed universal access, wired or otherwise). Facebook faces this threat less through its lack of filters than the defection of high-affinity users.
Postal mail has its own issues with quality, but the associated costs do in fact impose a minimum bar to malicious content.
You can do everything possible, have a perfectly clean IP, have a good amount of outbound email traffic, only send transactional email, etc. Still, there will be edge cases where email does not go through. AT&T email servers would constantly blacklist me and not respond to requests to remove me, gmail/yahoo/outlook would silently put emails in the spam folder, and companies using email firewall products would blacklist me, with an IT Dept too inept to fix it.
The solution was to pay a small fee and proxy all outbound email through a transactional SMTP sender, like Postmark or Mailgun. It's easy to do, with one line of code in Postfix. You can be selective, and only proxy emails sent to certain troublesome domains. If you try an email provider and it's not working out, it's one line of code to change to another provider.
This allows me to still manage nearly all aspects of hosting my email server and control my email data, while not dealing with deliverability issues. I use Postmark and I have not dealt with a deliverability issue in two years.