DDoS Protection(docs.digitalocean.com)
docs.digitalocean.com
DDoS Protection
https://docs.digitalocean.com/products/networking/ddos/
55 comments
That’s pretty much how insurance works. This is a 20% fee for DDoS insurance.
AWS has an (expensive) option with DDoS cost protection.
https://aws.amazon.com/shield/features/#AWS_Shield_Advanced
https://aws.amazon.com/shield/features/#AWS_Shield_Advanced
They gotta get more money from all the scammers they host somehow.
I partially wanted to LOL this. Realized that's not a thing for HN.
Thought that this will likely be downvoted because HN.
Then I thought, what a great way to tax the spammers / hackers that use DO!
I would gladly pay the $5 ddos fee to hammer the ips that keep trying to login to my wordpress sites.
just kidding, I would not actually do that.. but the thought is pleasant.
I've used DO many times and I am a fan btw - just looked back at their pricing page a couple days ago considering spinning up a droplet to self host a git thing.
Then I thought, what a great way to tax the spammers / hackers that use DO!
I would gladly pay the $5 ddos fee to hammer the ips that keep trying to login to my wordpress sites.
just kidding, I would not actually do that.. but the thought is pleasant.
I've used DO many times and I am a fan btw - just looked back at their pricing page a couple days ago considering spinning up a droplet to self host a git thing.
not sure if I was clear enough for the downvoters to get it, ddos the ips and have DO charge the spammers / hackers a fee is the point..
Not that I advocate for ddos - I think it's a terrible thing, and I've been through it a few times.
DO and OVH are my most blocked ip blocks on several servers. I also get some hetzner and aws and microsoft blocks a lot among others sure.
Interesting that I just launched a brand new WP on a brand new domain, and in less than 24 hours half of the hack attempts are from DO ips.
You could lecture me about reporting and blah blah. I've been down those roads spending literal months doing that. With DO's cheap boxes and rotating IPs it's not worth it, I just block the entire CIDR every time, today it's 157.245.0.0/16 and 174.138.0.0/17
If DO was serious about stopping these abuses they would offer a WP plugin or opt-in setting that could check data from wordfence and similar and easily see which of their boxes are being used to hack into sites, all this could be automatic, without the form filling and delays that are required.
Not that I advocate for ddos - I think it's a terrible thing, and I've been through it a few times.
DO and OVH are my most blocked ip blocks on several servers. I also get some hetzner and aws and microsoft blocks a lot among others sure.
Interesting that I just launched a brand new WP on a brand new domain, and in less than 24 hours half of the hack attempts are from DO ips.
You could lecture me about reporting and blah blah. I've been down those roads spending literal months doing that. With DO's cheap boxes and rotating IPs it's not worth it, I just block the entire CIDR every time, today it's 157.245.0.0/16 and 174.138.0.0/17
If DO was serious about stopping these abuses they would offer a WP plugin or opt-in setting that could check data from wordfence and similar and easily see which of their boxes are being used to hack into sites, all this could be automatic, without the form filling and delays that are required.
I have some cheap(east) VPS with OVH that I didn't even know had DDoS protection until I got the emails that my host was temporarily migrated to mitigation infrastructure during a DDoS, and back a few minutes later. Was pretty impressed especially since I don't pay extra for it or even know I had it!
it benefits them, since DDoS will take down not just your VPS, but more infrastructure along the way. there are probably downsides as well, like blocking crawlers from search engines
Very vague. Doesn’t specify if it is in-line or offload. Linode, with some research you can figure out they use Corero appliances that will cover 40 gbps floods.
Also didn’t see what their policy on tweaks are and or expectation on mitigating a more advanced attack.
I.e. DNS, NTP floods are low hanging fruit but it doesn’t take much nowadays to do something more custom.
Also didn’t see what their policy on tweaks are and or expectation on mitigating a more advanced attack.
I.e. DNS, NTP floods are low hanging fruit but it doesn’t take much nowadays to do something more custom.
DO killed 2 of our production server some weeks ago erroneously due to an issue on their end that claimed we were part of a ddos attack. Took us an entire week to recover properly... maybe this might have helped... also was promised credits for the downtime but never received them, minor after the fact as we're pretty happy with the service overall.
> The Incident:
Beginning at 17:10 UTC, May 9th, multiple DigitalOcean customers experienced Droplet network outages due to an action on Droplets by an automated mechanism. This mechanism has been in place at DigitalOcean since 2019. It helps us ensure that any potentially compromised Droplet seen participating in an outbound Denial of Service attack is quickly taken offline. This is in place to assist in protecting all DO customers by ensuring we have a network focused on delivering legitimate traffic at speed and scale, unencumbered by illegitimate traffic. When triggered, this mechanism suspends networking capabilities on the Droplet or Droplet-based services temporarily to allow the owner to investigate the issue. Users are informed via a support ticket and email that details the paths to recovery. This incident was triggered by an unannounced data change made by a third-party, which DigitalOcean uses to assist in analyzing traffic flow and metrics, as well as detecting malicious traffic patterns.
Due to this mechanism constantly running and no changes being made directly by DigitalOcean, our teams were delayed in beginning an incident response. After multiple reports from customers that they believed the notification of outgoing Denial of Service attacks from their Droplets were false positives, an internal incident was declared to investigate the issue and start remediation efforts.
After a thorough investigation by the DigitalOcean Security and Networking teams, the root cause was discovered to be an erroneous change made by a third-party service that reports data on traffic. Contact was established with the third-party, and they confirmed a change had been made. Investigation began on their side, and they confirmed there was a bug causing bad data to be returned from their API.
Remediation of this incident was done through multiple paths. Complete resolution was achieved once the third-party rolled back the change that was made, which was causing bad data to be reported to DigitalOcean systems. Before that rollback was able to be put in place, DigitalOcean took direct action to take the automated mechanism that disables Droplet networking offline, given the suspected bad data. The support teams also worked throughout this incident to directly address customer tickets and re-enable networking on impacted Droplets.
> The Incident:
Beginning at 17:10 UTC, May 9th, multiple DigitalOcean customers experienced Droplet network outages due to an action on Droplets by an automated mechanism. This mechanism has been in place at DigitalOcean since 2019. It helps us ensure that any potentially compromised Droplet seen participating in an outbound Denial of Service attack is quickly taken offline. This is in place to assist in protecting all DO customers by ensuring we have a network focused on delivering legitimate traffic at speed and scale, unencumbered by illegitimate traffic. When triggered, this mechanism suspends networking capabilities on the Droplet or Droplet-based services temporarily to allow the owner to investigate the issue. Users are informed via a support ticket and email that details the paths to recovery. This incident was triggered by an unannounced data change made by a third-party, which DigitalOcean uses to assist in analyzing traffic flow and metrics, as well as detecting malicious traffic patterns.
Due to this mechanism constantly running and no changes being made directly by DigitalOcean, our teams were delayed in beginning an incident response. After multiple reports from customers that they believed the notification of outgoing Denial of Service attacks from their Droplets were false positives, an internal incident was declared to investigate the issue and start remediation efforts.
After a thorough investigation by the DigitalOcean Security and Networking teams, the root cause was discovered to be an erroneous change made by a third-party service that reports data on traffic. Contact was established with the third-party, and they confirmed a change had been made. Investigation began on their side, and they confirmed there was a bug causing bad data to be returned from their API.
Remediation of this incident was done through multiple paths. Complete resolution was achieved once the third-party rolled back the change that was made, which was causing bad data to be reported to DigitalOcean systems. Before that rollback was able to be put in place, DigitalOcean took direct action to take the automated mechanism that disables Droplet networking offline, given the suspected bad data. The support teams also worked throughout this incident to directly address customer tickets and re-enable networking on impacted Droplets.
At the risk of sounding like the "why do you need DropBox if you have rsync herr derr" guy, why... do I need DDoS protection from my VPS provider if I have Cloudflare anyway?
Why do you need DDoS protection from Cloudflare if your VPS provider has it anyway?
Some people prefer all-in-one solutions, otherwise things like AWS wouldn't exists. Others prefer to spread out their eggs. One is not absolutely right and the other absolute wrong.
Some people prefer all-in-one solutions, otherwise things like AWS wouldn't exists. Others prefer to spread out their eggs. One is not absolutely right and the other absolute wrong.
Cloudflare doesn't proxy all applications, just HTTPS.
It can e.g. magic transit. Might cost a lot more and might not work as good, but you can.
Correct yes - although having used a provider that used Magic Transit as protection I can safely say it's a terrible option.
> I can safely say it's a terrible option.
Can you elaborate?
Can you elaborate?
Compared to other mitigation providers there were very few filters, and when you were under attack it just seemed to throttle your whole connection to as low as 50Kbps, meaning players would disconnect.
Magic Firewall exists now which is pretty powerful a la wire shark filters if I remember correctly. Otherwise MT filters are pretty good now. However I have encountered a few cases where a valid connection may drop depending on how sensitive your application is but pretty rare.
It can't block most attacks. This was a few years ago though. Maybe it's better now.
Magic Transit is priced for the enterprise, out of reach for for a big chunk of the cheap VPS market: Minecraft servers and gaming in general, common DDOS targets.
What if you want to run a service that can't go behind Cloudflare (such as a game server)?
You rent with a provider that uses Magic Transit by CF, Corero appliances, OVH, Psychz, or Path networks.
We have plenty of game servers behind us. See Cloudflare Spectrum: https://developers.cloudflare.com/spectrum/
I presume they were referring to gameservers run by communities and not by game studios? I run gameservers for events that raise money for charity and we're having more and more of our boxes get hit offline - last time I had a look Spectrum required an enterprise plan for anything other than Minecraft?
Spectrum is cool but priced completely out of reach for most Cloudflare users.
I proxy through Fly.io or AWS Global Accelerator when I can’t use Cloudflare for TCP as a hobbyist.
(Why only three protocols and such strict limits, years after launching? You won’t cannibalize Spectrum by allowing more hobbyists to use it, you’ll market it for use at scale - like Argo Tunnel’s trycloudflare)
I proxy through Fly.io or AWS Global Accelerator when I can’t use Cloudflare for TCP as a hobbyist.
(Why only three protocols and such strict limits, years after launching? You won’t cannibalize Spectrum by allowing more hobbyists to use it, you’ll market it for use at scale - like Argo Tunnel’s trycloudflare)
Isn't it trivial to discover the origin IP and then just hit it directly?
You can use Cloudflare tunnel so there's no "origin" exposed.
you deny all, accept <cloudflare IPs> to the machines actually serving content.
If you set it up correctly, it's not possible.
Depends on the usecase, and whether you're shelling out the huge sums for Cloudflare Enterprise. Don't think there's too many cheap options for obfuscation if e.g. you're hosting a game server, which also happens to be one of the most common DDoS targets.
How would you?
one method is to look up what IPs someone owns and try to direct connect to them. Or you can just guess and assume they picked an IP address near the start of the block they have.
That only works for companies that own blocks, though, and assumes a company big enough to own a block won't just drop non-Cloudflare traffic (which is trivial: https://www.stavros.io/posts/block-non-cloudflare-ips-with-u...).
>assumes a company big enough to own a block
You don't have to be a big company to own a block. An ipv4 block is <$10k and the price is on a downtrend now that covid is over. It is risky as a site if you don't own your IP block because false abuse complaints can be sent to the ASN you are renting an IP from and can result in downtime. If you own the IP, the abuse reports for it go to you to handle.
>won't just drop non-Cloudflare traffic
Not all sites do this. Also they likely have other stuff running on the machines that are accessible without cloudflare.
You don't have to be a big company to own a block. An ipv4 block is <$10k and the price is on a downtrend now that covid is over. It is risky as a site if you don't own your IP block because false abuse complaints can be sent to the ASN you are renting an IP from and can result in downtime. If you own the IP, the abuse reports for it go to you to handle.
>won't just drop non-Cloudflare traffic
Not all sites do this. Also they likely have other stuff running on the machines that are accessible without cloudflare.
For comparison, I believe AWS LightSail users might be covered for DDOS protection for free but maybe I'm reading too much into their statement.
https://docs.aws.amazon.com/waf/latest/developerguide/ddos-s...
https://docs.aws.amazon.com/waf/latest/developerguide/ddos-s...
Might be "free" but then you also have to endure using AWS LightSail which you'll regret quickly.
I run a couple of dev instances on LightSail and a couple on EC2 - what exactly is the difference one should be observing there? (except intended by design)
OVH offers DDoS protection for free, and BuyVM for around 3 bucks a month. This is an awful deal.
This is very tangentially related, some comments here made me think of this:
What happened to the DDoS Open Threat Signaling (RFC 8811) protocol? Do any of the many service providers, most of which include some sort of DDoS protection, use this system at all?
https://www.rfc-editor.org/rfc/rfc8811
What happened to the DDoS Open Threat Signaling (RFC 8811) protocol? Do any of the many service providers, most of which include some sort of DDoS protection, use this system at all?
https://www.rfc-editor.org/rfc/rfc8811
I’m going to bite and ask myself,
Do you only have to pay if you have been exposed to an attack?
Do you only have to pay if you have been exposed to an attack?
https://docs.digitalocean.com/products/networking/ddos/detai...
You pay for it at a price of 20% of your plan's price. If you have a $25/mo plan, it'll be a $5 add-on. You can enable/disable it so that it's only on when you want it and it is pro-rated in that case. If you want to respond to an attack for 2 weeks of the month, it'll cost $2.50. So you can manually turn it on if you're under attack and only pay for that time, but if you want it to automatically handle an attack you have to pay for the month.
I think most people would just turn it on if they were a likely attack target. You don't want your site to go down and either get paged or find out about it hours later. If you're an unlikely attack target or a site that can stand some downtime, you could leave it off and just enable it when under attack.
You pay for it at a price of 20% of your plan's price. If you have a $25/mo plan, it'll be a $5 add-on. You can enable/disable it so that it's only on when you want it and it is pro-rated in that case. If you want to respond to an attack for 2 weeks of the month, it'll cost $2.50. So you can manually turn it on if you're under attack and only pay for that time, but if you want it to automatically handle an attack you have to pay for the month.
I think most people would just turn it on if they were a likely attack target. You don't want your site to go down and either get paged or find out about it hours later. If you're an unlikely attack target or a site that can stand some downtime, you could leave it off and just enable it when under attack.
That makes sense. Thanks.
> Do you only have to pay if you have been exposed to an attack?
It's like insurance. Do you buy it after an accident or before?
It's like insurance. Do you buy it after an accident or before?
Can I purchase insurance while I'm crashing? Sounds like getting insurance and getting DDoS protection isn't really the same thing after all...
One question, what's current best practice for DDOS prevention on my own VPS Ubuntu box ?
Sometimes DDoS is 'layer 7', running you out of CPU etc on requests that are easy to make and hard to service. Try to avoid that?
Otherwise, DDoS is usually volumetric: send you more packets than will fit on your network interface. The only prevention is larger interfaces, but 1Gbps of DDoS was readily available when I was dealing with it in 2018ish and it was pretty clear that people were using the site I ran as a test target for DDoS as a service (always exactly 90 seconds of junk traffic on our www, very rarely a real service host, etc). There was a recent crackdown on DDoS as a service, but I'd be surprised if 10Gbps isn't easily available now.
Otherwise, DDoS is usually volumetric: send you more packets than will fit on your network interface. The only prevention is larger interfaces, but 1Gbps of DDoS was readily available when I was dealing with it in 2018ish and it was pretty clear that people were using the site I ran as a test target for DDoS as a service (always exactly 90 seconds of junk traffic on our www, very rarely a real service host, etc). There was a recent crackdown on DDoS as a service, but I'd be surprised if 10Gbps isn't easily available now.
[deleted]
D/DOS protections mostly have to happen before the network packet reaches the OS. Handling the incoming data request requires enough OS resources to be used for DOS. There are some things you can do application wise, such as avoiding reflection and amplification attacks. https://blog.cloudflare.com/reflections-on-reflections/
Nothing, if your self hosting in your home. Volumetric floods will saturate your ISP link.
If you’re hosting with a provider, your maximum factor will be how much your provider will “tank” for you.
Otherwise harden your ports, drop anything via IPTables, turn on NOTRACK. Better but more advanced would be to use tc (traffic control) to drop bad packets before they enter the net filter lifecycle
If you’re hosting with a provider, your maximum factor will be how much your provider will “tank” for you.
Otherwise harden your ports, drop anything via IPTables, turn on NOTRACK. Better but more advanced would be to use tc (traffic control) to drop bad packets before they enter the net filter lifecycle
Don’t get DDoS’d or use a provider that has built-in DDoS protection.
Depending on what you’re using it for you could “cloak” it.
Depending on what you’re using it for you could “cloak” it.
What if the situation is the provider doesn't have built-in DDos protection ?
Depends what you are protecting. A website or http traffic? Stick it behind cloudflare. Services on other ports or protocols like TCP or UDP? You could rent a cheap VPS at a provider that DOES have inline protection and use that instance to reroute traffic to your own server via a GRE tunnel.
Then you switch providers, go behind something like cloud flare, or contract with a DDoS protection provider (there aren’t many).
If it is an unimportant service you just suffer the DDoS or switch IPs.
Or you use a front end on a VPS that does have DDoS and use a IPv6 tunnel or tail scale to connect to your actual service.
If it is an unimportant service you just suffer the DDoS or switch IPs.
Or you use a front end on a VPS that does have DDoS and use a IPv6 tunnel or tail scale to connect to your actual service.
Well you just have to find a way to eat the traffic without using up too many resources. Rate-limit by IP, drop certain types of packets, cache aggressively, respond to 400 errors with empty response, timeout long-running requests etc.
[deleted]
I get that someone has to pay for this, but if I had 50 servers here and they all got shit on, I'd be on the hook for a lot of money through no fault of my own.