Internet traffic for Google, Facebook, Apple was briefly rerouted to Russia(bgpmon.net)
bgpmon.net
Internet traffic for Google, Facebook, Apple was briefly rerouted to Russia
https://bgpmon.net/popular-destinations-rerouted-to-russia/
115 comments
It's unfair to say it's a "feature". It was an oversight back in 1989 based on nobody expecting we'd end up where we are today.
Filtering is easy if your just doing it for downstream customers.
But how do Tier 1's filter from each other? Once you go up the tree a few times it gets really hard to build any sort of valid filter list of what you should be getting from provider X or provider Y.
BGPSEC and RPKI are unfortunately not really workable yet, in terms of full-path validation. And obviously the source data to do it is a long way off.
Filtering is easy if your just doing it for downstream customers.
But how do Tier 1's filter from each other? Once you go up the tree a few times it gets really hard to build any sort of valid filter list of what you should be getting from provider X or provider Y.
BGPSEC and RPKI are unfortunately not really workable yet, in terms of full-path validation. And obviously the source data to do it is a long way off.
I just have to ask, not knowing how to setup BGP but having a lot of experience with PF and ipfw; is it not possible to create a sort of default deny filter so you only exclude the routes you have?
The way your post read it sounded like you had to create a filter for all routes in the world but I assume that's not the case in the practical sense.
The way your post read it sounded like you had to create a filter for all routes in the world but I assume that's not the case in the practical sense.
You only need to create filters for the routes you know, e.g. if you are an ISP providing internet connectivity for company X, then you know that the only advertisement from the link to X can be for X and if the link to company X advertises routes from Google, Facebook or Apple AS's, then that's bogus.
It's just tedious to do this, and a lot of providers, especially smaller ones don't bother with it.
It's just tedious to do this, and a lot of providers, especially smaller ones don't bother with it.
creating a list of 'expected' receive routes and expected 'send' routes is fairly easy to create.
aka outbound and inbound prefix lists, I quickly googled and found the following config [0] that shows them 'advertising' their owned networks to prevent advertising networks they should not be and also accepting other networks they expect. this prevents and would have mitigated this sort of situation if implemented everywhere.
no I do not believe this to be difficult and anyone configuring BGP should have some awareness of most of its tools.
you can live your whole life without having to use a fire extinguisher or understanding the differences between a,b,c,d etc types but if you are a fire (man/woman) I expect you to understand which is used in what circumstance. if you do not you have the chance to spread a fire and make it worse then prevent it the same with someone who calls themself a network engineer.
[0] https://supportforums.cisco.com/legacyfs/online/legacy/0/5/4...
no I do not believe this to be difficult and anyone configuring BGP should have some awareness of most of its tools.
you can live your whole life without having to use a fire extinguisher or understanding the differences between a,b,c,d etc types but if you are a fire (man/woman) I expect you to understand which is used in what circumstance. if you do not you have the chance to spread a fire and make it worse then prevent it the same with someone who calls themself a network engineer.
[0] https://supportforums.cisco.com/legacyfs/online/legacy/0/5/4...
I understand ISP's are to blame here, but how is it not about Russia, the article specifically states that only certain entities were targeted in a suspicious manner. Seems like nefarious purposes at work here no?
Yes.
The particular networks announced, and particularly the fact that more specific routes were announced, suggests this is some nefarious incident. Not just an accident that someone announced routes they shouldn't have.
The particular networks announced, and particularly the fact that more specific routes were announced, suggests this is some nefarious incident. Not just an accident that someone announced routes they shouldn't have.
>I understand ISP's are to blame here, but how is it not about Russia, the article specifically states that only certain entities were targeted in a suspicious manner. Seems like nefarious purposes at work here no?
Sure there are. But compare this to an organization getting owned because they didn't patch a known vulnerability. We generally focus on the company that was breached because the power to stop such a breach was in their hands. The same can be said in this situation. ISPs are not proactively taking known measures to minimize the impact of a BGP hijack.
Malicious actors will be malicious...it's what they do. If it's not Russia, it'll be someone else.
Sure there are. But compare this to an organization getting owned because they didn't patch a known vulnerability. We generally focus on the company that was breached because the power to stop such a breach was in their hands. The same can be said in this situation. ISPs are not proactively taking known measures to minimize the impact of a BGP hijack.
Malicious actors will be malicious...it's what they do. If it's not Russia, it'll be someone else.
> how is it not about Russia
It could be, the key missing piece of information is the physical location of the actual router that is the ultimate source of this. It is only known by their peers or upstream and is not easily traceable (in the US it's usually under NDA and you'd need court order to find out "officially"). Some network admin somewhere probably knows. May be he/she reads HN. The second question is whether that device is hacked and is doing it unbeknown to its admin.
But without this information, the fact that this is an AS registered with a bogus .ru domain doesn't mean anything, this could be coming from Liechtenstein or Bolivia.
It could be, the key missing piece of information is the physical location of the actual router that is the ultimate source of this. It is only known by their peers or upstream and is not easily traceable (in the US it's usually under NDA and you'd need court order to find out "officially"). Some network admin somewhere probably knows. May be he/she reads HN. The second question is whether that device is hacked and is doing it unbeknown to its admin.
But without this information, the fact that this is an AS registered with a bogus .ru domain doesn't mean anything, this could be coming from Liechtenstein or Bolivia.
The open trusting nature of BGP is kind of problematic isn’t it?
https://en.wikipedia.org/wiki/BGP_hijacking
https://en.wikipedia.org/wiki/BGP_hijacking
Yes, but to be honest, it's not just BGP.
One of the things that BGPMon won't tell you about since they're in the business of monitoring BGP is that the same attack is often performed by changing forwarding tables i.e. in network's data plane.
An attacker compromises a router (there are known vulnerabilities and back-doors) or gains leverage over a technician authorized to configure routers in some network. The attacker then manually changes the router's local routing table to forward packets to specific destinations through a different path. In such cases, the BGP path and announcements look absolutely fine and you'd need to look at the actual path at the data plane level to detect that something is wrong.
This is often done in big exchange points where many networks interconnect. These are places in which routers from major western countries can sit basically side-by-side with routers from Russia and China.
This isn't a paranoid fantasy by the way, I work for a company that monitors these kinds of attacks for a living.
An attacker compromises a router (there are known vulnerabilities and back-doors) or gains leverage over a technician authorized to configure routers in some network. The attacker then manually changes the router's local routing table to forward packets to specific destinations through a different path. In such cases, the BGP path and announcements look absolutely fine and you'd need to look at the actual path at the data plane level to detect that something is wrong.
This is often done in big exchange points where many networks interconnect. These are places in which routers from major western countries can sit basically side-by-side with routers from Russia and China.
This isn't a paranoid fantasy by the way, I work for a company that monitors these kinds of attacks for a living.
> This isn't a paranoid fantasy by the way, I work for a company that monitors these kinds of attacks for a living.
Then you'll certainly be familiar with some citation or documentation of evidence? I'm curious to see some.
Then you'll certainly be familiar with some citation or documentation of evidence? I'm curious to see some.
I've asked around the company and unfortunately I can't talk about attacks (and/or suspicious "configuration errors") we've detected. There are legal issues involved, plus some of these cases are currently being investigated by law enforcement. Moreover, I don't know of any other company or person that deals with such attacks, so I have no one to quote. The only thing I can do is to offer some hand waving:
1. It's known publicly that nation states as well as criminal organizations are deflecting Internet routes. There are numerous reports of such cases; these cases aren't that hard to find since BGP information is (mostly) public. A few published examples:
- Russian network "Rostelecom" hijacks sites of financial services: https://arstechnica.com/information-technology/2017/04/russi...
- Global Large scale BGP hijacks in 2013, some through "Rostelecom": https://arstechnica.com/information-technology/2013/11/repea...
- Spammers use BGP to announce fake IP addresses to spam yahoo mail users: https://ripe72.ripe.net/presentations/45-Invisible_Hijacking...
- Traffic to UK organization that deals with nuclear weapons hijacked using BGP to the Ukraine: http://hub.dyn.com/dyn-research/uk-traffic-diverted-through-...
- BGP hijack of bitcoin miners: https://bgpmon.net/the-canadian-bitcoin-hijack/
- "Hacking Team" (Italian company selling spyware to law enforcement) helps Italian police perform BGP hijack: https://arstechnica.com/information-technology/2015/07/hacki...
- Chinese hijack of US military an governmental networks (2010): http://www.theregister.co.uk/2010/11/17/bgp_hijacking_report...
2. There are known attacks against routing protocols which aren't BGP (e.g. OSPF, Black Hat 2011) and against routers (see CVEs for CISCO IOS).
3. Routing changes that happen within an autonomous system leave a much smaller footprint compared to BGP.
4. Nation states are known to have an interest in direct access into routers:
- Suspected NSA example: http://www.theregister.co.uk/2015/09/15/compromised_cisco_ro...
- Suspected Chinese example: http://www.abovetopsecret.com/forum/thread350381/pg1
5. Network connect to each other (mainly) in central exchange points. In these exchanges networks can pay for a direct point-to-point connection from network to network, or use open peering through a layer 2 switch that aggregates connections from dozens or hundreds of different networks at once (Example: https://www.de-cix.net/en/de-cix-service-world/globepeer). Hundreds of networks use these switches with very little visibility. Everyone can talk to everyone else on the same switch without leaving a trace. If a router connected to such a switch is locally configured to forward traffic to some other router on the same switch, it can do so regardless of BGP routing or any common sense.
1. It's known publicly that nation states as well as criminal organizations are deflecting Internet routes. There are numerous reports of such cases; these cases aren't that hard to find since BGP information is (mostly) public. A few published examples:
- Russian network "Rostelecom" hijacks sites of financial services: https://arstechnica.com/information-technology/2017/04/russi...
- Global Large scale BGP hijacks in 2013, some through "Rostelecom": https://arstechnica.com/information-technology/2013/11/repea...
- Spammers use BGP to announce fake IP addresses to spam yahoo mail users: https://ripe72.ripe.net/presentations/45-Invisible_Hijacking...
- Traffic to UK organization that deals with nuclear weapons hijacked using BGP to the Ukraine: http://hub.dyn.com/dyn-research/uk-traffic-diverted-through-...
- BGP hijack of bitcoin miners: https://bgpmon.net/the-canadian-bitcoin-hijack/
- "Hacking Team" (Italian company selling spyware to law enforcement) helps Italian police perform BGP hijack: https://arstechnica.com/information-technology/2015/07/hacki...
- Chinese hijack of US military an governmental networks (2010): http://www.theregister.co.uk/2010/11/17/bgp_hijacking_report...
2. There are known attacks against routing protocols which aren't BGP (e.g. OSPF, Black Hat 2011) and against routers (see CVEs for CISCO IOS).
3. Routing changes that happen within an autonomous system leave a much smaller footprint compared to BGP.
4. Nation states are known to have an interest in direct access into routers:
- Suspected NSA example: http://www.theregister.co.uk/2015/09/15/compromised_cisco_ro...
- Suspected Chinese example: http://www.abovetopsecret.com/forum/thread350381/pg1
5. Network connect to each other (mainly) in central exchange points. In these exchanges networks can pay for a direct point-to-point connection from network to network, or use open peering through a layer 2 switch that aggregates connections from dozens or hundreds of different networks at once (Example: https://www.de-cix.net/en/de-cix-service-world/globepeer). Hundreds of networks use these switches with very little visibility. Everyone can talk to everyone else on the same switch without leaving a trace. If a router connected to such a switch is locally configured to forward traffic to some other router on the same switch, it can do so regardless of BGP routing or any common sense.
> These are places in which routers from major western countries can sit basically side-by-side with routers from Russia and China.
Are you saying that Russia and China should be worried?
Are you saying that Russia and China should be worried?
Everyone should be.
It's surprising how vulnerable and fragile Internet routing is.
It's surprising how vulnerable and fragile Internet routing is.
Yes, and, to be fair, it's been a known problem for years. There are some potential solutions in the works but you can't just pick an arbitrary day and flip a switch and turn it on.
Most of these incidents are accidental and could easily be prevented. The network operator community can't even get everyone to implement BCP38, though, so something like a full Internet-wide RPKI is a pipe dream. Even basic filtering would prevent much of this but that's apparently asking too much as well.
The amount of filtering, if any, varies tremendously from network to network. Some won't accept routes without IRR entries, LOAs, and a bunch of paperwork. Others (such as AS31133, apparently) will accept any prefix you want and as many of 'em as you want.
I'll give (some of) the Tier 1's a bit of a break, as filtering on every single BGP session just isn't feasible (although some of them wouldn't bother filtering even if they could!). Pretty much everyone else should be filtering.
But they aren't. And they won't be any time soon. And so we'll continue to have incidents like this for the foreseeable future.
Most of these incidents are accidental and could easily be prevented. The network operator community can't even get everyone to implement BCP38, though, so something like a full Internet-wide RPKI is a pipe dream. Even basic filtering would prevent much of this but that's apparently asking too much as well.
The amount of filtering, if any, varies tremendously from network to network. Some won't accept routes without IRR entries, LOAs, and a bunch of paperwork. Others (such as AS31133, apparently) will accept any prefix you want and as many of 'em as you want.
I'll give (some of) the Tier 1's a bit of a break, as filtering on every single BGP session just isn't feasible (although some of them wouldn't bother filtering even if they could!). Pretty much everyone else should be filtering.
But they aren't. And they won't be any time soon. And so we'll continue to have incidents like this for the foreseeable future.
Why can't higher Tier ISPs require that each prefix they get from downstream must be recorded in some DB and do the lookup before [propagating the prefix and] installing the forwarding table entries?
They generally do for their direct end-customers who announces their own address space.
The problem is when peering with other Tier 1's and large ISPs, with multiple downstream customers. How do they keep and update a list of the other ISPs customers and the routes they should expect from them?
Once someone manages to get a bogus route into a large ISPs table it tends to propagate further.
The problem is when peering with other Tier 1's and large ISPs, with multiple downstream customers. How do they keep and update a list of the other ISPs customers and the routes they should expect from them?
Once someone manages to get a bogus route into a large ISPs table it tends to propagate further.
I work for a medium sized ISP, we have about 500 peers over exchanges like LONAP and Linx. You'd be surprised how many of our peers won't use MD5 passwords or even filter properly.
Name and shame?
Sorry, no, I don’t really see how that contributes to the discussion.
Probably he's still working at there, so he'd be forbidden to disclose customers name.
BGP peerings are publically published, that's how they work.
Knowing which customers fail to secure their comms is another matter.
Knowing which customers fail to secure their comms is another matter.
> Knowing which customers fail to secure their comms is another matter.
Exactly what I meant.
Exactly what I meant.
Bad idea.
These are customers he's talking about. If you want to stay in business, naming and shaming your customers isn't a good strategy.
They propagate a few prefixes toward you, right?
So you have to filter what you accept from them, no?
What mechanisms do you have for this?
If it's a lookup, why it's not feasible for higher Tier ISPs?
So you have to filter what you accept from them, no?
What mechanisms do you have for this?
If it's a lookup, why it's not feasible for higher Tier ISPs?
'normally' it gets routed to US (even if they have servers elsewhere to handle same shit...), which for most countries is actually more worrying...
If you believe that Russia is less worrying than US, then... err... I don't believe you exist actually.
Well, if you are a security researcher, Russia is less apt to redirect your flight mid ocean and arrest you in a country you weren't even going to. So I guess the US just might be more worrying to particular people based upon their occupation.
s/security researcher/Sharepoint admin/
Security researcher is a pretty loose and fear mongering title to describe Snowden's actual role - Taking sensitive government secrets to a hostile foreign power in the name of whistleblowing.
But please, don't let my clarification get in the way of your whataboutism.
But please, don't let my clarification get in the way of your whataboutism.
'to a hostile foreign power'? You mean the public? The US government is why Snowden is in Russia, not Snowden. He didn't go to Russia. His flight was forced to land there when the US revoked his passport mid-flight.
This is incorrect, nobody was forced to land. He had a stopover in Russia (as he was flying Aeroflot).
Passport was not revoked mid-flight. It was supposed to be handled earlier, but it looks like a clerical error caused issues:
> Though the U.S. maintains it provided all the necessary facts, on Tuesday Hong Kong Secretary for Justice Rimsky Yuen told reporters that there was a discrepancy between U.S. and Hong Kong records over Snowden’s full name and that his department never received Snowden’s passport number, which it had requested.
(http://world.time.com/2013/06/25/snowdens-hong-kong-escape-w...)
Passport was not revoked mid-flight. It was supposed to be handled earlier, but it looks like a clerical error caused issues:
> Though the U.S. maintains it provided all the necessary facts, on Tuesday Hong Kong Secretary for Justice Rimsky Yuen told reporters that there was a discrepancy between U.S. and Hong Kong records over Snowden’s full name and that his department never received Snowden’s passport number, which it had requested.
(http://world.time.com/2013/06/25/snowdens-hong-kong-escape-w...)
Yes, to a hostile foreign power. The public isn't a place you can reside. Russia is, however.
> The US government is why Snowden is in Russia, not Snowden
The US government is not why Snowden is in Russia, Snowden is why Snowden is in Russia. He went to Russia intentionally. He wasn't kidnapped, he chose Russia as a destination specifically due to their reluctance to cooperate with the US.
> He didn't go to Russia. His flight was forced to land there...
That's not what happened, please reread his flight history because the facts are well known at this point. He told his superiors at the NSA that he was headed to the mainland USA for medical treatment, but instead flew direct to Hong Kong where he lived for over a month. 30+ days after arriving in HK he fled to Russia, with the intent of fleeing to Cuba the next day.
> ...when the US revoked his passport mid-flight.
His flight wasn't forced to land mid-flight because Snowden never even boarded the plane that day. His passport was revoked before the Cuba trip, leaving him stranded in the Russian airport.
If you're going to shill for someone who is unwilling to come back to the US to be held accountable (rightly or wrongly) for his actions, please be factually correct about it.
> The US government is why Snowden is in Russia, not Snowden
The US government is not why Snowden is in Russia, Snowden is why Snowden is in Russia. He went to Russia intentionally. He wasn't kidnapped, he chose Russia as a destination specifically due to their reluctance to cooperate with the US.
> He didn't go to Russia. His flight was forced to land there...
That's not what happened, please reread his flight history because the facts are well known at this point. He told his superiors at the NSA that he was headed to the mainland USA for medical treatment, but instead flew direct to Hong Kong where he lived for over a month. 30+ days after arriving in HK he fled to Russia, with the intent of fleeing to Cuba the next day.
> ...when the US revoked his passport mid-flight.
His flight wasn't forced to land mid-flight because Snowden never even boarded the plane that day. His passport was revoked before the Cuba trip, leaving him stranded in the Russian airport.
If you're going to shill for someone who is unwilling to come back to the US to be held accountable (rightly or wrongly) for his actions, please be factually correct about it.
>The US government is not why Snowden is in Russia, Snowden is why Snowden is in Russia. He went to Russia intentionally. He wasn't kidnapped, he chose Russia as a destination specifically due to their reluctance to cooperate with the US.
He chose Russia as a layover location, Hong Kong has a limited number of outgoing flights and that likely was the one the US would be least able to grab him. However, the US is why he is in Russia now. You mention that he had a flight to Cuba booked, and his ultimate goal was Ecuador.
There is no proof that he took "state secrets" to Russia. By his account he had destroyed any of his remaining copies before leaving Hong Kong. This is a far more egregious error than confusing a layover stop for being forced to land.
He chose Russia as a layover location, Hong Kong has a limited number of outgoing flights and that likely was the one the US would be least able to grab him. However, the US is why he is in Russia now. You mention that he had a flight to Cuba booked, and his ultimate goal was Ecuador.
There is no proof that he took "state secrets" to Russia. By his account he had destroyed any of his remaining copies before leaving Hong Kong. This is a far more egregious error than confusing a layover stop for being forced to land.
Who said anything about Snowden? There are many different professions that are legal in their country of operation that the US will redirect international flights to US controlled locations to arrest you. It seems like you can't accept that the US is a threat to some people's and nations sovereignty.
> Who said anything about Snowden?
You implicitly implied him.
> There are many different professions that are legal in their country of operation that the US will redirect international flights to US controlled locations to arrest you.
Which ones? Please provide sources of researchers who have had their flights redirected with the sole purpose of arresting the individuals.
> It seems like you can't accept that the US is a threat to some people's and nations sovereignty.
It seems like you struggle with providing facts and evidence as opposed to conjecture and hypotheticals.
You implicitly implied him.
> There are many different professions that are legal in their country of operation that the US will redirect international flights to US controlled locations to arrest you.
Which ones? Please provide sources of researchers who have had their flights redirected with the sole purpose of arresting the individuals.
> It seems like you can't accept that the US is a threat to some people's and nations sovereignty.
It seems like you struggle with providing facts and evidence as opposed to conjecture and hypotheticals.
In 1998, a hacker collective known as the L0pht testified in Congress that they could take down the Internet in 30 minutes.
It is believed BGP black holes could be one way of achieving this.
Basically tell every node "I've got the lowest routing cost to every node, send me all your traffic", and then drop all the traffic.
It is believed BGP black holes could be one way of achieving this.
Basically tell every node "I've got the lowest routing cost to every node, send me all your traffic", and then drop all the traffic.
This is much better nowadays but bgp is still problematic.
It's not really better today. Possibly even worse.
In 1998 there were fewer than 5000 different networks (Autonomous systems) on the Internet. Today that number is over 80K. Network operators used to personally know each other and their clients.
So the automatic filtering is better today than it was in 1998, but it can still be easily bypassed.
I know for a fact that even as a tiny network, all you need to do to get the some of the biggest networks in the world to accept your (real or fake) BGP announcement is to email their support center.
In 1998 there were fewer than 5000 different networks (Autonomous systems) on the Internet. Today that number is over 80K. Network operators used to personally know each other and their clients.
So the automatic filtering is better today than it was in 1998, but it can still be easily bypassed.
I know for a fact that even as a tiny network, all you need to do to get the some of the biggest networks in the world to accept your (real or fake) BGP announcement is to email their support center.
Aren't ASNs capped at 2^15 (65k and change)? Or has the IPv6 expansion already started happening?
Could someone explain in laymen terms what happened and what that means?
The internet works by organisations "announcing" the IP ranges they're allocated to other networks (ISPs, other large companies etc.) Networks that receive an announcement will route traffic for those IP ranges to the network that announced them.
The protocol by which this is done is called BGP. The original version of this was created in 1989 and back then everyone involved trusted each other. So there is no security built in to prevent a rogue company announcing addresses they don't own, and therefore pulling in traffic people send to those addresses.
Organisations can and do put filters in place to try and stop this, but because it's not a core baked in feature of BGP not everyone does, and issues can happen.
Sometimes these things are just mistakes engineers make, re-announcing routes they've learnt elsewhere. Sometimes they are deliberate hi-jacks.
The protocol by which this is done is called BGP. The original version of this was created in 1989 and back then everyone involved trusted each other. So there is no security built in to prevent a rogue company announcing addresses they don't own, and therefore pulling in traffic people send to those addresses.
Organisations can and do put filters in place to try and stop this, but because it's not a core baked in feature of BGP not everyone does, and issues can happen.
Sometimes these things are just mistakes engineers make, re-announcing routes they've learnt elsewhere. Sometimes they are deliberate hi-jacks.
Basically, a router somewhere in Russia claimed to be the owner of some ip addresses belonging to Google, Facebook, etc. Other neighbouring routers began forwarding packets from actual users to this router. The packets contain the HTTPS requests people were making to these sites.
Bit late to the party but what could be done with the information that they got their hands on?
Internet traffic goes through whatever path will get it to it's destination fastest. Big Routers advertise to each other how long it will take to get somewhere through them. There have been times when either due to bugs, or due to a desire to slurp traffic, some machines have falsely advertised shorter routes causing traffic to pas through routes traffic wouldn't normally pass through. This time it was somewhere in Russia. This can be mitigated like most malicious acts / network errors if you write clever enough rules, but is hard because you have to identify the bad actors, or paths that lead to them.
> but is hard because you have to identify the bad actors, or paths that lead to them.
It's worse than that. There are few truly "bad actors". Many BGP hijacks are performed by completely legitimate networks, which later claim that it was a configuration issue.
It's worse than that. There are few truly "bad actors". Many BGP hijacks are performed by completely legitimate networks, which later claim that it was a configuration issue.
If this is not a reason to punish (blackhole?) that AS I don't know what is.
Sure, it's a reason to blackhole the isp, but this is the Internet we're talking about. So say its upstream providers will backhole it, say even the tier 1 providers do the same.
That won't stop it from connecting to smaller networks and announcing the same fake routes there. And even if the small networks blackhole it, the attackers can just register a new AS.
Also, there are cases in which attackers are performed from large networks, with millions of users. Are you going to blackhole such a network? They'll apologize and claim that it was a configuration error.
Note that these types of attacks are generally not performed by amateurs. They're performed by states, intelligence agencies, criminal organizations and the like.
That won't stop it from connecting to smaller networks and announcing the same fake routes there. And even if the small networks blackhole it, the attackers can just register a new AS.
Also, there are cases in which attackers are performed from large networks, with millions of users. Are you going to blackhole such a network? They'll apologize and claim that it was a configuration error.
Note that these types of attacks are generally not performed by amateurs. They're performed by states, intelligence agencies, criminal organizations and the like.
Is it fair to say BGP in general relies on “good will” trust model? I wonder if there exists an authoritative mechanism similar to browser punishing rouge or incompetent CA. I supposed the integrity and the authentication is achieved by implementing and deploying RPKI, but what about punishment?
> Is it fair to say BGP in general relies on “good will” trust model?
Yes.
> I wonder if there exists an authoritative mechanism similar to browser punishing rouge or incompetent CA
Not really. Although if you get a reputation for being notoriously problematic, the "big boys" - the large networks who are the responsible adults in this game might deny you service or stop accepting your announcements.
> I supposed the integrity and the authentication is achieved by implementing and deploying RPKI
RPKI doesn't really solve the issue. It validates only the initial announcer, so BGP hijacks can still occur. It would however help reduce hijacks due to mistakes in configuration.
Yes.
> I wonder if there exists an authoritative mechanism similar to browser punishing rouge or incompetent CA
Not really. Although if you get a reputation for being notoriously problematic, the "big boys" - the large networks who are the responsible adults in this game might deny you service or stop accepting your announcements.
> I supposed the integrity and the authentication is achieved by implementing and deploying RPKI
RPKI doesn't really solve the issue. It validates only the initial announcer, so BGP hijacks can still occur. It would however help reduce hijacks due to mistakes in configuration.
It's better than it used to be. Back in the mid-90s I was a newly-minted network engineer working in my first datacenter and we were all very excited about this new "BGP" technology...
Fifteen minutes later, half of Fairfax County is stuck in a routing loop, and we're getting calls from NOCs as far away as London asking us what the f* we're doing to their traffic.
That was the wild west: back then, a typo could (and sometimes did) actually take down The Internet. There's more filtering and checking and second-guessing today. But, ultimately, it's based on a web of trust model, you're right, with all the weaknesses that come with that.
Fifteen minutes later, half of Fairfax County is stuck in a routing loop, and we're getting calls from NOCs as far away as London asking us what the f* we're doing to their traffic.
That was the wild west: back then, a typo could (and sometimes did) actually take down The Internet. There's more filtering and checking and second-guessing today. But, ultimately, it's based on a web of trust model, you're right, with all the weaknesses that come with that.
What is a prefix (in context), and what does it mean to ‘announce’ one?
A prefix is an IP subnet, basically.
Most here are familiar with Google's public DNS resolver, 8.8.8.8, so let's use that as an example.
Google has been assigned the autonomous system number (ASN) 15169. The 8.0.0.0/8 IP address space was allocated to Level3 and they have reallocated a small part of it, 8.8.8.0/24, to Google.
Google, when speaking to their BGP peers, "announces" routes, including an announcement for the 8.8.8.0/24 prefix.
An announcement is basically Google's router telling, for example, Level3's router, "Hey, I know how to get to 8.8.8.0/24. If you have traffic going there, you can send it to me."
Level3 then passes that along to other peers, like me. Level3 says, "If you have traffic for 8.8.8.0/24, you can send it to me and I'll send it on towards its origin". The announcements continue to be passed along to other peers.
A withdrawal is the opposite of an announcement: "Hey, I don't have a route to 8.8.8.0/24 anymore" or, in many cases, "the route to 8.8.8.0/24 has changed, here's the new one".
Here's a (slimmed down) example from one of my routers:
Here's the same thing for 66.232.224.0/24, mentioned in the article:
With regard to the hijack described in the article, this means that another organization announced the prefixes into BGP, effectively saying, "So, all of that traffic you have that's going to Apple/Facebook/Google/etc., just go ahead and start sending that to AS39523 now". Instead of ending up at Apple/Facebook/Google, it'll instead be redirected to this unknown organization in Russia.
(N.B.: Before anyone chimes in to "correct" something, this is a bit simplified. There's much more to BGP than this and there are a number of different factors which determine which route is chosen. A longer prefix -- mentioned in the article -- is one way to "defeat" a better route.)
Most here are familiar with Google's public DNS resolver, 8.8.8.8, so let's use that as an example.
Google has been assigned the autonomous system number (ASN) 15169. The 8.0.0.0/8 IP address space was allocated to Level3 and they have reallocated a small part of it, 8.8.8.0/24, to Google.
Google, when speaking to their BGP peers, "announces" routes, including an announcement for the 8.8.8.0/24 prefix.
An announcement is basically Google's router telling, for example, Level3's router, "Hey, I know how to get to 8.8.8.0/24. If you have traffic going there, you can send it to me."
Level3 then passes that along to other peers, like me. Level3 says, "If you have traffic for 8.8.8.0/24, you can send it to me and I'll send it on towards its origin". The announcements continue to be passed along to other peers.
A withdrawal is the opposite of an announcement: "Hey, I don't have a route to 8.8.8.0/24 anymore" or, in many cases, "the route to 8.8.8.0/24 has changed, here's the new one".
Here's a (slimmed down) example from one of my routers:
# sh ip bgp 8.8.8.0/24 | b 3356
3356 15169, (received & used)
4.69.248.15 from 4.69.248.15 (4.69.180.167)
Origin IGP, metric 0, localpref 100, valid, external, best
Community: 3356:3 3356:86 3356:575 3356:666 3356:2042
Here's the same thing for 66.232.224.0/24, mentioned in the article:
# sh ip bgp 66.232.224.0/24
BGP routing table entry for 66.232.224.0/24, version 1125190207
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Advertised to update-groups:
4
3356 209 40839, (received & used)
4.69.248.15 from 4.69.248.15 (4.69.180.167)
Origin IGP, metric 0, localpref 100, valid, external, best
Community: 209:209 209:13070 3356:3 3356:22 3356:86 3356:575 3356:666 3356:2042
With regard to the hijack described in the article, this means that another organization announced the prefixes into BGP, effectively saying, "So, all of that traffic you have that's going to Apple/Facebook/Google/etc., just go ahead and start sending that to AS39523 now". Instead of ending up at Apple/Facebook/Google, it'll instead be redirected to this unknown organization in Russia.
(N.B.: Before anyone chimes in to "correct" something, this is a bit simplified. There's much more to BGP than this and there are a number of different factors which determine which route is chosen. A longer prefix -- mentioned in the article -- is one way to "defeat" a better route.)
One curious thing about 8.8.8.8 (and 8.8.4.4) is that it gets redirected to the closest Google CDN.
> One curious thing
DNS is something that’s commonly Anycasted. In laymen’s terms, basically what that means is rather than 8.8.8.8 pointing to one server (or a load balanced cluster in one DC) as you might expect, it rather points to potentially thousands of servers all over the globe and the closest one (BGP route wise) is the one picked for you.
DNS is something that’s commonly Anycasted. In laymen’s terms, basically what that means is rather than 8.8.8.8 pointing to one server (or a load balanced cluster in one DC) as you might expect, it rather points to potentially thousands of servers all over the globe and the closest one (BGP route wise) is the one picked for you.
8.8.8.8 is already an IP address so DNS is not involved. The anycast is handled by just BGP.
I think the parent meant that anycast is often used for DNS servers, not that a DNS lookup is used to find the closest Google DNS server.
Thanks for this explanation.
In BGP you have sets of IP destinations, which you summarise as a prefix. Announcing or advertising means exactly what it sounds like, you announce to your peers that you have paths to these networks.
Sometimes it's slightly simpler to show rather than tell.
The CIDR Report is a publicly accessible compendium of BGP routing announcements. Mind that this isn't the entire Internet, or even entirely current, though it's a very large and generally current version of it. One of the complications is that you're only seeing the routes the CIDR Report itself is aware of (there may be others, though generally those will be small/obscure).
But if you want to know what "the Internet" is, this is pretty much it: a bunch of rules for routing data.
http://www.cidr-report.org/as2.0/
As for BGP, Border Gateway Protocol:
https://en.wikipedia.org/wiki/Border_Gateway_Protocol
The routes advertised by the AS (autonomous system) in question, AS39523:
http://www.cidr-report.org/cgi-bin/as-report?as=AS39523&view...
(There are a maximum of 2^16 == 65,536 autonomous systems possible under BGP, at least until those counters are extended. What happens under IPv6 may be a bit of a mess....)
Edit: There's 4 billion and change as ASNs are now 32 bit, see: https://news.ycombinator.com/item?id=15915417
The CIDR Report is a publicly accessible compendium of BGP routing announcements. Mind that this isn't the entire Internet, or even entirely current, though it's a very large and generally current version of it. One of the complications is that you're only seeing the routes the CIDR Report itself is aware of (there may be others, though generally those will be small/obscure).
But if you want to know what "the Internet" is, this is pretty much it: a bunch of rules for routing data.
http://www.cidr-report.org/as2.0/
As for BGP, Border Gateway Protocol:
https://en.wikipedia.org/wiki/Border_Gateway_Protocol
The routes advertised by the AS (autonomous system) in question, AS39523:
http://www.cidr-report.org/cgi-bin/as-report?as=AS39523&view...
(There are a maximum of 2^16 == 65,536 autonomous systems possible under BGP, at least until those counters are extended. What happens under IPv6 may be a bit of a mess....)
Edit: There's 4 billion and change as ASNs are now 32 bit, see: https://news.ycombinator.com/item?id=15915417
[deleted]
basically, a more specific routing address is used to direct packets through a specific route. In the earlier instance( the second example) Japan was the target. It is easy to see how the routing system in japan could have been crashed through such an overload (basically a ddos). Who would do that to Japan? China? Korea?
In the more recent instance, all the traffic from google, facebook, apple, and other major players was routed through russia. Is this china making russia look evil? after all, if the traffic is going through russia, all russia has to do is turn off the power for that router to momentarily stop all traffic.
or is this the NSA trying to #uck w russia and at the same time trying to create an incident so the fcc kills net neutrality to provide 'security'.
This is state of the art espionage
In the more recent instance, all the traffic from google, facebook, apple, and other major players was routed through russia. Is this china making russia look evil? after all, if the traffic is going through russia, all russia has to do is turn off the power for that router to momentarily stop all traffic.
or is this the NSA trying to #uck w russia and at the same time trying to create an incident so the fcc kills net neutrality to provide 'security'.
This is state of the art espionage
This is not the first time when BGP is misused this way. I think governments should regularly perform live tests to check whether their countries' AS's can be hijacked.
> I think governments should regularly perform live tests to check whether their countries' AS's can be hijacked.
That's a waste of money and effort. They should just write "yes" on a piece of paper, there's really no reason to check.
That's a waste of money and effort. They should just write "yes" on a piece of paper, there's really no reason to check.
Basic security. Like how everyone failed at keeping Equifax in check.
All AS's can be hijacked. Your suggestion is meaningless.
[deleted]
That AS was registered to dv-hyperlink.ru which is not in whois database
Looks suspicious. How did they register an AS to a non-existing company to begin with?
Looks suspicious. How did they register an AS to a non-existing company to begin with?
It was assigned by RIPE to "Vasilyev Ivan Ivanovich, 8 Pionerskaya st., office 10, Nekrasovka, Khabarovsk region Russia".
ALFA TELECOM s.r.o., in the Czech Republic seem to have acted as sponsoring LIR for the allocation.
RIPE rules require that the sponsoring LIR submit documents to them showing the legal standing of the organisation that will use the resources. So in theory that should have happened here.
ALFA TELECOM s.r.o., in the Czech Republic seem to have acted as sponsoring LIR for the allocation.
RIPE rules require that the sponsoring LIR submit documents to them showing the legal standing of the organisation that will use the resources. So in theory that should have happened here.
I always wonder how Russian people, either Russian Americans or Russian citizens, feel about all this Russia hysteria.
Putin's approval rating is in the 80s.
That should be enough to tell you what they think of you without looking at polls.
(IIRC they were trying to shut down the one independent poll firm recently)
That should be enough to tell you what they think of you without looking at polls.
(IIRC they were trying to shut down the one independent poll firm recently)
[deleted]
[deleted]
[deleted]
Isn't that what https is for ? So no one can read the traffic ?
Even with encrypted traffic, an attacker can learn things, especially if you get DNS and SNI data
I suspect (but cannot prove) that this might have been a leak from Russia's internal internet into the wider global net. Since Russia isn't well known for it's privacy respecting nature, it might have been a traffic scanner to see if people are being good citizens. However, that is just speculation and I hope it's wrong.
I suspect (but cannot prove) that this might have been a leak from Russia's internal internet into the wider global net. Since Russia isn't well known for it's privacy respecting nature, it might have been a traffic scanner to see if people are being good citizens. However, that is just speculation and I hope it's wrong.
Yep, all you need to know is the IP addresses of certain domains (say Facebook) and then look for user IP transferring a lot of data to it meaning they are probably uploading a photo. Now tie that IP to an ISP and maybe a user and you can find out who might be posting derogatory memes about Putin.
But you don't have to change routing for that - you can do that with just passive monitoring. And by the way, the law [1] that requires ISPs to store up to 6 months worth of traffic is coming into effect next year. So even monitoring won't be necessary.
Maybe they were testing effective ways to block foreign sites?
[1] https://en.wikipedia.org/wiki/Yarovaya_law
Maybe they were testing effective ways to block foreign sites?
[1] https://en.wikipedia.org/wiki/Yarovaya_law
> Since Russia isn't well known for it's privacy respecting nature, it might have been a traffic scanner to see if people are being good citizens.
Yes, indeed! Guilty until proven guilty! Keep up the upstanding attitude, good citizen!
Yes, indeed! Guilty until proven guilty! Keep up the upstanding attitude, good citizen!
China did the same but for bigger scale in the past. The reason to do this is to store all traffic for future analysis. Now it is protected by https but there is no protocol or cryptography which couldn't be exploited in the future.
https://arstechnica.com/information-technology/2010/11/how-c...
https://arstechnica.com/information-technology/2010/11/how-c...
Google themselves have done the same thing https://www.theregister.co.uk/2017/08/27/google_routing_blun....
But as we all know, NSA does store all traffic for future analysis. A BGP leak from China or Russia, be that as it may, almost surely has nothing to do with storing traffic.
But as we all know, NSA does store all traffic for future analysis. A BGP leak from China or Russia, be that as it may, almost surely has nothing to do with storing traffic.
> But as we all know, NSA does store all traffic for future analysis.
{{Citation needed}}
{{Citation needed}}
Don't be daft. People said the same thing about the NSA sniffing traffic before Snowden, and big fucking surprise, it turned out to be true.
Nothing in Snowden's docs says that the NSA was collecting and storing all the traffic it could find for later analysis, which was GGP's unsupported claim.
Isn't forward secrecy helpful in protecting against exactly this kind of thing?
Are you sure your apps fail to connect if they don't negotiate forward secrecy?
That only protects against breaches of the long-term key, not against attacks on the cipher itself, AFAIK. So if in the future they manage to attack AES (or whatever cipher the connection was using), forward secrecy won't help.
If you use 1024 bit DH with a common group (old/misconfigured web and email servers do this) then it is suspected nation states can break the DH, get the shared symmetric key and decrypt all traffic.
For ECDHE over P-256, they would need to wait for a big quantum computer (which will break all recorded traffic that used a non-quantum resistant key exchange, which is all current traffic).
For ECDHE over P-256, they would need to wait for a big quantum computer (which will break all recorded traffic that used a non-quantum resistant key exchange, which is all current traffic).
Assuming that state level actors don’t have ways to get around https? I’d expect a nation like Russia could acquire or procure new key for the targeted domains or perform some unknown 0 day attack on the targeted companies to acquire the original key.
Not with certificate pinning , which google certainly has, and I’d be surprised if the other two didn’t. It is an issue for smaller sites (which might have been the real target)
I don't think IE supports pinning, though, so if they could reliably detect the browser at the TLS handshake stage (don't know if it's possible) they could in theory serve their own cert to those users.
> so if they could reliably detect the browser at the TLS handshake stage (don't know if it's possible)
It's possible and easy, the list of ciphers in the ClientHello is different. Take a look at https://www.ssllabs.com/ssltest/clients.html to see what several popular browsers look like.
It's possible and easy, the list of ciphers in the ClientHello is different. Take a look at https://www.ssllabs.com/ssltest/clients.html to see what several popular browsers look like.
They still could perform a previously unknown attack to grab the original key. There could be a side channel attack that we are unaware of that could give out enough information to reconstruct the key. Heck I just saw a post today about an Oracle attack on TLS called ROBOT that a lot of big players are vulnerable to.
Performing a DNS hijack has more uses than just reading the traffic. It gives you information on the amount of data that is sent towards a given prefix (among other uses, this can help for creating DDoS attacks later). It also makes those prefixes unavailable for as long as the attack persists, which is already a form of denial of service.
hopefully Comcast and AT&T will fix this soon
notsgnik(1)
I understand that less network savvy readers think this story is about Russia, but it really is about ISP's and the old issue of lazy network admins who do not filter routes and what happens. You can replace "Russia" with "Canada" and it wouldn't change the meaning. The country attribution is based on the AS (Autonomous System) registration. If these were really the malicious state actors, they'd probably use an AS registered in another country to cover their tracks. This is either an honest mistake or possibly a "let's see what would happen" prank.