Hackers invade safety system, halt plant operations in ‘watershed’ cyberattack(japantimes.co.jp)
japantimes.co.jp
Hackers invade safety system, halt plant operations in ‘watershed’ cyberattack
https://www.japantimes.co.jp/news/2017/12/15/world/crime-legal-world/hackers-invade-safety-system-halt-mideast-plant-operations-watershed-cyberattack/
52 comments
> Why are systems like this connected to the internet?
Probably a variety of reasons. Here's one.
There was something I saw on my local PBS station that was about security and the electrical grid. They talked to some manager of a major power plant who had a phone app that could monitor that plant over the internet and pretty much completely control it.
They talked about how this app came about. The manager decided it would be convenient so he could keep up on things when he was away at conferences and such. There were no security experts involved in writing this thing, or in analyzing what the risks were of adding outside internet access to the LANs that control systems were on. It was all as casual as you or I might make a phone app that can turn on our garage lights.
The thought that there might be some danger in the control systems being directly accessible from the internet simply did not occur to anyone.
Probably a variety of reasons. Here's one.
There was something I saw on my local PBS station that was about security and the electrical grid. They talked to some manager of a major power plant who had a phone app that could monitor that plant over the internet and pretty much completely control it.
They talked about how this app came about. The manager decided it would be convenient so he could keep up on things when he was away at conferences and such. There were no security experts involved in writing this thing, or in analyzing what the risks were of adding outside internet access to the LANs that control systems were on. It was all as casual as you or I might make a phone app that can turn on our garage lights.
The thought that there might be some danger in the control systems being directly accessible from the internet simply did not occur to anyone.
The saddest thing about this is that manager likely got a bonus, either directly (great initiative!) or indirectly (more hours micromanaging everything), when he should have gotten fired and found it impossible to get a job in the industry.
Any network security authority worth their salt would tell this manager (and those like him) that "checking up on things" is a hopeless and useless endeavor. Like all log monitoring systems, only actionable items should be presented to end-users.
Set thresholds on monitoring systems where the monitors trigger alarms when thresholds are outside of normal operating parameters. It's monitoring 101.
Set thresholds on monitoring systems where the monitors trigger alarms when thresholds are outside of normal operating parameters. It's monitoring 101.
If I'm not mistaken, there is no mention in the article that the system was actually connected to the Internet.
The Stuxnet attack (which as another commenter said was the real "watershed" event) managed to infect the target system even if it was air-gapped, by infecting the PC of someone that eventually connected to the private network of the target onsite.
https://en.wikipedia.org/wiki/Stuxnet#Windows_infection
The Stuxnet attack (which as another commenter said was the real "watershed" event) managed to infect the target system even if it was air-gapped, by infecting the PC of someone that eventually connected to the private network of the target onsite.
https://en.wikipedia.org/wiki/Stuxnet#Windows_infection
Stuxnet propagated itself, on a pretty wide scale, through a BadUsb exploit to bypass the Iranian air-gap.
Okay, then really the comment should be renamed to why is critical infrastructure allow to be tampered with. Say this infection was brought in via hardware or bluetooth exploitation or wifi exploitation. None of those attack vectors should ever be accessible. No human working there should ever be able to "by accident" infect the system.
That's nice in theory, but perhaps you have to change the logic of that SIS. Perhaps some parts need to be decommissioned, or added to. A security hot patch is needed or a backup taken.
There are thousands of reasons someone may need to connect to a system
There are thousands of reasons someone may need to connect to a system
Why do you assume that none of those attack vectors should ever be accessible? It's reasonable to assume that the systems need to exchange data with external systems on a daily basis, possibly on a real-time basis.
They most likely need remote monitoring and reporting, they can need data that needs to be periodically inserted from an outside system, they might even need remote real-time control. Any such accessibility is a possible attack vector, and "security at all costs" isn't reasonable - even if the operation is literally priceless, you still need to balance the security risks of malicious attacks versus the increased risks of downtime or faults caused by more difficult/slower monitoring and control due to the security measures you implement; a strict airgap might prevent an attack but be a contributing factor in a non-malicious accident that's just as disastrous.
They most likely need remote monitoring and reporting, they can need data that needs to be periodically inserted from an outside system, they might even need remote real-time control. Any such accessibility is a possible attack vector, and "security at all costs" isn't reasonable - even if the operation is literally priceless, you still need to balance the security risks of malicious attacks versus the increased risks of downtime or faults caused by more difficult/slower monitoring and control due to the security measures you implement; a strict airgap might prevent an attack but be a contributing factor in a non-malicious accident that's just as disastrous.
The article says "In the incident, hackers used sophisticated malware to take remote control of a workstation running a Schneider Electric Triconex safety shutdown system"
"Remote control" certainly implies that it was connected at least indirectly to the internet, though something like a direct dial-up modem or leased line compromise could also be possible.
"Remote control" certainly implies that it was connected at least indirectly to the internet, though something like a direct dial-up modem or leased line compromise could also be possible.
It's the IT folks who tell everybody you should connect everything to the Internet, because that makes everything so easy.
It's the embedded folks, who tell that it is not made for this.
It's the manager folks, who demand that it be connected, because the IT folks told that it's is easy and innovative while the embedded folks are so old school.
It's the embedded folks, who tell that it is not made for this.
It's the manager folks, who demand that it be connected, because the IT folks told that it's is easy and innovative while the embedded folks are so old school.
> It's the embedded folks, who tell that it is not made for this.
Really?
If so, this is just a small minority of the embedded folks.
Think of the whole "Internet of Things" security mess, combined with a large part of "Industry 4.0". That's tons of embedded devices explicitly built to be connected to the Internet.
If it were really the developers of those embedded devices who didn't want them to have Internet connection, they would never have been in charge to build those in the first place.
Really?
If so, this is just a small minority of the embedded folks.
Think of the whole "Internet of Things" security mess, combined with a large part of "Industry 4.0". That's tons of embedded devices explicitly built to be connected to the Internet.
If it were really the developers of those embedded devices who didn't want them to have Internet connection, they would never have been in charge to build those in the first place.
Traditional embedded systems used micro controllers. Or for plants they used PLCs for automation. People have been warning for years that connecting factory automation (or power plant control) to the network gave it an interface to the whole world. IT insisted that it would be on a subnet or in some other way "safe". And besides, no hackers in the PC space knew anything about the hardware and software in these systems.
The world is converging to point where everything is just an SoC capable of running a full OS and applications. IoT is just marketing taking advantage of the capability.
The world is converging to point where everything is just an SoC capable of running a full OS and applications. IoT is just marketing taking advantage of the capability.
>explicitly built
Just because a network interface is part of the hardware design does not make it "explicitly built" for use on the internet. On the software side 99.999% of these products are a security mess and only manage to dodge that reputation because the market is so fragmented that automated ways to bypass the security on these devices with no fine tuning involved does not yet exist.
Just because a network interface is part of the hardware design does not make it "explicitly built" for use on the internet. On the software side 99.999% of these products are a security mess and only manage to dodge that reputation because the market is so fragmented that automated ways to bypass the security on these devices with no fine tuning involved does not yet exist.
Best of all, they tend to have mini web servers exposing device APIs, completely written in straight C code.
> Best of all, they tend to have mini web servers exposing device APIs, completely written in straight C code.
Well of course, that's what the 10 year old vendor SDK uses, why do extra work? /s
The state of industrial/embedded security in general is so poor I'm surprised there aren't more stories like this coming out. To give a concrete example, the industrial world was using modbus since the late 1970s and when Ethernet came around in the 1980s someone thought "well let's just encapsulate modbus data in TCP" and thus was born ModbusTCP. Not a single thought was given to security.
There's millions if not billions of dollars of industrial equipment using ModbusTCP today that cannot be secured without being replaced completely. I've yet to see anyone pushing a secure protocol to replace ModbusTCP in industrial environments.
When a utility is hacked (not if) and the attackers get into the SCADA network, it's going to cause a bloodbath of epic proportions.
Well of course, that's what the 10 year old vendor SDK uses, why do extra work? /s
The state of industrial/embedded security in general is so poor I'm surprised there aren't more stories like this coming out. To give a concrete example, the industrial world was using modbus since the late 1970s and when Ethernet came around in the 1980s someone thought "well let's just encapsulate modbus data in TCP" and thus was born ModbusTCP. Not a single thought was given to security.
There's millions if not billions of dollars of industrial equipment using ModbusTCP today that cannot be secured without being replaced completely. I've yet to see anyone pushing a secure protocol to replace ModbusTCP in industrial environments.
When a utility is hacked (not if) and the attackers get into the SCADA network, it's going to cause a bloodbath of epic proportions.
The internet is becoming the de facto way to connect anything. There's increasing automation, communication, and so forth between all these systems, and the internet is the way that's coordinated. But unfortunately the people taking these systems online don't always do so with the proper eye toward security rigor that is deserved. Sometimes that's down to a lack of expertise. Sometimes it's due to lack of resources or spending. Sometimes it's just because management folks refuse to listen to the professionals they've hired. In any event, it's a common problem even at the best of places. At organizations that lack a deep bench of current era IT/software talent (which is actually most organizations) it's very easy to make these mistakes.
If you want to see how to do it right, look at slot machines. There is a crazy amount of oversight on those devices. In comparison, voting machines and our industrial infrastructure is just a huge game of grab-ass. Whatever goes, goes, until something bad happens and then people slowly learn one lesson.
If you want to see how to do it right, look at slot machines. There is a crazy amount of oversight on those devices. In comparison, voting machines and our industrial infrastructure is just a huge game of grab-ass. Whatever goes, goes, until something bad happens and then people slowly learn one lesson.
Why are systems like this connected to the internet?
Consider an automated warehouse: https://www.youtube.com/watch?v=dAXdeqcHBp4
Internet <-> Website <-> Stock control system <-> Stock inbound station <-> Conveyor control system <-> Emergency stop monitoring system
It's possible to make such a system safe, of course. But a company making a web browser has their defences tested and their security spending validated by unceasing attack. The same resources might be hard to get in an industrial automation company if the higher-ups think "there hasn't been an attack, and they already have a firewall, a VPN and a virus scanner, whatever those are"
You don't make an online purchase when you open your tap to get a drink of water or flip on a switch for the lights.
Some systems need to be on the internet. cool story, but, why would critical infrastructure such as this need to be is still unaddressed by your comment.
Some systems need to be on the internet. cool story, but, why would critical infrastructure such as this need to be is still unaddressed by your comment.
With a smart meter, effectively you do.
Metering is not a critical/safety component of the grid. It's an important business component, but it does not by itself guarantee safety or preclude safety measures from kicking in.
There are other systems that are used to monitor load/draw on the grid and ensure safe operation.
Things like smart meters can be connected to the public net and if compromised just cause a headache.
Your critical safety system should not be connected to public networks or at least should have strong protections.
There are other systems that are used to monitor load/draw on the grid and ensure safe operation.
Things like smart meters can be connected to the public net and if compromised just cause a headache.
Your critical safety system should not be connected to public networks or at least should have strong protections.
no, you don't. When I activate a light switch and consume more power from the grid I am not directly adding more coal/fuel to the fire/generator. This is where the warehousing analogy breaks.
Those numbers are looked at on aggregate to determine how much electricity needs to be produced to supply the population. They are directly tied into finance departments to generate your bill, sure, but that is an independent system from the production of electricity.
In the case of water supply, same thing. Just because I turn on my tap doesn't mean I am directly affecting the water treatment plant to treat exactly 1 more cup of water today.
Those numbers are looked at on aggregate to determine how much electricity needs to be produced to supply the population. They are directly tied into finance departments to generate your bill, sure, but that is an independent system from the production of electricity.
In the case of water supply, same thing. Just because I turn on my tap doesn't mean I am directly affecting the water treatment plant to treat exactly 1 more cup of water today.
The emergency stop system should not be connected to the normal conveyor control system. It should be an independent system, since in an emergency the normal controls are presumably malfunctioning.
Because all kinds of companies are pushing for the Industrial Internet of Things, since that enables new subscription-type business models for sectors that used to sell big expensive machines ~once per customer.
https://en.wikipedia.org/wiki/Internet_of_things#Manufacturi...
https://en.wikipedia.org/wiki/Internet_of_things#Manufacturi...
Yeah but then expose to it at least with a VPN... doesn't even have to be a Cisco ASA, a simple box with OpenVPN on it is sufficient - hell, as long as you're not doing heavy traffic on it a $20 throwaway router can do this.
A VPN doesn't help very much if the computer you use to log in is compromised. If security were as simple as putting everything behind a VPN there would be far fewer problems.
Yeah but the stuff at least doesn't show up on Shodan for everyone to grab/hack. You'd have to compromise exactly the right computer, drastically increasing the attack difficulty.
Why do you assume that this isn't what happened in this case?
It is usually a result of a fight between control system engineers and IT people. The IT people see an ethernet cable and think that they should control it and in order to do that they connect the two networks. There are ways to do it safely, like placing a database in a DMZ that the PLCs write to and have computers on the other side read from the database without ever connecting to the PLCs, but that requires expertise and a security mindset.
Is it wrong for me to want those IT folks reprimanded or reassigned?
It's like they don't understand the first rule of Fight Club or something.
It's like they don't understand the first rule of Fight Club or something.
You would think that wouldn't you. But for that to happen, the system designer would have had to plan it that way probably 20 years ago.
This was predictable 20 years ago too though.
On that note, the Christmas Tree EXEC is 30 years old this month!
https://en.wikipedia.org/wiki/Christmas_Tree_EXEC
https://en.wikipedia.org/wiki/Christmas_Tree_EXEC
> There is no good reason for any of these devices to have a public IP address.
That's a bold statement to make for someone who, I'm assuming, has absolutely no idea what these systems are or what the company wants to do with them.
Security policy is super easy when you don't actually have to get anything done.
That's a bold statement to make for someone who, I'm assuming, has absolutely no idea what these systems are or what the company wants to do with them.
Security policy is super easy when you don't actually have to get anything done.
As somebody who is responsible for security policy: there is no good reason for any of these devices to have a public IP address. Full stop. We did fine back when they weren't networked, we don't need to start now.
What do you know about these devices or the company's requirements?
You're making a lot of assumptions because you're a know-it-all on the internet.
You're making a lot of assumptions because you're a know-it-all on the internet.
If a safety system is on the Internet, it is no longer safe. I don't see why this is so hard for people to understand - if these things get hacked, best-case is a bunch of actual physical productivity is lost; worst-case is that people fucking die.
You idiots don't stop to think about if you should do something, just because you can. Internal networks with an airgap are somewhat more acceptable, but the public Internet or accessible from a device that can be pivoted to is completely unacceptable.
You idiots don't stop to think about if you should do something, just because you can. Internal networks with an airgap are somewhat more acceptable, but the public Internet or accessible from a device that can be pivoted to is completely unacceptable.
Assumption made: the safety system mentioned had anything at all to do with the safety of human beings. For all we know, this is an entirely lights-out operation.
From the article, it sounds more like these were systems designed to protect the equipment, which they did. A loss of productivity due to equipment failing or being destroyed has an associated cost. If that cost, factoring in the probability of that event, is less than the cost of not having these things remotely accessible (however that is determined), then the math is pretty clear about what choice you make.
Even if humans are involved, that's just another cost to factor in. People get killed in work accidents all the time.
This is the problem with "security people". Their only care in life is stopping attacks, so they don't give a damn what the cost is and how it relates to the rest of the organization. We take calculated risks every single day, and if we didn't, nothing would ever get done.
From the article, it sounds more like these were systems designed to protect the equipment, which they did. A loss of productivity due to equipment failing or being destroyed has an associated cost. If that cost, factoring in the probability of that event, is less than the cost of not having these things remotely accessible (however that is determined), then the math is pretty clear about what choice you make.
Even if humans are involved, that's just another cost to factor in. People get killed in work accidents all the time.
This is the problem with "security people". Their only care in life is stopping attacks, so they don't give a damn what the cost is and how it relates to the rest of the organization. We take calculated risks every single day, and if we didn't, nothing would ever get done.
>From the article, it sounds more like these were systems designed to protect the equipment, which they did. A loss of productivity due to equipment failing or being destroyed has an associated cost. If that cost, factoring in the probability of that event, is less than the cost of not having these things remotely accessible (however that is determined), then the math is pretty clear about what choice you make.
People are not good at figuring out probabilities of events like being hacked - and that's assuming somebody even thought about that as a possibility, which is not at all a given in an industrial setting. The smaller factories that have clung to life here in the Midwest, for example, basically have one 'computer guy' who knows enough to keep things going.
>Even if humans are involved, that's just another cost to factor in. People get killed in work accidents all the time.
This reads like you think that people dying is acceptable risk to take. I'm going to leave it at that, lest I just call you a bunch of unpleasant names for an entire paragraph.
>This is the problem with "security people". Their only care in life is stopping attacks
It's as if that's my fucking job or something, eh?
>so they don't give a damn what the cost is and how it relates to the rest of the organization. We take calculated risks every single day, and if we didn't, nothing would ever get done
Sure we care. An airgapped internal network for this sort of thing would be an acceptable compromise for accessing something remotely. The public Internet is stupid to the point of gross irresponsibility.
People are not good at figuring out probabilities of events like being hacked - and that's assuming somebody even thought about that as a possibility, which is not at all a given in an industrial setting. The smaller factories that have clung to life here in the Midwest, for example, basically have one 'computer guy' who knows enough to keep things going.
>Even if humans are involved, that's just another cost to factor in. People get killed in work accidents all the time.
This reads like you think that people dying is acceptable risk to take. I'm going to leave it at that, lest I just call you a bunch of unpleasant names for an entire paragraph.
>This is the problem with "security people". Their only care in life is stopping attacks
It's as if that's my fucking job or something, eh?
>so they don't give a damn what the cost is and how it relates to the rest of the organization. We take calculated risks every single day, and if we didn't, nothing would ever get done
Sure we care. An airgapped internal network for this sort of thing would be an acceptable compromise for accessing something remotely. The public Internet is stupid to the point of gross irresponsibility.
> An airgapped internal network for this sort of thing would be an acceptable compromise for accessing something remotely.
Do you even listen to yourself? Do you understand the concept of "remote"?
>This reads like you think that people dying is acceptable risk to take.
Yes, clearly this is a ridiculous notion. No one ever signs up for things that carry a risk of death like driving, being a soldier, or getting shot into space.
Do you even listen to yourself? Do you understand the concept of "remote"?
>This reads like you think that people dying is acceptable risk to take.
Yes, clearly this is a ridiculous notion. No one ever signs up for things that carry a risk of death like driving, being a soldier, or getting shot into space.
>> Why are systems like this connected to the internet?
For convenience. In business they call it "cost".
For convenience. In business they call it "cost".
yup, they should NOT be connecting them, but because everyone is going cloud and remote service support.... they got to let "people" in.
Also the PLC administrators refuse to let proper network and systems admins go near their shit. They think their stuff is better left alone from the "geeks".
Also the PLC administrators refuse to let proper network and systems admins go near their shit. They think their stuff is better left alone from the "geeks".
One of my relatives is a safety engineer at a plant.
Remotely he can connect to a read only console, if there are any changes he has to call someone on the inside to make them.
I don’t know how robust that read only aspect is, but it seems like a good middle ground if faithfully implemented.
I don’t know how robust that read only aspect is, but it seems like a good middle ground if faithfully implemented.
So, a virus-based cyber attack on the computer system of a Saudi Arabian industrial operation, supposedly by Iran.
If "watershed" mean fundamentally new, this isn't a watershed event since this event was proceed by the Stuxnet attack on Iran, by many accounted committed by the US and Israel.
Still, it seems to reinforce the general situation that the gloves are off between nation states, every cyber avenue of attack that can be pursued, will be pursued - a war of all against all. Since not only are cyber attacks cheap, they offer endless plausible deniability.
If "watershed" mean fundamentally new, this isn't a watershed event since this event was proceed by the Stuxnet attack on Iran, by many accounted committed by the US and Israel.
Still, it seems to reinforce the general situation that the gloves are off between nation states, every cyber avenue of attack that can be pursued, will be pursued - a war of all against all. Since not only are cyber attacks cheap, they offer endless plausible deniability.
"Watershed moments" are those that cause everyone to take something seriously instead of beliving it couldn't happen to them. It's about people's reactions.
I still think we're a long way out from the real "watershed" moment, and I say that as a lifelong security engineer.
Nothing fundamental changed after Stuxnet. Nothing fundamental is changing from this. Nothing fundamental changed after Target, Home Depot, Equifax, nothing. We're still vulnerable to the exact same types of attacks in the exact same way, saved only by the fact that easier targets exist to draw away the attention of attackers. These "watershed" attacks only happen when someone specifically wants to go after that target as opposed to just any target.
We've been at cyber WWIII for a long time, nation-state attackers are nothing new. Problem is, loss of life is the only reason we have any reservations at all about attacking other nations. Until cyber war ends the life of a Western media-friendly victim, we will continue not caring.
Actual human casualties will be the real watershed moment.
Nothing fundamental changed after Stuxnet. Nothing fundamental is changing from this. Nothing fundamental changed after Target, Home Depot, Equifax, nothing. We're still vulnerable to the exact same types of attacks in the exact same way, saved only by the fact that easier targets exist to draw away the attention of attackers. These "watershed" attacks only happen when someone specifically wants to go after that target as opposed to just any target.
We've been at cyber WWIII for a long time, nation-state attackers are nothing new. Problem is, loss of life is the only reason we have any reservations at all about attacking other nations. Until cyber war ends the life of a Western media-friendly victim, we will continue not caring.
Actual human casualties will be the real watershed moment.
This isn't a watershed - Stuxnet kind of was, but the people who didn't start to take it seriously after Stuxnet will still believe it won't happen to them after this case.
Stuxnet was a virus that was sent in. It's not the same as direct remote control.
If malicious logical access is used to destroy industrial machinery, potentially killing operators, does it really matter if it came in via network or connection or physical access?
Yes.
You could do the same thing by slipping bugs into place at the ends of wires, and the difference would definitely matter for opsec.
You could do the same thing by slipping bugs into place at the ends of wires, and the difference would definitely matter for opsec.
"The malware, which FireEye has dubbed Triton, is only the third type of computer virus discovered to date that is capable of disrupting industrial processes."
This sounds incorrect ... Wasn't there (at least):
[¹] https://www.pandasecurity.com/cyprus/homeusers/security-info...
This sounds incorrect ... Wasn't there (at least):
1) Stuxnet (2010)
2) Shamoon (2012)
3) HAVEX (2013)
4) BlackEnergy (2015)
5) Industroyer (CrashOveride) (2016)
6) Triton (??)
[¹] https://www.pandasecurity.com/cyprus/homeusers/security-info...
I got quite a sense of whiplash just in the first page of the article. It is an article from Japan Times, with a leading photo from California, over a byline from Toronto, about an attack on equipment from a French firm in a site in Saudi Arabia.
...just in case you forgot that the internet is international.
...just in case you forgot that the internet is international.
As far as I can see, any safety monitoring system should be air-gapped, or, if remote control is absolutely needed, be connected via a robust physical interface, i.e a thick cable.
There is no good reason for any of these devices to have a public IP address. Physical access should be the only attack vector available when it comes to industrial sabotage.