Hit by Ransomware Attack, Florida City Agrees to Pay Hackers $600k(nytimes.com)
nytimes.com
Hit by Ransomware Attack, Florida City Agrees to Pay Hackers $600k
https://www.nytimes.com/2019/06/19/us/florida-riviera-beach-hacking-ransom.html
117 comments
How can you separate necessary security upgrades out from the $18m? $100k wouldn't be enough to maintain a single critial app securely for more than a couple of years.
If you're running a bunch of critical services you need to secure them, and that takes millions every year, it's unavoidable. A single hack that exposes this is irrelevant
If you're running a bunch of critical services you need to secure them, and that takes millions every year, it's unavoidable. A single hack that exposes this is irrelevant
It is true that the cause of the pain is that they haven't been running their IT in a secure fashion, and this is the real problem.
I don't know how much of the money/time that will be spent (the "$18 million" figure is entirely imaginary) to instead try to recover their data/systems to functional... but I don't think it's a great use of money.
But indeed, actually running secure systems is what they got to be focusing on, and will be expensive. I think many organizations are going to find that they can't actually afford to do what's needed to reliably run the systems they already rely on, having not been running them reliably or securely.
I hope that the 'recovery' efforts, done in an emergency fashion, don't distract them from getting to that point and figuring out what they're going to do about it.
I don't know how much of the money/time that will be spent (the "$18 million" figure is entirely imaginary) to instead try to recover their data/systems to functional... but I don't think it's a great use of money.
But indeed, actually running secure systems is what they got to be focusing on, and will be expensive. I think many organizations are going to find that they can't actually afford to do what's needed to reliably run the systems they already rely on, having not been running them reliably or securely.
I hope that the 'recovery' efforts, done in an emergency fashion, don't distract them from getting to that point and figuring out what they're going to do about it.
If you pay a ransomware ransom, you can't be assured that they're no longer present on the network. There's the possibility that they ransom the firm again or cause further damage. The firm would have to run a full set of incident response regardless of paying the ransom or not.
I'm not saying that Baltimore is handling the situation correctly but there would be costs well above the x hundred thousand dollars the ransomers would be asking for either way.
I'm not saying that Baltimore is handling the situation correctly but there would be costs well above the x hundred thousand dollars the ransomers would be asking for either way.
If it's that easy to blackmail cities I need to take up hacking...
If you're looking for ways to make money and aren't concerned about legal risk or ethics, certainly I'd expect hacking to be on your list of possibilities. There wouldn't be so much hacking going on if it wasn't lucrative.
It actually isn't very lucrative.
Some of the crypto-blackmailers use public bitcoin addresses, and they don't get as much money as you'd expect.
For the fact you'd probably have to move your life to Russia to escape the FBI, it doesn't pay very well.
Some of the crypto-blackmailers use public bitcoin addresses, and they don't get as much money as you'd expect.
For the fact you'd probably have to move your life to Russia to escape the FBI, it doesn't pay very well.
Could they have avoided the ransom by having daily (or hourly) backups to non-rewritable (write once) media? So the malware won’t encrypt the backups, obviously.
I think that the last day’s work (or last hour’s work) will be lost or will require a lot of manual fixing regardless of whether they pay the ransom. If they pay, they’ll still have to fix partial database transactions, corrupt files, etc., for the attack date. If they don’t pay, they can recover from earlier good backups and reconstruct that one day’s worth. My reasoning is that the attack date’s data is going to be corrupt and untrustworthy in either case, and it’ll be equal work either way. (Or at least it’ll be less than $600,000 of work to fix that one day.)
I imagine that they either weren’t doing backups at all, or their backups were directly accessible and writable by the malware.
I think that the last day’s work (or last hour’s work) will be lost or will require a lot of manual fixing regardless of whether they pay the ransom. If they pay, they’ll still have to fix partial database transactions, corrupt files, etc., for the attack date. If they don’t pay, they can recover from earlier good backups and reconstruct that one day’s worth. My reasoning is that the attack date’s data is going to be corrupt and untrustworthy in either case, and it’ll be equal work either way. (Or at least it’ll be less than $600,000 of work to fix that one day.)
I imagine that they either weren’t doing backups at all, or their backups were directly accessible and writable by the malware.
I work for the city of Skanderborg in Denmark and we do a few things to avoid it. One is to monitor our entire storage for malicious code and automatically isolate suspicious activity. Another is to do frequent backups of everything on our network shares and one drive for business which is where most employee data that doesn’t belong in a specific system lives. Our servers, database clusters and vital systems are all isolated from the employee network and also frequently backed up. When we get hit, it’s usually employees reading private email and we’re typically able to isolate the ransomware before it spreads from that specific employees network share. Once we kill it, we restore files to the most recent backup and roll their machine.
It’s worked well so far, but we do have an IT crew that would make most places jealous and IT is an area that is notoriously undervalued in the public sector.
It’s worked well so far, but we do have an IT crew that would make most places jealous and IT is an area that is notoriously undervalued in the public sector.
See this seems to be a really fun and challenging part of IT.. the part that I don't want to have to deal with is the helpdesk, guiding a 60 year old lady to login onto her system or configure her company iPhone.
Maybe there is a job where you do only the former, I just haven't found it yet.
Maybe there is a job where you do only the former, I just haven't found it yet.
But guiding that 60 year old lady through configuring her company phone is how you discover that your process for configuring company phones is broken. When the people who design a system never interact with the people who use the system, that is how you end up with shitty unusable IT systems.
If you are unwilling to get your hands dirty interacting with the people who use your network you'll be unaware of how your decisions affect their day to day lives and consequently they'll probably end up hating you, which will not do you any favors in the long run. It is important to remember that everything you do is ultimately in service to helping them do their jobs.
Network/infrastructure guys don't seem to me to do helpdesk very often?
It depends on the size of the company.
What do you think happens when a core core router breaks? The level one guy fixes it?
There are a few Palo Alto and Cisco guys near me. They definitely aren’t answering the help desk phone but they are level 3 support. Same with my development team.
Not to mention large orgs have NOCs dedicated to this. The last company I was at at a couple CCAs in the NOC. They were smart but an odd breed.
There are a few Palo Alto and Cisco guys near me. They definitely aren’t answering the help desk phone but they are level 3 support. Same with my development team.
Not to mention large orgs have NOCs dedicated to this. The last company I was at at a couple CCAs in the NOC. They were smart but an odd breed.
These will be separate jobs in any large enough IT org.
I imagine that they either weren’t doing backups at all, or their backups were directly accessible and writable by the malware.
Or they weren't able to detect exactly when the malware was deployed so they didn't know how far back the data corruption went, meaning they couldn't trust the backups even if they looked OK. One of the problems you face after any attack is trusting the system again. Verifying everything is correct is a very hard problem.
Or they weren't able to detect exactly when the malware was deployed so they didn't know how far back the data corruption went, meaning they couldn't trust the backups even if they looked OK. One of the problems you face after any attack is trusting the system again. Verifying everything is correct is a very hard problem.
I think there is alot more going on here than just some encrypted files.
The interim IT manager had to setup a new website for the city w/ new email addresses?
They lost control of DNS and registration of their domain, it seems.
SCADA systems for water pumps were inoperable.
Online payments no longer functioned.
This seems like a very premeditated attack to me.
The interim IT manager had to setup a new website for the city w/ new email addresses?
They lost control of DNS and registration of their domain, it seems.
SCADA systems for water pumps were inoperable.
Online payments no longer functioned.
This seems like a very premeditated attack to me.
Knowing all the "flat" networks I've seen that all sounds par for the course. Segmentation is still exotic (and met with either disdain or blank stares by software vendors). The attack damaged all those systems because, very likely, they were all accessible to each other and under one locus of control (probably a single Active Directory domain).
More than likely the SCADA systems themselves were fine, but the PCs for managing those systems were AD joined and people couldn't login to actually run the management software.
Its very common in custom hardware setups to have a standing system that interfaces with the physical hardware and PLC and then the user friendly software for instructing that controller on what you want to be on a PC talking to it over serial or the network. Obviously if the computer is inaccessible you can't adjust settings, but the system continues to run fine.
Its very common in custom hardware setups to have a standing system that interfaces with the physical hardware and PLC and then the user friendly software for instructing that controller on what you want to be on a PC talking to it over serial or the network. Obviously if the computer is inaccessible you can't adjust settings, but the system continues to run fine.
Right, but the story states otherwise.
SCADA functioning is not dependent on AD.
The AD server should be able to be destroyed without preventing the water supply from functioning.
I know of three small municipalities in my area (smaller than this town), and the utilities are not part of the flat network.
The AD server should be able to be destroyed without preventing the water supply from functioning.
I know of three small municipalities in my area (smaller than this town), and the utilities are not part of the flat network.
A very good solution here is ZFS. Office file server runs ZFS and makes regular ZFS snapshots (hourly, or more often). Snapshots are immutable so even if the employee machines get infected, nothing can be lost (except the last N minutes since the last snapshot).
Obviously, still have to do backups (off of those snapshots).
I absolutely love ZFS, but I think it's only a piece of the puzzle necessary for mitigating this threat. In terms of data integrity, I sure hope no system running ZFS would have it's snapshots also corrupted, but if that did happen, then hopefully those ZFS snapshots would have been backed up to some offline media like tape or optical disk. Example: http://girlyngeek.blogspot.com/2014/02/zfs-on-read-only-devi...
> Snapshots are immutable so even if the employee machines get infected
Not really. You can easily extract data and overwrite the disk surface.
Not really. You can easily extract data and overwrite the disk surface.
Not from the machines which typically get infected, which are the employee desktops accessing the file server.
If the malware manages to get itself running on the file server itself (and with root privileges), then sure. That's not a common case though.
If the malware manages to get itself running on the file server itself (and with root privileges), then sure. That's not a common case though.
What if the malware puts itself between the hardware and the filesystem?
Pull instead of push backups are one way of trying to mitigate this. You dont allow clients to start backups. Your backup server does instead, "pulling" backups at standard times.
As long as you harden it from crypto, there is no way for malware on client machines to force an overwrite of current backups.
As long as you harden it from crypto, there is no way for malware on client machines to force an overwrite of current backups.
The malware could wait for a pull, then feed the server false data.
Then you have a backup server that's privileged to access data from clients, which makes the backup server an attractive target.
Then you reduce the attack surface on this one special-purpose machine, which is much easier than doing the same on all your employees' desktops.
Very clever.
If their backups are being placed somewhere like S3 they should have a stand alone server that simply copies daily backups from the S3 bucket the network has access to, over to another S3 bucket that only the backup managing app has access to.
I expect they are allowing their backup app to have read write access to manage cleaning up removing backups but just giving it write only access from the network would work too. And using a standalone server/app to manage the backups.
I'd still copy over a backup to a stand alone bucket not accessible by the network for something this critical.
I expect they are allowing their backup app to have read write access to manage cleaning up removing backups but just giving it write only access from the network would work too. And using a standalone server/app to manage the backups.
I'd still copy over a backup to a stand alone bucket not accessible by the network for something this critical.
Most of these places have their backups on directory joined machines with storage that is badly configured. Backup systems should never be joined to the domain specifically for this reason--but it makes configuration harder so most places skip it.
We use frequent ZFS snapshots (every 15 minutes for 24 hours) then getting coarser the future out you go. Came in handy when the Dean got hit with ransomware.
or use Linux instead of Windows..
Once Linux is juicy enough it will be the new target.
People have been saying exactly this for 20 years now, and it still hasn't happened: organizations still entrust Windows with all their sensitive data and critical services. The fact is, if these victims had been on Linux instead of Windows, they would have been mostly immune to attacks, certainly to these ransomware attacks that only target Windows.
Remember, when you're hiking and you're attacked by a bear, you don't need to outrun the bear. You only need to outrun the slowest hiker in your group.
Remember, when you're hiking and you're attacked by a bear, you don't need to outrun the bear. You only need to outrun the slowest hiker in your group.
Linux is way harder to target, as any software vendor will readily tell you, so it would need to be significantly juicier for worthwhile return on investment.
>On Monday, Councilwoman KaShamba Miller-Anderson, the chairwoman of the board, asked Justin Williams, the interim information technology manager, for something seemingly simple. Could the elected officials’ new email addresses be posted online for the public to get in touch with them?
>Underscoring the enormity of the city’s troubles, Mr. Williams explained that the webmaster hoped to get to that soon.
>“He’s been working very feverishly to get that done,” Mr. Williams said.
...the webmaster is working feverishly to post a static piece of text to a website? I guess it really is hard to fire government workers.
>Underscoring the enormity of the city’s troubles, Mr. Williams explained that the webmaster hoped to get to that soon.
>“He’s been working very feverishly to get that done,” Mr. Williams said.
...the webmaster is working feverishly to post a static piece of text to a website? I guess it really is hard to fire government workers.
Most City websites are going to be some sort of CMS and it might not be as simple as putting a text file in the public folder and linking to it.
Add to that all sorts of other things that might be getting in the way and you can have someone working feverishly to get it done.
Add to that all sorts of other things that might be getting in the way and you can have someone working feverishly to get it done.
It's the government. He probably needs to find long lost keys to SSH in, file four forms (each in triplicate), and receive approval from each person to get it out.
It's municipal government. There almost certainly were no SSH keys.
The problem is that the credentials that the webmaster needs likely only exist in an Excel spreadsheet that was saved on the desktop of the "Administrator" user account on one of the machines that got hit.
The problem is that the credentials that the webmaster needs likely only exist in an Excel spreadsheet that was saved on the desktop of the "Administrator" user account on one of the machines that got hit.
That's easy to say, but if the deployment server is down you might not be able to do it easily, and if you try to do it manually you need credentials which were stored on in a text file on another server that's down... etc.
He is the interim guy.
I wanna know what happened to the DNS records.
I wanna know what happened to the DNS records.
maybe state governments should preempt this and make it illegal for municipalities or state agencies to pay ransoms, so they are less attractive targets
"We never pay any-one Dane-geld,
No matter how trifling the cost;
For the end of that game is oppression and shame,
And the nation that plays it is lost!"
But the problem is that those that don't pay hurt even more.
The US makes it illegal to pay kidnapper ransom and, as a result, US citizens have much worse outcomes (often murdered) when they are ransomed abroad.
The theory that it makes US citizens less attractive targets is confounded by the fact that some families/friends of the victims can and do pay anyway (illegally).
Planet money did a podcast on it.
https://www.npr.org/sections/money/2017/09/01/548032302/epis...
But the problem is that those that don't pay hurt even more.
The US makes it illegal to pay kidnapper ransom and, as a result, US citizens have much worse outcomes (often murdered) when they are ransomed abroad.
The theory that it makes US citizens less attractive targets is confounded by the fact that some families/friends of the victims can and do pay anyway (illegally).
Planet money did a podcast on it.
https://www.npr.org/sections/money/2017/09/01/548032302/epis...
Another interesting podcast on the topic of kidnapping: http://www.econtalk.org/anja-shortland-on-kidnap/
Goes into the details of kidnapping as a business venture from the kidnappers perspective, and how a price equilibrium is found between 'buyers' (ransom payers) and 'sellers' (kidnappers).
On interesting story was when the partner of a small business owner got kidnapped as punishment for failing to pay some protection money. When the business owner went to negotiate the ransom, the kidnappers had had an accountant already go through the businesses fiances so they knew exactly how much they could ask for, without it bankrupting the business owner (so that the owners company could keep thriving and thus could keep paying protection money).
Goes into the details of kidnapping as a business venture from the kidnappers perspective, and how a price equilibrium is found between 'buyers' (ransom payers) and 'sellers' (kidnappers).
On interesting story was when the partner of a small business owner got kidnapped as punishment for failing to pay some protection money. When the business owner went to negotiate the ransom, the kidnappers had had an accountant already go through the businesses fiances so they knew exactly how much they could ask for, without it bankrupting the business owner (so that the owners company could keep thriving and thus could keep paying protection money).
> The US makes it illegal to pay kidnapper ransom
What? I read the book "Never Split the Difference" written by a former FBI hostage negotiator, and it had stories of him helping with some kidnapping cases and offering a ransom. If I remember correctly his goal was not to avoid paying a ransom, but to make it as small as possible.
Good book by the way.
What? I read the book "Never Split the Difference" written by a former FBI hostage negotiator, and it had stories of him helping with some kidnapping cases and offering a ransom. If I remember correctly his goal was not to avoid paying a ransom, but to make it as small as possible.
Good book by the way.
> The theory that it makes US citizens less attractive targets is confounded by the fact that some families/friends of the victims can and do pay anyway (illegally).
but this wouldn't be true of governments, compared to private citizens, since government spending is a matter of public record.
but this wouldn't be true of governments, compared to private citizens, since government spending is a matter of public record.
Then there'll be workarounds such as those intermediary companies that claim to unlock it but in reality just pay off the cryptolocker guys and keep a part of the fee for themselves.
That said, this city could have used some negotiation help.
That said, this city could have used some negotiation help.
I would be more impressed if government at every level were held to higher standards than they impose on businesses. the hoops we jump through for SOX compliance which includes cybersecurity since 2017 [1] why aren't local, city, and state, officials, if not Federal, held accountable for the same? Similar to the story we had recently where state colleges across New York and elsewhere were not complying with the ADA and costing the tax payers millions.
[1] https://www.congress.gov/bill/114th-congress/house-bill/5069...
[1] https://www.congress.gov/bill/114th-congress/house-bill/5069...
The competing pressure is to drive down costs because no one wants to pay taxes.
Only if that comes with funding to defend against attacks and restore services. Otherwise, congratulations, you've turned off a city and it won't be working again for six months.
Note the specific detail here:
the City Council unanimously agreed to have its insurance carrier paySeems like a classic case of moral hazard here.
I'm surprised cybersecurity insurance doesn't mandate best-practice auditable backups as part of the process to grant a policy.
I'm surprised cybersecurity insurance doesn't mandate best-practice auditable backups as part of the process to grant a policy.
I mean, do health insurance companies mandate best health practices in order to get health insurance? Of course not, they just charge premiums commensurate with the risk they're taking on.
Interestingly, basing insurance on healthiness seems to be a new trend happening right now.
I'm a runner, and recently I've seen a lot of ads for a company called HealthIQ (I think) that offers cheap life insurance, but only for people who can run a 9 minute mile.
I think breaking into health insurance would be much harder because a) the administration is way more complicated and b) most people get health insurance through their employers, and normal employers won't be able to guarantee that every employee can pass a healthiness test, but I imagine they're working on getting around these problems right now.
I'm a runner, and recently I've seen a lot of ads for a company called HealthIQ (I think) that offers cheap life insurance, but only for people who can run a 9 minute mile.
I think breaking into health insurance would be much harder because a) the administration is way more complicated and b) most people get health insurance through their employers, and normal employers won't be able to guarantee that every employee can pass a healthiness test, but I imagine they're working on getting around these problems right now.
Depends on the carrier and plan.
Fully-integrated HMOs (think Kaiser) have extensive tracking and best practices that reduce future risk and liabilities: well mother / we'll baby care and training, vaccinations and nutrition, preventive chechups, monitoring of dangerous conditions, ob/gyn checkups, breast, colon & prostate exams, etc.
There's only so much that individual initiative can accomplish, but systemic measures really can move the needle.
Fully-integrated HMOs (think Kaiser) have extensive tracking and best practices that reduce future risk and liabilities: well mother / we'll baby care and training, vaccinations and nutrition, preventive chechups, monitoring of dangerous conditions, ob/gyn checkups, breast, colon & prostate exams, etc.
There's only so much that individual initiative can accomplish, but systemic measures really can move the needle.
Any insurer will have an initial state you have to meet to get insurancem and they audit that and sign you up. They should then follow up every listed time period (annually, weekly, daily, hourly etc) to make sure you running properly. That way, all you will use is the changes that occur in the listed period. This may mean hourly backups offsite - all you lose is one hour's worth.
Then, the insurance company and the insured company must rigorously keep to that routine.Experience shows that people get slack, they skip backups and other forms of lazy boys at work - they still cash their paychecks very regularly though - never skip that.
Tis is a typical civil service operation. highly paid union people locked into their jobs, their IT knowledge frozen in time years back and useless now. Older 'rusty' people can not be fired, so that current IT capable new people can be hired, and if you hire a consultant it will also be unionised (Yes, Toronto has a law that says that). so they stumble on until shit happens.
Quite a few escape this fate, get good IT people and keep it up, these idiots are a rare exception
Probably they haven't been hit hard so far...
Look, virtually nowhere in the public sector is security taken seriously. And nowhere in local government is security taken seriously. City governments might as well be pinatas... the way their budgets work, they'd never be able to replace large systems that were compromised. Without legislation banning them from paying, paying the ransom is likely really appealing to them. Security should be bumped up, but let's face it... that's not going to happen given how nobody who knows anything about tech would be caught dead working for local government. So many things have to change.
It really doesn't make sense to me that software isn't created at the government level for cities and states to use. This way it's easier to make sure everything is functioning properly and the cost is only paid once.
> It really doesn't make sense to me that software isn't created at the government level for cities and states to use.
What government level? The federal government?
> This way it's easier to make sure everything is functioning properly and the cost is only paid once.
The federal government is no paragon of software virtue, nor is it likely to produce software adapted all that well to all of the needs of various states and cities, so what you'd end up with is software less fit for purpose, not particularly free from vulnerability, and where all of the vulnerabilities expose every state and local government in the country rather than just one jurisdiction.
And that's still assuming good intent, but in many cases the state and federal government have adversarial relations on particular issues, which might lead to the federal government actively designing software in a way to frustrate the needs of particular states.
What government level? The federal government?
> This way it's easier to make sure everything is functioning properly and the cost is only paid once.
The federal government is no paragon of software virtue, nor is it likely to produce software adapted all that well to all of the needs of various states and cities, so what you'd end up with is software less fit for purpose, not particularly free from vulnerability, and where all of the vulnerabilities expose every state and local government in the country rather than just one jurisdiction.
And that's still assuming good intent, but in many cases the state and federal government have adversarial relations on particular issues, which might lead to the federal government actively designing software in a way to frustrate the needs of particular states.
Government jobs like creating software are required by law in most cases to be done as contracts that are put out for bid. Then the lowest cost or most corrupt choice wins. This entire system creates massive perverse incentives. Failing to complete the project on time is a business win. Making it hard to maintain is a business win (because maintenance will be separate ongoing contract and the company that authored it is thought to have some advantage on getting that contract). Making it hard to modify is a business win. Delivering as little actual functionality as possible is a business win (they might make new contracts or contract mods to add more functionality).
I worked on a system for over a decade that originally got awarded as a contract to build 3 systems. When all was said and done, 3 contracts had been awarded to get it to completion and 1 system (which required 9 months of full time work to get into a state to be used in production) was delivered. There are only 2 words for that in the contracting world: stellar success. That was a bigger win than they could ever have dreamed. Just drag it on, hiring the lowest paid new grads you can find to slap something together, stack the project with absurd layers of management, and collect the checks. Eventually, after years and years of this leeching, someone in the government will decide to make it an achievement in their career that they actually got the thing across the finish line. To do that they will sign off on the project and accept it no matter how short of contract requirements anything is. Their goal is to get the thing in the door and get credit for that, no one is going to blame them when its terrible. And the idea of actually punishing the companies that do this, penalizing them financially and legally for violating their contract when they don't deliver a working system on time? Forget it. Never happen. The companies will get the public fighting against 'big government' and crying crocodile tears for how harangued the billion-dollar megaconglomerates are with the RNC clanging finger cymbals while whirling around chanting 'jobs jobs jobs'.
I worked on a system for over a decade that originally got awarded as a contract to build 3 systems. When all was said and done, 3 contracts had been awarded to get it to completion and 1 system (which required 9 months of full time work to get into a state to be used in production) was delivered. There are only 2 words for that in the contracting world: stellar success. That was a bigger win than they could ever have dreamed. Just drag it on, hiring the lowest paid new grads you can find to slap something together, stack the project with absurd layers of management, and collect the checks. Eventually, after years and years of this leeching, someone in the government will decide to make it an achievement in their career that they actually got the thing across the finish line. To do that they will sign off on the project and accept it no matter how short of contract requirements anything is. Their goal is to get the thing in the door and get credit for that, no one is going to blame them when its terrible. And the idea of actually punishing the companies that do this, penalizing them financially and legally for violating their contract when they don't deliver a working system on time? Forget it. Never happen. The companies will get the public fighting against 'big government' and crying crocodile tears for how harangued the billion-dollar megaconglomerates are with the RNC clanging finger cymbals while whirling around chanting 'jobs jobs jobs'.
Do we know the problem is in the software? There's a lot of other things that go into securing a large system like this, training and testing staff to resist phishing attacks, apply security patches promptly, maintaining least privilege as requirements, hardware and staff change, etc. It seems to me that unless your software package encapsulates every use case and enforces the security protocols itself the only defense is an on-site security professional who is listened to.
Every state is going to have their own requirements. Some intentionally different, others unintentionally different. I can totally get on board with mandated open-source software for governments, but the reason the federal government doesn't just make the software is because most of the contracts involve support and training services. You can't rust government employees to do things for themselves now, come on. (=
Think of government as your grandparents. You can give them the best computer and software, but odds are they'll still call you in the middle of the work day to ask questions you don't really have time to answer. This is why Accenture and other big shops get big government contracts. At some point, it's easier to just send Geek Squad to your grandparent's house... knowing full well that they'll get upsold on crap they don't need, and charged more than they should... it's still easier than having to deal with teaching your grandparents not to write their passwords on PostIts they leave next to the computer.
Think of government as your grandparents. You can give them the best computer and software, but odds are they'll still call you in the middle of the work day to ask questions you don't really have time to answer. This is why Accenture and other big shops get big government contracts. At some point, it's easier to just send Geek Squad to your grandparent's house... knowing full well that they'll get upsold on crap they don't need, and charged more than they should... it's still easier than having to deal with teaching your grandparents not to write their passwords on PostIts they leave next to the computer.
[deleted]
With full SCADA integration for any type of system (water, sewer, electric)?
Might be some problems with the competitive bidding aspect of things, as well.
Might be some problems with the competitive bidding aspect of things, as well.
> Look, virtually nowhere in the public sector is security taken seriously.
The same is true of private sector. It's not like a bunch of businesses haven't had to pay ransom too you know.
> nobody who knows anything about tech would be caught dead working for local government
People like you are part of the problem.
The same is true of private sector. It's not like a bunch of businesses haven't had to pay ransom too you know.
> nobody who knows anything about tech would be caught dead working for local government
People like you are part of the problem.
> People like you are part of the problem.
The problem of what? We seem to be at an equilibrium, non-federal government offers neither the compensation nor the work environment to attract people who have better options. Maybe you can make an argument for civic duty, but that runs into the same problems as working for a company that's "changing the world" but not treating you very well. It just seems like a recipe for burnout, except the tech experience you accumulate probably won't be as good. No?
The problem of what? We seem to be at an equilibrium, non-federal government offers neither the compensation nor the work environment to attract people who have better options. Maybe you can make an argument for civic duty, but that runs into the same problems as working for a company that's "changing the world" but not treating you very well. It just seems like a recipe for burnout, except the tech experience you accumulate probably won't be as good. No?
"wouldn't be caught dead" is pretty strong language to be using when you're only suggesting that compensation isn't as good as it could be, which I'd have to disagree with anyway. Outside of SV startup funbux, the salaries, insurance, and retirement benefits I've seen for public sector postings have been at least competitive with other local business.
I think the real issue is the "work environment" aspect you're talking about. The public sector jobs I've worked have not been keen to chase resume-padding fads and have generally much preferred sensible and simple solutions to the complicated over-engineering that is commonly fetishized of our industry.
Anyway, my point about you being the problem is that you're unwilling to put up with whatever you see as the inadequacies of working for local government. You wouldn't be caught dead doing so, in your own words. You care far more about your own personal wealth enrichment than your community, so is it any wonder you end up with a government that isn't any good?
I think the real issue is the "work environment" aspect you're talking about. The public sector jobs I've worked have not been keen to chase resume-padding fads and have generally much preferred sensible and simple solutions to the complicated over-engineering that is commonly fetishized of our industry.
Anyway, my point about you being the problem is that you're unwilling to put up with whatever you see as the inadequacies of working for local government. You wouldn't be caught dead doing so, in your own words. You care far more about your own personal wealth enrichment than your community, so is it any wonder you end up with a government that isn't any good?
I didn't write the original comment, so I didn't say I "wouldn't be caught dead". But I'm skeptical that the choice is between "personal wealth enrichment" and "[my] community". It's far from obvious to me that working for local government means working for "[my] community", especially in my rather scandal-plagued city. Instead, working for my local government seems like it would force me to work with/for people with very different ethics who are nonetheless way better at playing the political power game, and who often don't care about tech to boot. That sounds bad.
> I didn't write the original comment, so I didn't say I "wouldn't be caught dead".
Fair enough, I wasn't paying attention.
> It's far from obvious to me that working for local government means working for "[my] community", especially in my rather scandal-plagued city.
How do you think it is that governments get this way? It's because the people who care about the community don't take part, so it is left to the people who want to exploit it.
Fair enough, I wasn't paying attention.
> It's far from obvious to me that working for local government means working for "[my] community", especially in my rather scandal-plagued city.
How do you think it is that governments get this way? It's because the people who care about the community don't take part, so it is left to the people who want to exploit it.
The laxness of infosec in government continues to astonish me. It's not like these types of attacks are new either. I can only assume that the people in charge of infosec in such situations are bureaucrats without much technical knowledge.
It's a failing of the American condition. The country was founded by radical conspiracy theorist farmers that didn't want to pay taxes. Distrusting government is in our national ethos. It pervades to this day in the form of governments generally being staffed with people too incompetent for private sectors. The pay sucks. It's hard to get raises. It's hard to do anything because Americans hate taxes; they'll help their neighbors, but they won't help those they can't see beyond their porch. There's little personal incentive to work in local/state governments. And that's partly how we end up with events like this.
> It's a failing of the American condition. The country was founded by radical conspiracy theorist farmers that didn't want to pay taxes. Distrusting government is in our national ethos.
Not true at all. Distrust of government is a relatively new phenomenon in American politics. It can be traced back to Reagan's infamous "The most terrifying words in the English language are: I'm from the government and I'm here to help" quote. This is because Reagan strictly believed in small government, and wanted to limit government interference in most things.
Before Reagan, Americans had no issues trusting government to solve big problems or accomplish major goals. See the Space Race, and Roosevelt's New Deal policies two decades before that. American people were largely optimistic about those endeavors because they trusted their government.
Not true at all. Distrust of government is a relatively new phenomenon in American politics. It can be traced back to Reagan's infamous "The most terrifying words in the English language are: I'm from the government and I'm here to help" quote. This is because Reagan strictly believed in small government, and wanted to limit government interference in most things.
Before Reagan, Americans had no issues trusting government to solve big problems or accomplish major goals. See the Space Race, and Roosevelt's New Deal policies two decades before that. American people were largely optimistic about those endeavors because they trusted their government.
The Vietnam War did a lot to destroy Americans' trust in their government.
One time I accidentally left in an HTML tag when adding a note on the Social Security website. That page got messed up and would no longer load properly. I just stopped using it since I didn't want to report it and get in trouble. It was shocking that they fail to do even the most basic input sanitization.
The decision makers are the city council and the mayor.
If none of these people have the knowledge to make an informed decision, they will defer to either internal IT staff (if that even exists), or their contracted MSP.
I seriously doubt there is much, if any, proactive coordination between the council and IT.
If none of these people have the knowledge to make an informed decision, they will defer to either internal IT staff (if that even exists), or their contracted MSP.
I seriously doubt there is much, if any, proactive coordination between the council and IT.
More likely bean-counters unwilling to pay for experienced IT staff.
I think it’s basically the same problem as small businesses, people have this psychological expectation that I’m too small to target. They don’t realize that they are just another IP address or email address in a sea of IP/email addresses that is being automatically targeted every hour of every day.
Business is similarly susceptible. Small business especially, but even medium/large often do shockingly poorly.
Great, now they have encouragement to do it again.
Will the attackers restore the attacked machines once the payment is made? They are just as likely to take the ransom and run.
If they want the next city to pay, they will restore the attacked machines. Gotta remember, this is a business for the attackers.
I think that time has passed. I've read about too many people who pay up but don't get their files back.
Similar for ransom for actual lives: http://www.econtalk.org/anja-shortland-on-kidnap/
I would very much like to hear from the insurance carrier here, and know what the post mortem and preventive countermeasures will be.
Update: The servicer appears to be Gallagher Basset based on the 2018-19 budget and legal cases cited online.
City records (CC agendas, minutes) are painful if not impossible to navigate.
Update: The servicer appears to be Gallagher Basset based on the 2018-19 budget and legal cases cited online.
City records (CC agendas, minutes) are painful if not impossible to navigate.
Curiously, no open IT / security positions listed:
https://rivierabch.applicantpro.com/jobs/
https://rivierabch.applicantpro.com/jobs/
The first thing I do when I read a news article that includes any company or organization saying that they are 'serious about security' is go to their website and check their job listings. Sometimes they have openings for security-related positions, but what is actually telling is their software openings. They NEVER so much as mention a single thing about security or knowing how to create secure applications. I've literally not seen a single exception, personally. Most companies do seem to have reached a level of 'caring about security' but it amounts to hiring some people to play Patch Patrol and nothing more. I guess that's better than nothing, but it won't actually help much.
I've noticed outside software or technical companies IT is basically 100% turn key with off the shelf mostly junkware (even 'enterprise') software being used. I attribute this to the mismatch between HR and the position being hired for and what higher education teaches w.r.t IT. Honestly what needs to happen is interviews need to be farmed out to places that understand the respective industry and not just certifications and higher education.
Let's look at this a bit differently - Ransomware is a type of malicious software designed to block access to a computer system or computer files until a sum of money is paid. Most ransomware variants encrypt the files on the affected computer, making them inaccessible, and demand a payment to restore access.
Ransomware is a type of malicious software designed to block access to a computer system or computer files until a sum of money is paid. Most ransomware variants encrypt the files on the affected computer, making them inaccessible, and demand a ransom payment to restore access.
Ransomware is rarely individually targeted, but rather a “shotgun” approach where the attackers (Clue I) acquire lists of emails or compromised websites and blast out ransomware.
Microsoft used a method to install software giving it superuser rights without a login. (Clue II) Most ransomware is based on this same install job. It is lightweight but identifiable.
Ransomware is a tripartite intruder and is based on what's already there on Windows (mscexe) in your compute and a substitution of legit program (outlook encrypt) Once the 3 parts are there your system is theirs and only a windows product key method "EFHST-G6ERT-VXWMT-FF8MB-MYERR" can free it - all thanks to Microsoft's product key methodology.
Oh and "backups" & PCmatic won't help and because Microsoft uses the same method to stop you from sharing software. You have seen the screen yourself => you have entered an invalid the product key!
Ransomware can be shipped with a NSA crack( EternalBlue ) forced onto the city of Baltimore (Clue III ) but the same code to create a superuser is open to the public is the end to all protection - because it hides using Microsoft's hidden directory method.
Well what to do now, pay the BTC? Yes and NO Yes buy BTC and NO this is where we create a pigeon drop for out NSA connected friends - we don't accept the face price and try to keep our BTC keys and Encrypt theirs.
For the FBI and NSA the profit from robbing Venezuela, Iran, Russia, Ukraine and Switzerland has been too great for them to stop. As witnessed with Venezuelan money gone and power outage.
That said, demand that Microsoft be held liable for product defects and to make all actions visible to the end user community ( no hidden files or directories ).
Ransomware is a type of malicious software designed to block access to a computer system or computer files until a sum of money is paid. Most ransomware variants encrypt the files on the affected computer, making them inaccessible, and demand a ransom payment to restore access.
Ransomware is rarely individually targeted, but rather a “shotgun” approach where the attackers (Clue I) acquire lists of emails or compromised websites and blast out ransomware.
Microsoft used a method to install software giving it superuser rights without a login. (Clue II) Most ransomware is based on this same install job. It is lightweight but identifiable.
Ransomware is a tripartite intruder and is based on what's already there on Windows (mscexe) in your compute and a substitution of legit program (outlook encrypt) Once the 3 parts are there your system is theirs and only a windows product key method "EFHST-G6ERT-VXWMT-FF8MB-MYERR" can free it - all thanks to Microsoft's product key methodology.
Oh and "backups" & PCmatic won't help and because Microsoft uses the same method to stop you from sharing software. You have seen the screen yourself => you have entered an invalid the product key!
Ransomware can be shipped with a NSA crack( EternalBlue ) forced onto the city of Baltimore (Clue III ) but the same code to create a superuser is open to the public is the end to all protection - because it hides using Microsoft's hidden directory method.
Well what to do now, pay the BTC? Yes and NO Yes buy BTC and NO this is where we create a pigeon drop for out NSA connected friends - we don't accept the face price and try to keep our BTC keys and Encrypt theirs.
For the FBI and NSA the profit from robbing Venezuela, Iran, Russia, Ukraine and Switzerland has been too great for them to stop. As witnessed with Venezuelan money gone and power outage.
That said, demand that Microsoft be held liable for product defects and to make all actions visible to the end user community ( no hidden files or directories ).
So the US is willing to pay ransoms - just not for people.
> So the US is willing to pay ransoms - just not for people
A city government in Florida is not the US federal government.
A city government in Florida is not the US federal government.
Well now the various groups beheading people on YouTube know where to ask.
Those people care about creating a spectacle, not getting paid.
for people's data?
It would probably be cheaper to resolve citizen's issues directly for all missing data not present in the backup (even if it's not, that was a crazy decision made by the city council)
It would probably be cheaper to resolve citizen's issues directly for all missing data not present in the backup (even if it's not, that was a crazy decision made by the city council)
I will continue to smugly assert that backup must include an offline component. Given that total data loss is a non-zero possibility (and, increasingly, more and more likely) the argument that having even a simple offline component (say, some encrypted USB disks for a small business, tape or such for a larger business) is too expensive or cumbersome doesn't make sense to me.
What evidence do they have that the hackers will actually send the keys? seems like a pretty big gamble on trusting proven criminals.
Ransomware relies on trust, because if you don't actually hold up your end and decrypt the data then no one will bother to pay you in the future. The entire "market" is best served by playing "fair", in so far as that can be applied here.
That doesn't mean some criminals won't just take the money, but it does mean that most of them wont and that the larger players have a vested interest in keeping that behavior to a minimum.
That doesn't mean some criminals won't just take the money, but it does mean that most of them wont and that the larger players have a vested interest in keeping that behavior to a minimum.
What reason so they have to not send the keys (apart from not having them)? If they send the keys, all the sooner the can hit them again.
Lady opens a random email (most likely in her junk folder) from someone she doesn't know and end's up costing the company hundreds of thousands of dollars? In 2019? Something is rotten in the state of Denmark.
[deleted]
Apparently some people do negotiate with terrorists.
Shouldn't the one responsible personally have to pay for it rather than the city and its taxpayers?
Do you have to pay the damages for every mistake you make at your job?
How on earth do you attribute responsibility here? Usually these things are an underfunded disaster waiting to happen, because the city can't find the money to upgrade from XP or whatever.
You mean the one who wrote the attacking code? Or the one who wrote the vulnerable code? Why do we even assume there is a "one" here?
Whoever made the decision not to take backups for example. The ones who will have to pay for their mistakes will be the taxpayers otherwise.
This is a public service, aren't the voters responsible? They could have voted in competent leaders.
This is it exactly. The voters are the ones who are ultimately responsible, and they'll be the ones to ultimately pay, just as it should be. They should be voting for competent leaders, and for sufficient taxes to pay decent salaries to attract good IT talent, but they don't, so this is what they get.
Every nation gets the government it deserves. - Joseph de Maistre
Every nation gets the government it deserves. - Joseph de Maistre
The voters are not one person. Sadly democracy ends up being the fascism of the many.
It sounds like you don't understand what "fascism" is, because this statement is plainly wrong.
The common statement is that "democracy is tyranny of the majority*, which is basically true IMO. Tyranny is not synonymous with fascism, though fascism can certainly be a form of tyranny.
Anyway, it doesn't matter if the voters aren't one person; they're a collective, and collectively they generally approve whatever government they're living under, or else they wouldn't have elected it, or allowed it to continue to rule them. If they elected it, they're getting what they voted for and what they deserve. If they didn't elect it, but allow it to rule them anyway, they're still getting what they deserve (though I'd make an exception for a small country being forcibly occupied by a much larger and more powerful country).
The common statement is that "democracy is tyranny of the majority*, which is basically true IMO. Tyranny is not synonymous with fascism, though fascism can certainly be a form of tyranny.
Anyway, it doesn't matter if the voters aren't one person; they're a collective, and collectively they generally approve whatever government they're living under, or else they wouldn't have elected it, or allowed it to continue to rule them. If they elected it, they're getting what they voted for and what they deserve. If they didn't elect it, but allow it to rule them anyway, they're still getting what they deserve (though I'd make an exception for a small country being forcibly occupied by a much larger and more powerful country).
No. $18 million was an estimate somebody gave once, who knows where it came from.
In fact, damages have not been repaired in Baltimore. 6 weeks later, most city services are still down. You can't pay a parking ticket or a water bill online. (You can send a check in; I am not sure where they record that you paid when they cash your check, and am not particularly confident they'll actually have a record I paid).
We in fact do not know how much money they've spent thus far, there have been no press briefings on this. Estimates of how much they will spend before it's over (will it ever be over?)... we all know how IT estimates work.
I think it will probably be quite a bit more than $18 million. And then there's estimating "damage to the economy." (There were two weeks when real estate transfers were frozen, because there was no way to check city liens. They can be done now, using a paper-based system that actually has those involved in the transaction sign an unusual contract agreeing to take on liability for unknown liens in unusual ways (I'm being vague cause I don't totally understand it), that some but not all title companies are willing to use).
The Baltimore ransomers only wanted ~$100K. If I were the mayor, yeah I'd pay it.
</Baltimore resident>