CISA Releases Its Cybersecurity Strategic Plan(cisa.gov)
cisa.gov
CISA Releases Its Cybersecurity Strategic Plan
https://www.cisa.gov/news-events/alerts/2023/08/04/cisa-releases-its-cybersecurity-strategic-plan
56 comments
I don't think CISA has anything close to the powers required to compel that level of top down action. This strategic plan has clearly been crafted to be at least somewhat attainable given their current remit and capabilities.
You'd be surprised. Any organization regulated by federal government entities, such as banks via the OCC, FDIC, etc are being required to "meet" CISA guidelines. ESPECIALLY those with federal/military contracts.
CISA has a lot more sway than you'd think in how businesses operate from a security point of view.
Being this broad allows for some more latitude by the businesses / sectors following these guidelines. But they certainly could've been more thorough in their approach without much push back.
CISA has a lot more sway than you'd think in how businesses operate from a security point of view.
Being this broad allows for some more latitude by the businesses / sectors following these guidelines. But they certainly could've been more thorough in their approach without much push back.
As someone who is involved in compliance in these industries, I would be surprised to see CISA having anything close to the impact you described.
All I see are watered down checklists that can be verified by any human being who is semi-literate and may or may not have any relevance to security best practices. They probably were influenced on some level by CISA guidance if you're talking about .gov or commercial entities, but is nowhere near the level of impact you mentioned.
Do you have any examples of CISA guidelines having a meaningful impact on business operations?
All I see are watered down checklists that can be verified by any human being who is semi-literate and may or may not have any relevance to security best practices. They probably were influenced on some level by CISA guidance if you're talking about .gov or commercial entities, but is nowhere near the level of impact you mentioned.
Do you have any examples of CISA guidelines having a meaningful impact on business operations?
Indeed. They’re a security awareness arm of DHS. Frameworks, photo ops, pdf flyers. No teeth.
I mean, if you think of the federal government as a gigantic conglomerate enterprise network, and then read the "Measures of Effectiveness" in this plan as the current action items for the security practice in that enterprise, it's a pretty sane list, more forward thinking than e.g. most bank security teams.
> gigantic conglomerate enterprise network
It's a conglomerate that operates in every field of endeavor imaginable. One 'division' may be the world's largest 'conglomerate' itself: It employs over 1M people, has endless internal divisions, has global 24/7 operations, and an ~ $800B budget. With that budget, I would guess that their assets are worth more than Apple's market cap.
There's not a single CIO who can dicate 'we're blocking Facebook - get to work people!'.
It's a conglomerate that operates in every field of endeavor imaginable. One 'division' may be the world's largest 'conglomerate' itself: It employs over 1M people, has endless internal divisions, has global 24/7 operations, and an ~ $800B budget. With that budget, I would guess that their assets are worth more than Apple's market cap.
There's not a single CIO who can dicate 'we're blocking Facebook - get to work people!'.
I understand where you're coming from but "more forward thinking than a bank" should not be the aspiration for the organization primarily responsible for cybersecurity of the United States gov. This is not a good look for CISA.
You're going to have to be more specific than "this is not a good look". It looks pretty reasonable to me, given CISA's remit. Which part do you have a problem with, the limited role CISA has to motivate and guide security adoption inside government agencies, or the specific recommendations and metrics they're managing?
This "strategic plan" is devoid of any meaningful, measurable metric. The language throughout this document is carefully crafted to appear measurable at the surface, but meticulously written to be able to accomplish one thing after X number of years: stand infront of a podium and declare that the metric has been achieved.
Example: "Help organizations safely use AI to advance cybersecurity."
How do you measure this? What does this even mean? What does success look like if this is achieved?
Example: "Help organizations safely use AI to advance cybersecurity."
How do you measure this? What does this even mean? What does success look like if this is achieved?
I extracted all the metrics from the document and put them in a comment downthread. They look pretty reasonable to me. I'm sure every security team in America has some dumb metric about AI somewhere, but AI stuff is like 5% of the whole plan.
I think you're missing the point of my comment - the point is not that there is a meaningless metric about AI, the point is that it is -not a measurable metric- by any stretch of the imagination.
> Stock up on long lead time items, especially HV grid transformers.
Even better would be to onshore production of such items with bonuses for demonstrating ability to rapidly scale manufacturing.
Even better would be to onshore production of such items with bonuses for demonstrating ability to rapidly scale manufacturing.
>Beef up black start capability for energy grids, so that in the event of a major failure, power is 90% back up in an hour. Test this annually.
From the hypothetical scenarios detailed after the Texas winter shutdown a few years ago, this rule sounds unhinged from reality without mucho mucho dollars. Up to an including spare power plant(s) "just in case" a black start would be required.
I could believe it takes multiple hours just to fully start a single coal plant. Let alone align the frequencies across multiple stations in the grid.
From the hypothetical scenarios detailed after the Texas winter shutdown a few years ago, this rule sounds unhinged from reality without mucho mucho dollars. Up to an including spare power plant(s) "just in case" a black start would be required.
I could believe it takes multiple hours just to fully start a single coal plant. Let alone align the frequencies across multiple stations in the grid.
I imagine in most black start scenarios the coal plant would already be warm. If black start takes a day and the coal plant cannot operate without external power then maybe not so much. I don't think you need spare power plants so much as backup generators that can provide internal power to start a black start capable power plant up. If you do not have enough black start capable power plants that is not a good thing of course.
But it's not just that: to black start the grid you have to shed load - that is, you need to trip all the breakers along the grid, and then very carefully restore loads to avoid knocking yourself offline again.
You just plain aren't going to do that in an hour when you're offline because someone trashed your automated systems - resetting and restoring the grid would require people on site, in communication. That's not happening in an hour without staggering expense.
You just plain aren't going to do that in an hour when you're offline because someone trashed your automated systems - resetting and restoring the grid would require people on site, in communication. That's not happening in an hour without staggering expense.
I think all recent wars have demonstrated including the Ukraine war, that there are enough guided missiles to take out all power generating facilities, if you wanted to reap havoc on a country.
So at the top, all the leaders are just trolling the populace.
So at the top, all the leaders are just trolling the populace.
There's an industry group working on this.[1] They're worried.
One big concern that hasn't been well addressed is the interaction between natural gas infrastructure and electrical power generation. The 2021 Texas outage made this clear. "Five times more natural gas than wind power had been lost. When power was cut, it disabled some compressors that push gas through pipelines, knocking out further gas plants due to lack of supply."[2] If more of the Texas grid had gone down (it was close), it would have taken far longer to restart. Especially with frozen water in the pipelines. Nuclear, coal, hydro, solar, and wind plants have their own local fuel supply, but natural gas plants usually do not. The pipeline system and compressors need to be brought up first. This is especially a concern after the 2021 pipeline ransomware attack.[3]
PJM's internal 2019 study has concerns about this.[4]
[1] https://protectourpower.org
[2] https://en.wikipedia.org/wiki/2021_Texas_power_crisis
[3] https://en.wikipedia.org/wiki/Colonial_Pipeline_ransomware_a...
[4] https://www.pjm.com/-/media/committees-groups/committees/oc/...
One big concern that hasn't been well addressed is the interaction between natural gas infrastructure and electrical power generation. The 2021 Texas outage made this clear. "Five times more natural gas than wind power had been lost. When power was cut, it disabled some compressors that push gas through pipelines, knocking out further gas plants due to lack of supply."[2] If more of the Texas grid had gone down (it was close), it would have taken far longer to restart. Especially with frozen water in the pipelines. Nuclear, coal, hydro, solar, and wind plants have their own local fuel supply, but natural gas plants usually do not. The pipeline system and compressors need to be brought up first. This is especially a concern after the 2021 pipeline ransomware attack.[3]
PJM's internal 2019 study has concerns about this.[4]
[1] https://protectourpower.org
[2] https://en.wikipedia.org/wiki/2021_Texas_power_crisis
[3] https://en.wikipedia.org/wiki/Colonial_Pipeline_ransomware_a...
[4] https://www.pjm.com/-/media/committees-groups/committees/oc/...
> When power was cut, it disabled some compressors that push gas through pipelines
Every Factorio player knows you never power your generator pumps with the primary power source
Every Factorio player knows you never power your generator pumps with the primary power source
I agree. Most of the 30ish page document is completely open for interpretation, or misinterpretation. It would be much better to focus on the capabilities and outcomes.
> * Systems which handle other people's money must have continuous backups to write-once media and be able to 99% recover from a total loss of online data within 24 hours.
This is mostly in place, and has been for a while. Not so much “people’s” money, more for the banks money, which is where the systemic risk resides.
This is mostly in place, and has been for a while. Not so much “people’s” money, more for the banks money, which is where the systemic risk resides.
Can I vote for you? I would feel so much better if I heard anyone in government talk like this.
Oh, the fun idle musing of a non-security person. Its fun reading this list like watching Hollywood haxors attempt to hack something with some bizarre elaborate graphic user interface nobody would ever write.
> Identify risks that can kill people.
While we are imagining things people are more likely to die from excess sugar intake, alcohol, or driving on a freeway than in military combat or from some "hacker". Those are huge risks costing more lives than a holocaust and at much greater expense. We need to isolate these systems.
Talking about security with software developers who have never worked in security reminds of talking about performance. These things are like opinions. Everybody has opinions and most people become emotionally invested in their opinions as though their Monopoly money were something other than artificial. Usually though there is no actual investment of time, money, or professional experience in any of this. An outside observer might refer to this by colorful language.
If you want your opinions on security (or performance) to be more than bullshit... write some software to solve for some part of this.
> Identify risks that can kill people.
While we are imagining things people are more likely to die from excess sugar intake, alcohol, or driving on a freeway than in military combat or from some "hacker". Those are huge risks costing more lives than a holocaust and at much greater expense. We need to isolate these systems.
Talking about security with software developers who have never worked in security reminds of talking about performance. These things are like opinions. Everybody has opinions and most people become emotionally invested in their opinions as though their Monopoly money were something other than artificial. Usually though there is no actual investment of time, money, or professional experience in any of this. An outside observer might refer to this by colorful language.
If you want your opinions on security (or performance) to be more than bullshit... write some software to solve for some part of this.
nice list!!
Small addition:
Systems which handle other people's money... ==> Systems which track other people's money or possessions exceeding $x in value individually or $y in total...
(e.g. real estate, insurance policies, ...)
Small addition:
Systems which handle other people's money... ==> Systems which track other people's money or possessions exceeding $x in value individually or $y in total...
(e.g. real estate, insurance policies, ...)
And then your State-like adversary also gets hold of that information and you're now less secure before you had had access to it.
On a more general note, the same thing goes for a lot of centralisation efforts backed by technology ("Digitalisation", as it is called here in the EU), it's akin to building highways pointing straight from the border to your country's capital city and to your vital industries, all this while your very dangerous neighbour has a big tank army. Of course, tech-focused agencies like the CISA will never go against the process of digitalisation per se.
On a more general note, the same thing goes for a lot of centralisation efforts backed by technology ("Digitalisation", as it is called here in the EU), it's akin to building highways pointing straight from the border to your country's capital city and to your vital industries, all this while your very dangerous neighbour has a big tank army. Of course, tech-focused agencies like the CISA will never go against the process of digitalisation per se.
there’s very few industries in the world, if any, regulated this closely
I'm surprised there are so many negative comments on this release, which I suppose is timed for discussion at Blackhat/Defcon.
The report's identifies its audience as four groups of stakeholders: (1) federal civilian executive branch agencies (2) target rich, resource poor entities where federal assistance and support is most needed, including SLTT partners and our nation’s election infrastructure; (3) organizations that are uniquely critical to providing or sustaining National Critical Function (4) technology and cybersecurity companies with capability and visibility to drive security at scale
The overlap with the HN audience is probably primarily under the last category, where they have 5 objectives listed in the report (increasing threat modeling, secure software development frameworks, accurate CVE data, secure-by-design roadmaps, and publishing stats like MFA adoption and % of customers using unsupported product version). These all seem like good priorities for an agency like CISA and I've been impressed by their level of direct industry interaction even in our company's corner of the security (appsec) space.
The report's identifies its audience as four groups of stakeholders: (1) federal civilian executive branch agencies (2) target rich, resource poor entities where federal assistance and support is most needed, including SLTT partners and our nation’s election infrastructure; (3) organizations that are uniquely critical to providing or sustaining National Critical Function (4) technology and cybersecurity companies with capability and visibility to drive security at scale
The overlap with the HN audience is probably primarily under the last category, where they have 5 objectives listed in the report (increasing threat modeling, secure software development frameworks, accurate CVE data, secure-by-design roadmaps, and publishing stats like MFA adoption and % of customers using unsupported product version). These all seem like good priorities for an agency like CISA and I've been impressed by their level of direct industry interaction even in our company's corner of the security (appsec) space.
Looking at the details of the plan to secure America's IT infrastructure, it leads me to Secure Software Development Framework which then leads me to an Excel spreadsheet which then leads me to a tick the box exercise I can get from any generic consultant.
https://csrc.nist.gov/files/pubs/sp/800/218/final/docs/nist....
This is how big corp rubberstamps their security "review". As an American, I was hoping for the government to come up with a real solution. Like telling the big tech companies, that if America goes down the toilet, so do you. So stop with nonsensical security theater, and come up with real solutions. Like how to identify who is doing what. Real identity authentication and real logging. No more VPN/TOR/I can use any IP address I want then spoof a federal employee. No more I can arbitrarily change any setting/value because MSFT/UNIX doesn't believe in auditing.
https://csrc.nist.gov/files/pubs/sp/800/218/final/docs/nist....
This is how big corp rubberstamps their security "review". As an American, I was hoping for the government to come up with a real solution. Like telling the big tech companies, that if America goes down the toilet, so do you. So stop with nonsensical security theater, and come up with real solutions. Like how to identify who is doing what. Real identity authentication and real logging. No more VPN/TOR/I can use any IP address I want then spoof a federal employee. No more I can arbitrarily change any setting/value because MSFT/UNIX doesn't believe in auditing.
You're doing a lot of hand-waving. As someone who has managed remote access to... Internal networks, I'll say it isn't as easy as shoulder surfing at a coffee shop anymore to get into a secure network.
And if that isn't known, I do consulting!
And if that isn't known, I do consulting!
I’m a “target rich, resource poor” entity which is where federal assistance would most be needed, so I read the report eagerly. I didn’t like it and it doesn’t seem to contain any useful plans or roadmap.
Are you a federal agency? That's most of the target of CISA's strategic work.
I do work at the federal agency level, which is what I think you’re asking. I am clearly the intended audience of CISA’s strategic work, and that work is of very poor quality at the moment (as shown by the document we’re discussing) and does not serve my interests. CISA also declined to take my feedback in an unpaid advisory role, which is the first time in my life that this has happened. I’ve never met another organization that goes to as much lengths to avoid the possibility of hearing from its customers.
Where in the linked document do you see any part of their vision to listen to their targets and solve their problems? Their strategy does not include allowing any reporting of problems and threats, nor gathering any feedback about the security issues on the ground. In fact their document doesn’t even contain basic contact information. It is an opaque document discussing non-threats and ignoring gathering information about threats, understanding and responding to them. It is the worst strategic plan from any organization on any subject and fails to mention any mechanisms toward necessary outcomes.
https://www.cisa.gov/sites/default/files/2023-08/FY2024-2026...
Where in the linked document do you see any part of their vision to listen to their targets and solve their problems? Their strategy does not include allowing any reporting of problems and threats, nor gathering any feedback about the security issues on the ground. In fact their document doesn’t even contain basic contact information. It is an opaque document discussing non-threats and ignoring gathering information about threats, understanding and responding to them. It is the worst strategic plan from any organization on any subject and fails to mention any mechanisms toward necessary outcomes.
https://www.cisa.gov/sites/default/files/2023-08/FY2024-2026...
I suspect the issue here is that you are not, in fact, the intended audience of this work.
> Drive implementation of measurably effective cybersecurity investments
Metrics,metrics,metrics the usgov way.
Much if actually effective security is not effective until you get attacked. Especially when it comes to APTs. A huge furstration for me is how management takes the USGOV seriously. I mean, I have no idea what they do internally but their standards and intel are subpar and very slow to adapt to current threats.
Most of the things you have to do to defend against APTs are not on NIST CSF or any gov publication I have seen (and i have searched! Just to show value to management). It id like building an ark in the desert, it is far from measurably effective until there is a massive flood and possibly never again.
Just one word there makes a huge difference: measurably. If it was "provably" the red teaming can prove the effectivess if defenses but how managers will end up interpreting (not the good ones) this is you need to have KPIs to show for anything you do, so your ability to improve security posture is limited by your ability (give limited resources) to measure them.
Even then, it isn't straightforward to simulate zerodays being exploited. You can focus on known exploit primitives being used but it isn't uncommon for attacks to use novel techniques that evade your defenses.
I wish the usgov will either help or get out of the way. But i am clearly biased in the topic.
Metrics,metrics,metrics the usgov way.
Much if actually effective security is not effective until you get attacked. Especially when it comes to APTs. A huge furstration for me is how management takes the USGOV seriously. I mean, I have no idea what they do internally but their standards and intel are subpar and very slow to adapt to current threats.
Most of the things you have to do to defend against APTs are not on NIST CSF or any gov publication I have seen (and i have searched! Just to show value to management). It id like building an ark in the desert, it is far from measurably effective until there is a massive flood and possibly never again.
Just one word there makes a huge difference: measurably. If it was "provably" the red teaming can prove the effectivess if defenses but how managers will end up interpreting (not the good ones) this is you need to have KPIs to show for anything you do, so your ability to improve security posture is limited by your ability (give limited resources) to measure them.
Even then, it isn't straightforward to simulate zerodays being exploited. You can focus on known exploit primitives being used but it isn't uncommon for attacks to use novel techniques that evade your defenses.
I wish the usgov will either help or get out of the way. But i am clearly biased in the topic.
Here are all the metrics, metrics, metrics in this strategy document:
Reduction in our time-to-detect adversary activity affecting federal agencies and critical infrastructure partners.
Reduction in the time-to-remediation across each identified intrusion.*
Reduction in impact of incidents affecting CISA stakeholders
Number of malicious domain requests blocked.
Percentage increase in agencies that have fully automated key vulnerability and asset management processes and can report advanced measurements such as time-to-remediate, scan frequency, and scan quality.
Percentage decrease in prevalence of, and time-to-remediate, vulnerabilities in all participating organizations and percentage increase in visibility across all sectors.
Increase in vulnerabilities identified via agency Vulnerability Disclosure Platforms prior to adversary exploitation.
Increase in eligible organizations enrolled in DotGov.
Number of potential threats detected by the CyberSentry capability prior to identification by participating entity
Reduction in the time-to-remediate Known Exploited Vulnerabilities across critical infrastructure and government networks.
Increase in percentage of recommendations from CISA’s vulnerability and risk assessments adopted by assessed organizations.
Reduction in the number of vulnerabilities disclosed without appropriate coordination or provision of necessary mitigations.
Increase in the volume of unique, timely, and relevant information shared by industry or government partners through our persistent collaboration channels.
Increase in specific actions codified in cyber defense plans adopted by industry and government
Increase in post-incident after-action reports demonstrating that actions developed in cyber defense plans reduced negative outcomes.
Increase in the percentage of recommendations in CISA’s guidance and directives that are directly based upon specific data showing how adversaries successfully execute intrusions and the most effective mitigations to stop them.
Increase in the average number of Cybersecurity Performance Goals effectively adopted by organizations across each critical infrastructure sector.
Where possible, reduction in confirmed impactful incidents in organizations that have adopted a higher number of Cybersecurity Performance Goals.
Increase in the number of organizations outside of the FCEB that have adopted applicable requirements in CISA directives.
Increase in the percentage of FCEB agency adoption of CISA directive requirements.
Increase in the number of technology providers that have published detailed threat models, describing what the creators are trying to protect and from whom.
Increase in the number of technology providers that have regularly and publicly attested to implementation of specific controls in the Secure Software Development Framework (SSDF).
Increase in the number of technology providers that have published a commitment to ensure that product CVE entries are correct and complete.
Increase in the number of technology providers that have published a secure-bydesign roadmap, including how the provider is making changes to their software development processes, measuring defect rates, and setting goals for improvement, and transitioning to memory-safe programming languages.
Increase in the number of technology providers that regularly publish securityrelevant statistics and trends, such as MFA adoption, use of unsafe legacy protocols, and the percentage of customers using unsupported product versions.
Help organizations safely use AI to advance cybersecurity.
Protect AI systems from adversarial manipulation or abuse, building upon NIST’s AI Risk Management Framework.
Protect critical infrastructure organizations from adversarial AI systems.
Publish evaluation of potential cryptographic vulnerabilities in critical infrastructure, particularly focused on ICS/OT systems.
As verifiably quantum-safe products enter the market, increase in migration to quantum-safe cryptography by Systemically Important Entities and FCEB agencies.
Increase in the number of cybersecurity students trained in courses offered or funded by CISA.
Increase in the percentage of cybersecurity courses offered or funded by CISA that target underrepresented populations.
Increase in the number of organizations provided with training and resources to deliver cybersecurity training
As these things go, these are pretty straightforward, relatively forward thinking, and mostly pragmatic. For context: if the USG was (say) Walmart, CISA would be one small arm of its corporate security team.
Reduction in our time-to-detect adversary activity affecting federal agencies and critical infrastructure partners.
Reduction in the time-to-remediation across each identified intrusion.*
Reduction in impact of incidents affecting CISA stakeholders
Number of malicious domain requests blocked.
Percentage increase in agencies that have fully automated key vulnerability and asset management processes and can report advanced measurements such as time-to-remediate, scan frequency, and scan quality.
Percentage decrease in prevalence of, and time-to-remediate, vulnerabilities in all participating organizations and percentage increase in visibility across all sectors.
Increase in vulnerabilities identified via agency Vulnerability Disclosure Platforms prior to adversary exploitation.
Increase in eligible organizations enrolled in DotGov.
Number of potential threats detected by the CyberSentry capability prior to identification by participating entity
Reduction in the time-to-remediate Known Exploited Vulnerabilities across critical infrastructure and government networks.
Increase in percentage of recommendations from CISA’s vulnerability and risk assessments adopted by assessed organizations.
Reduction in the number of vulnerabilities disclosed without appropriate coordination or provision of necessary mitigations.
Increase in the volume of unique, timely, and relevant information shared by industry or government partners through our persistent collaboration channels.
Increase in specific actions codified in cyber defense plans adopted by industry and government
Increase in post-incident after-action reports demonstrating that actions developed in cyber defense plans reduced negative outcomes.
Increase in the percentage of recommendations in CISA’s guidance and directives that are directly based upon specific data showing how adversaries successfully execute intrusions and the most effective mitigations to stop them.
Increase in the average number of Cybersecurity Performance Goals effectively adopted by organizations across each critical infrastructure sector.
Where possible, reduction in confirmed impactful incidents in organizations that have adopted a higher number of Cybersecurity Performance Goals.
Increase in the number of organizations outside of the FCEB that have adopted applicable requirements in CISA directives.
Increase in the percentage of FCEB agency adoption of CISA directive requirements.
Increase in the number of technology providers that have published detailed threat models, describing what the creators are trying to protect and from whom.
Increase in the number of technology providers that have regularly and publicly attested to implementation of specific controls in the Secure Software Development Framework (SSDF).
Increase in the number of technology providers that have published a commitment to ensure that product CVE entries are correct and complete.
Increase in the number of technology providers that have published a secure-bydesign roadmap, including how the provider is making changes to their software development processes, measuring defect rates, and setting goals for improvement, and transitioning to memory-safe programming languages.
Increase in the number of technology providers that regularly publish securityrelevant statistics and trends, such as MFA adoption, use of unsafe legacy protocols, and the percentage of customers using unsupported product versions.
Help organizations safely use AI to advance cybersecurity.
Protect AI systems from adversarial manipulation or abuse, building upon NIST’s AI Risk Management Framework.
Protect critical infrastructure organizations from adversarial AI systems.
Publish evaluation of potential cryptographic vulnerabilities in critical infrastructure, particularly focused on ICS/OT systems.
As verifiably quantum-safe products enter the market, increase in migration to quantum-safe cryptography by Systemically Important Entities and FCEB agencies.
Increase in the number of cybersecurity students trained in courses offered or funded by CISA.
Increase in the percentage of cybersecurity courses offered or funded by CISA that target underrepresented populations.
Increase in the number of organizations provided with training and resources to deliver cybersecurity training
As these things go, these are pretty straightforward, relatively forward thinking, and mostly pragmatic. For context: if the USG was (say) Walmart, CISA would be one small arm of its corporate security team.
I am not suggesting the metrics are meaningless, I am suggesting everything else you need to do not covered under these or nist csf becomes lower priority or even seen as wasteful. In case you are not aware if it look up "mcnamara fallacy", a phenomena I have seen many times in corporate infosec, measurement itself is (wrongly) the goal instead of understanding and interpreting the measurement.
Check this:
> Number of malicious domain requests blocked.
Does this mean your domain reputation system sucks if that number goes down or does it mean you are cleaning up your assetts well? If you knew for certain they were malicious, and didn't block them, that is what it is meant to capture but in reality a downward linechart is all that is needed to fulfill the metric.
> Increase in the number of cybersecurity students trained in courses offered or funded by CISA.
> Increase in the percentage of cybersecurity courses offered or funded by CISA that target underrepresented populations.
The few "cyber security" grads I have seen start their career knew less than a helpdesk analyst. From well reputed colleges! Where is the qualitative metric?
> Reduction in our time-to-detect adversary activity affecting federal agencies and critical infrastructure partners.
Which adversaries? So, if defender remediates 10000 malware infection attempts within a day and you have 2 APTs with 90day+ dwell time, how does this work out? How does one's efforts to reduce APT dwell time fit in?
> Reduction in the time-to-remediation across each identified intrusion
This seems like a good idea on the surface but really, it should be time-to-containment. The IRL impact is, analysts will rush to remediate without properly analyzing and scoping the compromise. So long as the containment was effective and the eradication time is not unreasonable, who cares? The worst APTs are very hard to contain if you don't take your time to analyze their behavior, even containment is discourages in certain contexts to avoid tipping them off.
But stepping back a bit, I am with you that some of the metrics (especially around vulnmgmt) are solid. Lack of details, what isn't said and lack of emphasis on understanding are what make this harmful if taken as-is. For an org like the US government, I can see how this can be a good set of metrics for governmental departments and agencies, to enforce some reasonable level of security posture, expecting security teams to go beyond this and implement a much better set of goals and metrics according to their resources. But in the corporate world, this becomes the helm that drives the ship. You have one security org in the company and managers look bad when resources are spent doing things that don't help this metric and analysts "make" their numbers, unable to change deficiencies they see day to day that could help management understand the metrics (because that isn't the goal, the metric alone is the goal!).
Check this:
> Number of malicious domain requests blocked.
Does this mean your domain reputation system sucks if that number goes down or does it mean you are cleaning up your assetts well? If you knew for certain they were malicious, and didn't block them, that is what it is meant to capture but in reality a downward linechart is all that is needed to fulfill the metric.
> Increase in the number of cybersecurity students trained in courses offered or funded by CISA.
> Increase in the percentage of cybersecurity courses offered or funded by CISA that target underrepresented populations.
The few "cyber security" grads I have seen start their career knew less than a helpdesk analyst. From well reputed colleges! Where is the qualitative metric?
> Reduction in our time-to-detect adversary activity affecting federal agencies and critical infrastructure partners.
Which adversaries? So, if defender remediates 10000 malware infection attempts within a day and you have 2 APTs with 90day+ dwell time, how does this work out? How does one's efforts to reduce APT dwell time fit in?
> Reduction in the time-to-remediation across each identified intrusion
This seems like a good idea on the surface but really, it should be time-to-containment. The IRL impact is, analysts will rush to remediate without properly analyzing and scoping the compromise. So long as the containment was effective and the eradication time is not unreasonable, who cares? The worst APTs are very hard to contain if you don't take your time to analyze their behavior, even containment is discourages in certain contexts to avoid tipping them off.
But stepping back a bit, I am with you that some of the metrics (especially around vulnmgmt) are solid. Lack of details, what isn't said and lack of emphasis on understanding are what make this harmful if taken as-is. For an org like the US government, I can see how this can be a good set of metrics for governmental departments and agencies, to enforce some reasonable level of security posture, expecting security teams to go beyond this and implement a much better set of goals and metrics according to their resources. But in the corporate world, this becomes the helm that drives the ship. You have one security org in the company and managers look bad when resources are spent doing things that don't help this metric and analysts "make" their numbers, unable to change deficiencies they see day to day that could help management understand the metrics (because that isn't the goal, the metric alone is the goal!).
Does this mean your domain reputation system sucks if that number goes down or does it mean you are cleaning up your assetts well? If you knew for certain they were malicious, and didn't block them, that is what it is meant to capture but in reality a downward linechart is all that is needed to fulfill the metric.
Per the document, I think they're referring to a particular DNS service they themselves operate.
Per the document, I think they're referring to a particular DNS service they themselves operate.
I am guessing they have undisclosed private intel vendor telling them which domains are "malicious", or they are looking at compromises and auditing how malware domain is/isn't being blocked as part of their response.
Correct - that refers to the Protective DNS service, which is a centralized DNS service that is supposed to be used by all agencies.
Link to Actual Plan [.pdf]
https://www.cisa.gov/sites/default/files/2023-08/FY2024-2026...
https://www.cisa.gov/sites/default/files/2023-08/FY2024-2026...
How long until Jen Easterly is a partner at a16z? Seems like she's on a never-ending speaking tour, rather than leading an agency.
How cross sharing and integrated is CISA?
A lot of blame for 9/11 was security agencies being siloed.
A lot of blame for 9/11 was security agencies being siloed.
logicallee(2)
* Identify risks that can kill people. Strongly isolate systems where risk exists. Assume a hostile capability at the StuxNet level.
* Beef up black start capability for energy grids, so that in the event of a major failure, power is 90% back up in an hour. Test this annually.
* Stock up on long lead time items, especially HV grid transformers.
* Systems which handle other people's money must have continuous backups to write-once media and be able to 99% recover from a total loss of online data within 24 hours.
* Telecommunications systems must be capable of a cold restart from a known good state for 90% of users within one hour, 99% within 24 hours.