The communications systems at the US Central Command headquarters(electrospaces.net)
electrospaces.net
The communications systems at the US Central Command headquarters
https://www.electrospaces.net/2021/04/the-communications-equipment-at-us.html
34 comments
Unfortunately this is common throughout the DoD, where the longstanding assumption is that it is possible to have a single enterprise solution for all X, and that doing this will be much more effective and efficient overall.
I don't think it's because the people involved are completely insane; they see that IT projects in DoD are expensive, and the expense is seemingly due to government/DoD overhead not intrinsically associated to solving the software development problem. So if you could increase the amount of software you can deliver with the same overhead, it should be more efficient, right?
Of course, it's nearly impossible to make this work, but the real solution (significantly reduce the overhead so that it's again efficient to build smaller solutions and iterate improvements from there) is impossible to imagine in DoD.
I don't think it's because the people involved are completely insane; they see that IT projects in DoD are expensive, and the expense is seemingly due to government/DoD overhead not intrinsically associated to solving the software development problem. So if you could increase the amount of software you can deliver with the same overhead, it should be more efficient, right?
Of course, it's nearly impossible to make this work, but the real solution (significantly reduce the overhead so that it's again efficient to build smaller solutions and iterate improvements from there) is impossible to imagine in DoD.
Surprisingly enough you see this in other security driven, slow moving (static) industries.
Just look at industrial controls. At the base you can say “air gap the system and it’s pretty secure.” Then you add on reliability, maintainability, optimization, etc. The people responsible for maintaining the control system are not IT, so they deploy a concept OT (operations technology). Now OT and IT have to play nice to deliver business solutions that make everyone feel warm and fuzzy about stability and uptime, but in reality the overhead and complication actually diminish returns usually.
The clash between the two often results in an endless cycle of trying to achieve security and efficiency, but since the two groups are beholden to different incentives they are regularly not aligned.
This often leads to reimplementing solutions regularly in an endless churn. I’ve worked a couple times on projects to replace solutions that have not been fully developed.
Just look at industrial controls. At the base you can say “air gap the system and it’s pretty secure.” Then you add on reliability, maintainability, optimization, etc. The people responsible for maintaining the control system are not IT, so they deploy a concept OT (operations technology). Now OT and IT have to play nice to deliver business solutions that make everyone feel warm and fuzzy about stability and uptime, but in reality the overhead and complication actually diminish returns usually.
The clash between the two often results in an endless cycle of trying to achieve security and efficiency, but since the two groups are beholden to different incentives they are regularly not aligned.
This often leads to reimplementing solutions regularly in an endless churn. I’ve worked a couple times on projects to replace solutions that have not been fully developed.
I’ve been in a leadership position for smaller consolidation projects / standardization projects outside of DoD, but pretty large.
The issue is all answers are correct. A lot of the differences between the “standard” and whatever is out there come down to politics - whatever is being done somewhere isn’t necessarily there by design, it’s just there.
Usually, 80% of problems are solved by the big consolidation play. The 20% is the problem, because those are either complex technical problems or intractable political problems. Nobody can get a budget justified to spend more money on 20% of the environment, and the the people running the consolidation project push until they declare victory and get promoted.
The issue is all answers are correct. A lot of the differences between the “standard” and whatever is out there come down to politics - whatever is being done somewhere isn’t necessarily there by design, it’s just there.
Usually, 80% of problems are solved by the big consolidation play. The 20% is the problem, because those are either complex technical problems or intractable political problems. Nobody can get a budget justified to spend more money on 20% of the environment, and the the people running the consolidation project push until they declare victory and get promoted.
As recently as 2018, it was actually working really well (for agencies it rolled out to). Without going into too much specifics, the technical challenges were more or less solved. There were some messy issues to work through still, but there are institutional and even practical barriers that needed to be overcome. When DNI Clapper left/after the administration change, ICITE presumably lost its champion(s) and new political appointees wanted to make their mark. Such is life inside the beltway, for better or worse.
Edit: slight edit - I will say the need for ICITE in its original conceived form has diminished somewhat as services have moved to shared cloud services. It’s not all doom and gloom.
Edit: slight edit - I will say the need for ICITE in its original conceived form has diminished somewhat as services have moved to shared cloud services. It’s not all doom and gloom.
It wasn’t thrown away. It’s going through a new iteration based on newly identified requirements. There’s a link in the article with more details.
https://www.c4isrnet.com/show-reporter/dodiis/2018/08/14/the...
https://www.c4isrnet.com/show-reporter/dodiis/2018/08/14/the...
"newly identified requirements" is the most predictable possible thing in DoD software writing though, as it is in other industries. This is why it is so important to reduce the amount of time it takes to go from identifying requirements to shipping software to product that starts to meet those requirements, as pointed out by the Defense Innovation Board's Software Acquisition Practices report: https://media.defense.gov/2019/May/01/2002126690/-1/-1/0/SWA...
To be fair, it was five years later. A lot can happen to organizational processes in that time. As organizational processes change (or new ones come about) so will the software that serves those processes.
> the most predictable possible thing in DoD software buying though,
FTFY
FTFY
And if it had worked as they thought it would, it would have been totally worth it.
It’s a different level of money altogether, but 300M is a small amount of 715 BN dollars in the DoD budget planned for FY 2022.
But when you’re operating at that scale, 300M is almost in the “let’s try it and see if it works”.
DNI had 60+ BN in appropriations for FY 2020.
It’s a different level of money altogether, but 300M is a small amount of 715 BN dollars in the DoD budget planned for FY 2022.
But when you’re operating at that scale, 300M is almost in the “let’s try it and see if it works”.
DNI had 60+ BN in appropriations for FY 2020.
I'm plenty familiar with the DoD's scale -- I worked at an IC contractor for a few years, and much of my current work is funded by the DoD.
The problem isn't the amount of money itself, it's that it can't work and never will: too many stakeholders ruin normal and simple software projects, and the DoD is that at its absolute worst.
Asking all of the IC agencies to settle on something common is like asking all of the construction workers at a site to get their tools from a common toolbox: everything is going to be in there, but it's going to be a jumbled mess of different priorities that everybody resents.
The problem isn't the amount of money itself, it's that it can't work and never will: too many stakeholders ruin normal and simple software projects, and the DoD is that at its absolute worst.
Asking all of the IC agencies to settle on something common is like asking all of the construction workers at a site to get their tools from a common toolbox: everything is going to be in there, but it's going to be a jumbled mess of different priorities that everybody resents.
Except it is working and another commenter pulled a link from the same document that describes the next iteration of the exact same project.
https://www.c4isrnet.com/show-reporter/dodiis/2018/08/14/the...
https://www.c4isrnet.com/show-reporter/dodiis/2018/08/14/the...
"Iteration" can mean just about anything in DoD-speak.
Barring further releases of information (or a HN reader being willing to do a little bit of leaking), I don't think anybody can confidently assert that the $300M spent on DTE will be meaningfully reused in the next "iteration."
Barring further releases of information (or a HN reader being willing to do a little bit of leaking), I don't think anybody can confidently assert that the $300M spent on DTE will be meaningfully reused in the next "iteration."
> a jumbled mess of different priorities that everybody resents
Sounds par for the course. Link to Pentagon Wars:
https://www.youtube.com/watch?v=aXQ2lO3ieBA
Sounds par for the course. Link to Pentagon Wars:
https://www.youtube.com/watch?v=aXQ2lO3ieBA
I hear you, and putting the scale there was something that I wanted to check and why I commented. At that scale it’s almost like one of Google’s moon shots…
But yeah - joint programs are fraught with problems and dynamics (tragedy of the commons is one major one).
But yeah - joint programs are fraught with problems and dynamics (tragedy of the commons is one major one).
> …so we paid BAE a third of a billion dollars to build a software platform that we then threw away in less than a decade, all because of an unrealistic original goal (“a single platform for the US IC”).
In their defense, that "unrealistic original goal" is a mistake all kinds of enterprises make, because at a high level there are obvious benefits and the problems are harder to see because they're at a lower level.
In their defense, that "unrealistic original goal" is a mistake all kinds of enterprises make, because at a high level there are obvious benefits and the problems are harder to see because they're at a lower level.
Thanks for the IRC bot!
I'm curious about the speed dial labels that were not called out in the article.
529-6100 529-6101 529-9001 important numbers but no names? dummy? but then why on the screen with the big wigs?
MA SECDEF - SecDef's mother?? MASECDEF QTRS - some temporary on-location housing for MA SECDEF, whoever that is?
https://1.bp.blogspot.com/-dxWBANA0MC0/YGRrCEZoKWI/AAAAAAAAF...
529-6100 529-6101 529-9001 important numbers but no names? dummy? but then why on the screen with the big wigs?
MA SECDEF - SecDef's mother?? MASECDEF QTRS - some temporary on-location housing for MA SECDEF, whoever that is?
https://1.bp.blogspot.com/-dxWBANA0MC0/YGRrCEZoKWI/AAAAAAAAF...
"MA SECDEF" is probably the SECDEF's Military Assistant:
> In the United States Department of Defense, a military assistant is a military officer serving as aide to very senior civilian (typically a presidential appointee in Office of the Secretary of Defense or in the service secretariats)[1]
I'd guess that "MASECDEF QTRS" is that officer's "home phone" (at their living quarters).
Based on the office phones I've used, I'm guessing that the buttons with the 7-digit phone numbers are just used to select phone lines that this phone can make/receive calls on, not speed-dial numbers. I'd also guess that they're on an internal military phone network that's not directly reachable from the regular phone system.
[1] https://en.wikipedia.org/wiki/Military_assistant
> In the United States Department of Defense, a military assistant is a military officer serving as aide to very senior civilian (typically a presidential appointee in Office of the Secretary of Defense or in the service secretariats)[1]
I'd guess that "MASECDEF QTRS" is that officer's "home phone" (at their living quarters).
Based on the office phones I've used, I'm guessing that the buttons with the 7-digit phone numbers are just used to select phone lines that this phone can make/receive calls on, not speed-dial numbers. I'd also guess that they're on an internal military phone network that's not directly reachable from the regular phone system.
[1] https://en.wikipedia.org/wiki/Military_assistant
You get to learn about the the Defense Switched Network[0] today. The US military does have an internal phone network, but the unclassified side can be reached from the normal telephone network just fine. One of the fun things is that if the network is busy, certain phones can do priority calls and kick lower priority calls off to get the bandwidth freed up. The red buttons to the right of the key pad allow that phone to do that.
[0]https://en.wikipedia.org/wiki/Defense_Switched_Network
[0]https://en.wikipedia.org/wiki/Defense_Switched_Network
> Based on the office phones I've used, I'm guessing that the buttons with the 7-digit phone numbers are just used to select phone lines that this phone can make/receive calls on, not speed-dial numbers. I'd also guess that they're on an internal military phone network that's not directly reachable from the regular phone system.
Yeah, listing both lines there also might allow this phone to "barge" in on an existing call on one of those lines. If one is in use, it might show up in a different color, allowing the user to select the unused one if they wanted a new line, or the used one if they wanted to join in on the call in process on that line.
Yeah, listing both lines there also might allow this phone to "barge" in on an existing call on one of those lines. If one is in use, it might show up in a different color, allowing the user to select the unused one if they wanted a new line, or the used one if they wanted to join in on the call in process on that line.
The plain numbers are likely the incoming lines/numbers. Like an indicator to show whether or not 'line 1' is ringing.
Ah, they're still struggling with multi-level security. Sigh.
Most of this stuff is modified commercial technology. Today, everything looks like a call center. The rooms full of custom made gear are decades into the past.
It's embarrassing that they have to have desk phones modified so that the hook switch has a hard cutoff on the microphone.
There's a mention of "KVM switches". One unusual feature of military command centers is that there's usually a way to look at other people's displays. That's done so you don't get everybody crowding behind the person who has the good stuff on their screen. This dates from 1960s NASA, where they just put all the display video on a cable TV network and everybody had a cable TV channel selector. The USAF was using the identical technology into at least the mid-1980s at some locations.
Most of this stuff is modified commercial technology. Today, everything looks like a call center. The rooms full of custom made gear are decades into the past.
It's embarrassing that they have to have desk phones modified so that the hook switch has a hard cutoff on the microphone.
There's a mention of "KVM switches". One unusual feature of military command centers is that there's usually a way to look at other people's displays. That's done so you don't get everybody crowding behind the person who has the good stuff on their screen. This dates from 1960s NASA, where they just put all the display video on a cable TV network and everybody had a cable TV channel selector. The USAF was using the identical technology into at least the mid-1980s at some locations.
>>>There's a mention of "KVM switches".
I'm not at a Combatant Command, but work in a smaller military operations center. KVMs are prolific because we frequently need to switch between computers on discrete networks with different classifications. For example: an unclassified network (with access to the World Wide Web, most common workstations for daily business), a US-only SECRET network, and 1-2 other US + $friendlycountry SECRET networks. You'll have 2 monitors and 2 keyboards at one workstation, each supporting 2 of the 4 networks. It's a mess.
I'm not at a Combatant Command, but work in a smaller military operations center. KVMs are prolific because we frequently need to switch between computers on discrete networks with different classifications. For example: an unclassified network (with access to the World Wide Web, most common workstations for daily business), a US-only SECRET network, and 1-2 other US + $friendlycountry SECRET networks. You'll have 2 monitors and 2 keyboards at one workstation, each supporting 2 of the 4 networks. It's a mess.
This is explained in the article in the same sentence that mentions KVM switches.
Sounds like a mess but I don't know how else you would do it without introducing huge security vulnerabilities.
Still? It’s an intentional form of MAC based upon a combination of labels and physical separation.
https://en.m.wikipedia.org/wiki/Mandatory_access_control
https://en.m.wikipedia.org/wiki/Mandatory_access_control
Is there a particular reason why all these images were uncensored in the original broadcast?
I’m curious if military systems use the same software that civilians use. By that I mean, using SSH, OpenSSL, OpenVPN/Wireguard, GPG, iptables, etc.
From Snowden leaks, it seems NSA actually used Gpg to secure their own communications.
Or they have their own encryption software, remote access applications etc?
From Snowden leaks, it seems NSA actually used Gpg to secure their own communications.
Or they have their own encryption software, remote access applications etc?
Usually with government systems, they use a lot of commercial off the shelf (COTS) either configured or modified (MOTS) to only use FIPS certified implementation/module.
You'll see technologies like S/MIME, IPSEC, and CAC Cards or other tokens/smartcards.
You'll see technologies like S/MIME, IPSEC, and CAC Cards or other tokens/smartcards.
In my experience from the vendor side of FIPS/CC certified software and hardware, yes and no. It was pretty common for us to fully support running the servers/devices we sold in FIPS/CC compliant mode. However most customers just used that as a RFC checkbox during procurement and never turned it on.
If you look at releases of OpenSSL that are FIPS certified for example, they're incredibly out of date. FIPS != Security, it's more about being able to audit the system. Most operators would file for some exemption and not actually run their devices in that mode, opting for actual security in the way of turning on newer encryption settings and the like.
If you look at releases of OpenSSL that are FIPS certified for example, they're incredibly out of date. FIPS != Security, it's more about being able to audit the system. Most operators would file for some exemption and not actually run their devices in that mode, opting for actual security in the way of turning on newer encryption settings and the like.
from I've been told this is fake and totally staged.
If you're referring to the walk-through by the news crew, there is a certain amount of sanitization and scripting that happens anytime a military organization hosts a PR event like this, but the comms equipment overview on this website is spot-on.
Source: spent way too much time in my military days in staff/HQ buildings like this.
Source: spent way too much time in my military days in staff/HQ buildings like this.
> This "Intelligence Community Desktop Environment" (IC DTE) was conceived in 2012 as a single, identical platform for the US Intelligence Community. As such it's the heart of a huge modernization project called Intelligence Community IT Enterprise (IC ITE), under which data will be stored and processed at the Commercial Cloud Services (C2S) managed by the CIA and the IC GovCloud managed by the NSA.
> The implementation of the DTE was managed by the Joint Program Management Office (JPMO) led by DIA and NGA, while the software system was built by BAE Systems under a $300 million contract for five years.
> However, in 2018, John Sherman, chief information officer of the Intelligence Community, said they had come to the realization that it no longer made sense to deliver a standard capability to every agency and user given the differing architectures, security requirements and mission needs.
…so we paid BAE a third of a billion dollars to build a software platform that we then threw away in less than a decade, all because of an unrealistic original goal (“a single platform for the US IC”).