I'm afraid that is also not correct. Dunning-Kruger is defined as people with low ability in a domain overestimating their own ability in that domain – it does not require someone to be really good in another domain.
It's definitely a dilemma. I guess the sweet spot would be security systems that are well understood in house but built on existing, well understood and studied standards and theory.
A starting point might be to use battle-tested open source systems but subject them to detailed in-house analysis and audit.
That sounds like it would be a big win for law enforcement. If you can force your opponent to avoid some modern ways to communicate, you can put a big dent in their efficiency.
It's possible that paranoia might lead to criminals avoiding even technology that they could be using safely, further slowing them down.
An old fashioned system also seems like it would require more people, opening up more opportunities for human intelligence operations targeting the network.
Even if the primary service itself was compromised, it would be possible to have an automated dead man's switch running on an alternative service that would alert the world that you were being held incommunicado.
Of course, failing to disclose the switch to the authorities and thus allowing it to trigger could be considered a violation of a court order and lead to punishment.