You're absolutely right, and I share the frustration.
I'm thinking a possible solution to this signal-to-noise problem is to embrace the longitudinal view: instead of comparing each scan with the normal across the population compare only against past self, unless there's a risk factor that warrants it.
This way we could presumably make use of plentiful scan data and mostly look at the stuff that evolves in suspicious ways, not what looks suspicious.
Don't "let it" generate tests. Be intentional. Define them in a way that's slightly oblique to how the production code approaches the problem, so the seams don't match. Heck, that's why it's good to write them before even thinking about the prod side.
- I'm not convinced the graph is necessarily cyclic. Often two codependents are actually dependent on some common bits and otherwise independent.
- this is essentially deterministic propagation of configuration (think dhall, jsonnet, etc) plus reconciliation loops for external state, terraform style — not dissimilar to how the rest of CI/CD should operate, in fact my view is this is an extension of CI/CD practices up the value stream.
I'm definitely strive for something like this when possible.
Although it isn't yet clear how much the brakes did actually brake, it is known they would never be enough.
So the cable was a critical component and initial findings suggest it wasn't being verified as rigourously, thoroughly and often as it perhaps should have.
[ my public key: https://keybase.io/vlfig; my proof: https://keybase.io/vlfig/sigs/m5-4eTUpqZsAvvl2vInWmcBN29lCs_WSgL94CZuS3jo ]