Open source for bot protection specifically would be difficult. If I as a bot developer can see the tests you run I can just modify my bot to pass them (either trivially or by brute force).
Wonder if it would be better to auto translate to broken rust, ie forcing the user to fix memory issues. I imagine that would lead to pretty big refactors in some cases though.
I think this is a facet of human memory - eg thinking childhood was better than it was because the bad/boring parts aren’t memorable. I also get this with anxious/stressful periods of time, which are overwhelmingly bad at the time but very quickly forgotten.