If anyone is looking for ideas for how to build tooling that fights flaky tests, I consolidated a number of lessons into a tool I open sourced a while ago.
It will do things like separate out different kinds of test failures (by error message and stacktrace) and then measure their individual rates of incidence.
You can also ask it to reproduce a specific failure in a tight loop and once it succeeds it will drop you into a debugger session so you can explore what's going on.
My understanding is that there is some evidence that Alzheimer's is transmissible (e.g. higher rate of incidence in caregivers and neurosurgeons), but more study is needed.
I think there is something romantic about the siren's song that is "we don't need object detection or velocity estimation if all objects self-report their location, velocity, and intent".
I hope you're right and that once we establish best case utility, as a community we refocus on handling component failures more gracefully.
Although it's unclear if the second and third order system effects in our financial system will ever get that sort of treatment. So let's hope driving gets a bit closer to flying (and farther from wall street) in terms of attitude towards safety.
I am not aware of any sensor system that involves meaningful cryptographic claims about sensor readings. I'd love to learn more about anything along those lines.
Especially the associated threat modelling and engineering principles!
Fair enough. Though my point was trying to say that protocols should assume a variety of component failure modes.
I'd rather just have to trust the safety standards of the manufacturer of my car (and to a lesser extent the cars I might directly collide with), not the safety standards of every vehicle within transmit distance.
Nothing about my comment advocates for perfectionism.
My point is that V2V systems and research papers I've seen just don't make meaningful claims about safety. They instead make claims about convenience and efficiency, which are not substitutes for safety.
We know to test vehicles for crash safety before putting them on the road.
While I believe there are certainly ways to make use of V2V communication that increase overall safety, I haven't seen anything remotely resembling a crash test for V2V systems.
Creating a system that relies on the correct behavior of all components is not a recipe for safety or reliability. This is especially true as the number of components increases (this happens when a car exchanges messages with all the cars around it).
We need V2V systems that rely only on correct behavior of any component and allow for malicious behavior of some components.
The idea that a protocol version mismatch from some rolling deploy can cause injuries in cars made by other manufacturers is the sort of thing that I haven't seen a single person point out. How would something like this even be caught and debugged?
Advocating for an ecosystem where these things are likely but neither addressed nor considered is just plain irresponsible.
Low Speed/Fault Tolerant CAN offers baud rates from 40 Kbit/s to 125 Kbits/sec. This standard allows CAN bus communication to continue in case of a wiring failure on the CAN bus lines. In low speed/fault tolerant CAN networks, each device has its own termination.
Car manufacturers are extremely careful about the failure modes of components and sensors they put in their vehicles. Consider the design of the CAN bus, which has explicit support for failed and misbehaving peers.
I have yet to see these sorts of considerations in any V2V communication system. The idea that my car might act on (or propagate) incorrect/sabotaged information that it receives from a "peer" is a terrifying form of fragility.
This sort of failure mode needs to be studied and addressed directly before any of these systems are deployed.
It should also be assumed that these new patterns of information propagation will create a huge financial incentive for people to sell after market modifications that exploit the trust models of these protocols.
Hi @danicgross, glad to see you're iterating in this space! :)
It feels like your comment about the network and community being the most valuable part of program is probably right. Given that, is it possible to be a Pioneer, but not accept the money?
Beyond providing security, reproducible builds also provide an important ingredient for caching build artifacts (and thus accelerating build times) across CI and developer machines. They also can form the basis of a much simpler deploy and update pipeline, where the version of source code deployed is no longer as important. Instead a simple (recursive) binary diff can identify which components of a system must be updated, and which have not changed since the last deploy. This means a simpler state machine with fewer edge cases that works more quickly and reliably than the alternative.
I'm very grateful for the work that this project has done and continues to do. Thank you!
Cool seed for a community! Seems to lack a place for discussion about submitted ideas, so I'll follow others' lead and discuss here.
The "industry specific deep learning" project is similar to something I'm working on right now. Though I'm not planning to charge for it.
To any folks here interested in this: Are you looking for a tool to get started in ML or for a resource to apply existing ML knowledge to a specific (possibly new) domain?
It might be an interesting starting point for you: https://observablehq.com/@ajbouh/editor