It ultimately depends on how much of the car's action is driven by DNNs, a miss-classification in the CNN can easily lead to a car crashing on a highway and causing a pileup. DNNs are so complex, you can't really write unit tests like you typically do for deterministic code.
That's why some of the big banks flat out refuse to implement any form of deep learning for risk analytics. They're much more reliant on simpler ML models like random forests and logistic regression that are easier to analyse and diagnose by model governance teams.
Well considering that most of the autopilot software is driven by black box deep neural networks, I don't think hiring someone with a strong coding background is going to make that much of a difference when it comes to safety.
No we don't, I am not anti-progress but my personal opinion is that increased automation will lead to further inequality and potentially economic contraction. Thing is, we are not born equal, different people have different skills, we can't be expecting everyone to become an engineer or a scientist. Blue collar workers losing their jobs will not lead to interesting times...
I have been extensively using the dateframe/sql API and I just love it. Most of the issues I have had stemmed from the cluster / Spark configuration and not the API itself. Using SQL is so much more intuitive them using multiple joins, selects, filter etc on an rdd.