Thanks for posting. This seems interesting. It takes a while to get to making good points but they provide a good overview of how the various parts of safety, trust, ethics are explored by various explanatory methods like visualization, model distilation, and "intrinsic" methods. It seems like they're not actually addressing the issue of explainability but more the tools that are available for trying to debug extremely large programs composed of matrix multiplications and function applications. It seems like "explainable" in this context really means "debuggable" and "debugging tools".
Because fundamentally neural networks really have a debuggability problem. It's impossible to say if the program/code is actually correct and I'm not sure how visualization is actually going to solve the problem of correctness. If someone explained something to me I'd want to know why it actually addresses whatever problem they claim it addresses and their reasons of appealing to distilled models would not convince me because as long as we are looking at a compressed version of the program then why would we conclude that the larger program is actually correct and never misclassifies a pigeon as a stop sign? So if I can't be sure that a pigeon will never be classified as a stop sign then what has been actually explained and of what value is it?
Because fundamentally neural networks really have a debuggability problem. It's impossible to say if the program/code is actually correct and I'm not sure how visualization is actually going to solve the problem of correctness. If someone explained something to me I'd want to know why it actually addresses whatever problem they claim it addresses and their reasons of appealing to distilled models would not convince me because as long as we are looking at a compressed version of the program then why would we conclude that the larger program is actually correct and never misclassifies a pigeon as a stop sign? So if I can't be sure that a pigeon will never be classified as a stop sign then what has been actually explained and of what value is it?