The essential point is that machine can teach us by giving solutions to problem (or games) that we can't solve. As times goes by, the role of machines as teachers will need to be explored, and probably if such a system provides good results it will be standardized. If there is a new way to learn we need to reflect about how to apply it in the best way and how to explore all the juice we can extract from mit, far from being a pointless action, we need to frame it with the information we are getting from the improvement in ML. We need to develop the concepts and ideas about it because the reward could be immense.
it is very interesting to measure at what rate AI improves. If the AI can improve a lot, a very interesting problem is to develop a scale or elo for measuring this progress. I am more interested in what lesson we can take home from this match, I don't have a wish here. We have many thinks to learn about this. In a situation of great uncertainty about the pace at with AI can improve its capabilities, measuring the velocity or rate of change can gives a hint about the future and applications of this new technology.
Having a machine that gives you feedback in the middle of the game perhaps could be used to describe what is the weak point of a decision tree, and in which situations the method is good. It could detect some situations in which decision trees are good, then use that decision tree to understand what is happening and with that new understanding devise a new method in the middle. We could train a decision tree using new very powerful information about the value of the game in the middle of the game, that is new and powerful.
Not so sure about the extra mile. It could happen that the AI couldn't be able to go beyond a certain level. That is this AI could be walled. We have a wonderful opportunity to see an example in which current AI is measured to be near but not far from best human competence.
This is very interesting. We have a way to test what he say. A match between the machine and this player with the game in the situation he describes. If he defeats the machine he proves that the machine game is nothing beyond our capabilities.
As a follow up to your idea, we should explore two paths: first create the most powerful AI, second create subsystems devised to be interpretable. The powerful method could be used to train the interpretable method, that is we need an interpreter to translate from machine AI to human AI, and interpretable systems provide a middle ground.
It seems that you are trying to create a new word that describe this new way of looking at the world. If human are able to decode the information contained in those unexpected moves, perhaps by creating a new heuristic, that could be viewed as a way of understanding the features the machine use internally, that is reading the machine brain. If human are able to decode that information creating new heuristics we could say that we are in a new state in IA in which learning among different intelligent species should be studied.
A great breakthrough could be to decode the information contained in the feature space of the nn or the rnn. A topological language in which shapes and chains are explained by analogies with real world situations and actions. Being able to share our vision and communicate our intentions (the weight given to the distinct features and the links among the several layers of the nn - the overall plan) should transform the concept of AI into one of CAI communication between intelligent agents to create a synergistic approach).
To solve the mystery of the significance of the p-value:
If mankind were to make one unique experiment a p-value would be a useful tool, but if many experiments are realized and people hide the results only showing those results that they want to show for self-promotion, then selecting a part of the real information (purposely or not) is a lie.
I can't understand what is the problem with p-values. You should never talk about the results, the essential point is that the method provides valid conclusions 95% and invalid conclusions 5% of the time. If your conclusion is wrong (you are in the 5% part) that is not a paradox. Also you should not try to prove things since your conclusion can be wrong, you should be glad to have a procedure that gives you useful information but is not infallible. By being humble you solve the problem.
I can't understand what is the problem with p-values. You should never talk about the results, the essential point is that the method provides valid conclusions 95% and invalid conclusions 5% of the time. If your conclusion is wrong (you are in the 5% part) that is not a paradox. Also you should not try to prove things since your conclusion can be wrong, you should be glad to have a procedure that gives you useful information but is not infallible. By being humble you solve the problem.