Deep distilling: Automated discovery of algorithms from data(github.com)
github.com
Deep distilling: Automated discovery of algorithms from data
https://github.com/pauljblazek/deepdistilling
4 comments
Code...Matters...
You can tell everyone "who is anyone" is in hysterics when this gets virtually no attention at Salamandar's old stomping ground here at ycombinator.
You can tell everyone "who is anyone" is in hysterics when this gets virtually no attention at Salamandar's old stomping ground here at ycombinator.
The paper is behind a paywall: https://www.nature.com/articles/s43588-024-00593-9
The repo links to the authors' preprint from 11/2021: https://arxiv.org/abs/2111.08275
The repo links to the authors' preprint from 11/2021: https://arxiv.org/abs/2111.08275
The repo also links to a non-paywalled version of the paper:
https://rdcu.be/dy2Go
https://rdcu.be/dy2Go
This paper makes me think of the underlying premise of perceptrons -- the universal approximation theorem -- which states that a neural network of simple perceptrons can be combined to approximate most any mathematical equation with reasonable accuracy. At least, I hope that's a reasonable approximation of what it says. :)
Even though real-life biological neurons are much more complex (and some can perform complex operations like XOR operations in a single node), a regular perception cannot produce XOR by itself.
But combine a bunch of perceptrons together across a few layers, and you can not only perform XOR with a neural network, but pretty much any mathematical equation.
This paper makes me think that they're attempting to go the reverse direction with it -- after training a neural network to model a quadratic formula or something -- can we then go backwards and approximate the neural network's behavior with a discrete math equation?
This is a fascinating idea, and if it works, would send us a LONG way towards building interpretable AI!