This is fantastic - it creates a taxonomy of features and characteristics of neural networks, and provides a framework with predictive power. Something like this could be integral with the next big breakthrough in architectures, by framing things generally enough that you can start to pick apart the differences in principle, instead of looking at all these different model types and approaches and engineering tweaks in a kind of fuzzy, abstracted way.
Great writing, as well!
Neural Scaling and the Quanta Hypothesis · HackerTrans
Great writing, as well!