The Universal Weight Subspace Hypothesis(arxiv.org)
arxiv.org
The Universal Weight Subspace Hypothesis
https://arxiv.org/abs/2512.05117
1 comments
"We show that backpropagated neural networks trained on a variety of datasets - which could be
disjoint and unrelated - diverse hyper-parameter settings, initializations and regularization methods, often learn an architecture-specific, layer-wise similar, low-rank joint subspaces (we refer to this as the Universal Subspace). We provide the first large-scale empirical analysis - across a diverse set of models - that neural networks tend to converge to these joint subspaces, largely independent of their initialization or the specific data used for training."