I'm the author. This is for readers interested in spectral methods, structured linear algebra, or interpretable tabular models.
The post looks at restricting learned symmetric matrix pencils, which serve as a nice intermediate between fully interpretable linear models and fully opaque neural networks, to tridiagonal form. That makes the eigensolve much cheaper, still leaves enough expressiveness to fit nontrivial functions, and lets me wire SciPy's `eigh_tridiagonal` into PyTorch autograd for training.
I also compare against the earlier dense version on California Housing: in the 45x45 example, the tridiagonal run takes about 3-4 minutes on CPU versus about 31 minutes for the earlier dense run on an L4 GPU, with similar error in that experiment. The post includes code, plots, and a Colab notebook.
Having developed in C++ a large portion of my career up until a 5+ years ago, it's nice to see C++ went a long way.
However one thing bothers me - error messages. Have the compilers evolved? Eventually, all this stuff translates to templates under the hood, and I remember always getting the compiler error far away from the place I've actually made the mistake.
The post looks at restricting learned symmetric matrix pencils, which serve as a nice intermediate between fully interpretable linear models and fully opaque neural networks, to tridiagonal form. That makes the eigensolve much cheaper, still leaves enough expressiveness to fit nontrivial functions, and lets me wire SciPy's `eigh_tridiagonal` into PyTorch autograd for training.
I also compare against the earlier dense version on California Housing: in the 45x45 example, the tridiagonal run takes about 3-4 minutes on CPU versus about 31 minutes for the earlier dense run on an L4 GPU, with similar error in that experiment. The post includes code, plots, and a Colab notebook.