That's a nice article, thanks! I learned something new from it (somehow I didn't know that typical finite difference error plot is linearly V-shaped).
tang library looks cool, and I wonder whether it will become a modern full-fledged math library independent of ancient BLAS and solver libs... I wish it would, but that would require huge effort.
A couple of nitpicks for the article:
> The pathological function barely hits 10⁻⁵.
Huh? The pathological function's error plot goes below 10⁻⁸.
> What does that buy you? Watch two gradient descent particles race down Himmelblau's function
The default value in the visualisation for h is 0.5. Of course it performs poorly! Switch to h=0.1 and the difference is gone. I wonder if there exists a more significant demonstration
tang library looks cool, and I wonder whether it will become a modern full-fledged math library independent of ancient BLAS and solver libs... I wish it would, but that would require huge effort.
A couple of nitpicks for the article:
> The pathological function barely hits 10⁻⁵.
Huh? The pathological function's error plot goes below 10⁻⁸.
> What does that buy you? Watch two gradient descent particles race down Himmelblau's function
The default value in the visualisation for h is 0.5. Of course it performs poorly! Switch to h=0.1 and the difference is gone. I wonder if there exists a more significant demonstration