Grokking(en.wikipedia.org)
en.wikipedia.org
Grokking
https://en.wikipedia.org/wiki/Grokking_(machine_learning)
1 comments
Feels like a rebranding of "eventually the inductive bias wins": train long enough and you can jump from memorization to a simpler rule that generalizes. Interesting, but I’m not sure how often this shows up outside toy algorithmic tasks.