The Unreasonable Effectiveness of Recurrent Neural Networks (2015)(karpathy.github.io)

2 pointsby Kotlopouil y a 4 jours1 comments

karpathy.github.io

The Unreasonable Effectiveness of Recurrent Neural Networks (2015)

http://karpathy.github.io/2015/05/21/rnn-effectiveness/

1 comments

Kotlopouil y a 4 jours

I thought this is a cool premonition of the eventual success of language models. One can nicely see a microcosm of everything with generative AI (text generated in a domain you don't understand looks fine at first glance), already a mention of attention models, and there's even a reference to the model "hallucinating" some URL.

It's also all charmingly tiny. I was surprised at the quality of the LaTeX generated from a network trained on a single book -- it looks like a proof, has sequential lemmas, closes all brackets...

Also interesting to read the original discussion with a decade of hindsight: https://news.ycombinator.com/item?id=9584325