Over the last year I’ve been building RacoGrad, a deep learning library written in Racket (a Lisp). It started as an experiment in understanding autodiff and neural nets from first principles, but it’s grown into something surprisingly usable.
Recently, with help of opencalw I added transformer support and finished a working GPT-2 implementation. The model loads pretrained GPT-2 weights and runs end to end inference entirely in Racket.
Recently, with help of opencalw I added transformer support and finished a working GPT-2 implementation. The model loads pretrained GPT-2 weights and runs end to end inference entirely in Racket.