Show HN: GPT-2 inference in pure C#, 0 bytes allocated per token(github.com)

1 pointsby dev-on-bike2 months ago1 comments

Show HN: GPT-2 inference in pure C#, 0 bytes allocated per token

https://github.com/DevOnBike/Overfit

2 comments

For me, the interesting part here isn't GPT-2, it's the memory discipline. I feel like most inference runtimes slowly leak allocations everywhere as features pile up.

grelikt2 months ago