HackerTrans
TopNewTrendsCommentsPastAskShowJobs

eugenhotaj

no profile record

Submissions

[untitled]

1 points·by eugenhotaj·2 ปีที่แล้ว·0 comments

Spelled out implementation of LLM parallelisms in pure C

github.com
3 points·by eugenhotaj·2 ปีที่แล้ว·0 comments

Zig GPT-2 inference engine

github.com
4 points·by eugenhotaj·3 ปีที่แล้ว·1 comments

comments

eugenhotaj
·2 ปีที่แล้ว·discuss
This is because everyone is training with synchronous sgd. all gpus need to synchronize on each gradient step so tail latency will kill you.
eugenhotaj
·3 ปีที่แล้ว·discuss
This is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2

Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.
eugenhotaj
·3 ปีที่แล้ว·discuss
In my experience processes like these rarely work out as intended and usually add layers of bureaucracy for marginal benefit. It’s usually senior engineers or middle managers looking for “org wide impact” so they can get promoted.
eugenhotaj
·3 ปีที่แล้ว·discuss
This post would sound so dumb if it didn’t come from the almighty pg.
eugenhotaj
·4 ปีที่แล้ว·discuss
The issue is not that ChatGPT will kill things off, the issue is that ChatGPT 4.0 will kill things off. If you don’t think that’s a real possibility, you’re sleeping.
eugenhotaj
·4 ปีที่แล้ว·discuss
Some of these are so odd. When Kylie Jenner launched her company she could have sold celery and still made $1B+. It wasn’t because of the effectiveness of her small team.
eugenhotaj
·4 ปีที่แล้ว·discuss
My experience is exactly the opposite in almost all cases. Most software is much more complicated than it needs to be. Reads like the author is just butthurt at feedback they received about their work.
eugenhotaj
·4 ปีที่แล้ว·discuss
I’ve likely had it all my life, but really started noticing about two years ago during the pandemic. Now I can’t unhear it. Went to a doctor a couple of times but nothing they tried really helped. Thankfully mine is not too bad and I’m mostly able to ignore it.
eugenhotaj
·4 ปีที่แล้ว·discuss
The network communication overhead would be way too high to make this useful. At least for current methods of training large models.