HackerTrans
TopNewTrendsCommentsPastAskShowJobs

kvark

no profile record

Submissions

Local ML inference benchmark: PyTorch vs. llama.cpp vs. the Rust ecosystem

kvark.github.io
2 points·by kvark·vor 3 Monaten·1 comments

Inferena: Local benchmark of PyTorch vs. Llama.cpp vs. Rust frameworks

inferena.tech
2 points·by kvark·vor 3 Monaten·0 comments

Inference Arena – new benchmark of local inference and training

kvark.github.io
4 points·by kvark·vor 3 Monaten·1 comments

Inferena – benchmarking inference of popular models on consumer hardware

inferena.tech
2 points·by kvark·vor 3 Monaten·0 comments

comments

kvark
·vor 29 Tagen·discuss
We had a process at one company where you had to create an issue before filing a PR. I found it most non-sensical and introducing friction for no good reason. Very surprised to see the author suggesting it in the article.

Review is indeed the main bottleneck now for open source, and we need to solve it. Introducing more friction is hardly helping.
kvark
·vor 3 Monaten·discuss
Author of the benchmark here, as well as Meganeura - one of the tested frameworks. Results are constantly being refined, gaps are filled, but it's hard to get more hardware coverage by myself. Would appreciate someone running it on Intel Arc GPUs, both discreet and integrated. Happy to answer questions about methodology!
kvark
·vor 3 Monaten·discuss
Thank you for support!

Part of the problem with timing variety is frameworks not always picking the right gpu/backend.

If you want to inspect or tweak the setup, be my guest at https://github.com/kvark/inferena
kvark
·vor 4 Monaten·discuss
Sadly the form doesn't even show up in Firefox. If they want to appeal to computer nerds, gotta anticipate not everyone will be on Blink engine.
kvark
·vor 5 Monaten·discuss
The main problem with Vulkan isn't the programming model or the lack of features. These are tackled by Khronos. The problem is with coverage and update distribution. It's all over the place! If you develop general purpose software (like Zed), you can't assume that even the basic things like dynamic rendering are supported uniformly. There are always weird systems with old drivers (looking at Ubuntu 22 LTS), hardware vendors abandoning and forcefully deprecating the working hardware, and of course driver bugs... So, by the time I'm going to be able to rely on the new shiny descriptor heap/buffer features, I'll have more gray hair and other things on the horizon.
kvark
·vor 9 Monaten·discuss
I've spent years playing with BWT and suffix sorting at school (some work can be found be names of archon and dark). It's a beautiful domain to learn!

Now I'm wondering: in the era of software 2.0, everything is figured out by AI. What are the chances AI would discover this algorithm at all?
kvark
·vor 5 Jahren·discuss
I'm sorry, the game on PC was a crappy bugfest. And as for being "impossible" on consoles... Isn't it effectively GTA in a different setting? GTA managed to run fine, and Cyberpunk could, but they weren't able to reach it.