electroglyph

8 days agodiscuss

ONNX has AVX512 CPU kernels too, and openvino uses ONNX internally (and ONNX supports openvino backend)

electroglyph

8 days agodiscuss

it's a pain in the ass to do properly.

what we really need it something like auto-round for ONNX

electroglyph

8 days agodiscuss

ONNX is my first suggestion to people looking for speed gains on CPU

electroglyph

17 days agodiscuss

heh, i came here to say basically the same thing

electroglyph

19 days agodiscuss

existing embedding models like alibaba's modernbert tune or one of the jina v5s would probably map query to category automatically. (i.e. store embeddings of each category and calculate cosine sim for each incoming query vs. categories and pick the closest)

also, you could stick a classifier head on a BERT model as another option.

electroglyph

25 days agodiscuss

heh, when Raymond Chen dunks on the MSVC team =)

electroglyph

26 days agodiscuss

it's probably a lie

electroglyph

last monthdiscuss

deepseek 4 pro is insanely good for the price

electroglyph

last monthdiscuss

that link actually recommends not doing it from UEFI and doing it via software

electroglyph

last monthdiscuss

any divergence (even if the benchmark is better) from full precision is error

electroglyph

last monthdiscuss

this is better than TFA

electroglyph

last monthdiscuss

sometimes i wonder if the left hand knows what the right is doing. it looks like we arrested our own spy in this case: https://www.politico.com/news/2026/05/25/american-journalist...

electroglyph

2 months agodiscuss

absolutely. somebody online was wanting an LLM with Georgian language support, and that's exactly what i suggested: start digitizing Georgian text.

electroglyph

2 months agodiscuss

i'll upvote this each time it's submitted

electroglyph

2 months agodiscuss

you should be using dflash with that model, look it up

electroglyph

2 months agodiscuss

heretic maintainer: https://github.com/p-e-w/heretic

the fun bits are in another branch or PRs

electroglyph

2 months agodiscuss

p-e-w was just talking about this the other day in his Discord. seems doing the one neuron method is quite bad for KLD and that's why the newer techniques have stuck.

electroglyph

2 months agodiscuss

site has so little information there doesn't seem to be much to discuss

electroglyph

2 months agodiscuss

this looks awesome. i've been struggling with vector compression, and have been trying PCA + all sorts of rotations. looking forward to trying this out

electroglyph

2 months agodiscuss

nice writeup! looking forward to doing some more training as soon as i get some more data sorted. it'll be a custom arch, but i'll probably shoehorn it into unsloth for a speed boost.

electroglyph

Submissions

1.8-3.3x faster Embedding finetuning now in Unsloth

comments