HackerTrans
トップ新着トレンドコメント過去質問紹介求人

mesuvash

no profile record

投稿

[untitled]

1 ポイント·投稿者 mesuvash·3 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·3 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·4 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·4 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·4 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·4 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·4 か月前·0 コメント

[untitled]

1 ポイント·投稿者 mesuvash·4 か月前·0 コメント

DeepSeek's Dualpath Paper explained with animations

mesuvash.github.io
2 ポイント·投稿者 mesuvash·4 か月前·0 コメント

Intuitive Intro to Reinforcement Learning for LLMs

mesuvash.github.io
3 ポイント·投稿者 mesuvash·5 か月前·0 コメント

An Intuitive Introduction to PPO and GRPO

mesuvash.github.io
5 ポイント·投稿者 mesuvash·5 か月前·2 コメント

コメント

mesuvash
·4 か月前·議論
IIUC, The paper's notation S^(d-1) means the unit sphere in R^d (e.g., the familiar unit circle is S^1 living in R^2). So, i think, x in the algorithm is already a unit vector.

Reference: Section 2:Preliminaries ... We use the notation S^d−1 to denote the hypersphere in R^d of radius 1.

Section 3.1 Let x ∈ S^d−1 be a (worst-case) vector on the unit sphere in dimension d.
mesuvash
·4 か月前·議論
Fair point. I've updated the animation to address this. The grid now uses the correct non-uniform centroids (optimal for the arcsine distribution in 2D), so you'll see grid lines cluster near the edges where unit-circle coordinates actually concentrate, rather than being evenly spaced. The spacing does change with bit depth.

On the second quantization step: the paper's inner-product variant uses (b-1) bits for the MSE quantizer shown here, then applies a 1-bit QJL (Quantized Johnson-Lindenstrauss) encoding of the residual to make dot-product estimates unbiased. I chose to omit QJL from the animation to keep it digestible as a visual, but I've added a note calling this out explicitly.
mesuvash
·4 か月前·議論
Yes, this is important in high dimension. But sadly, very hard to visualize. In 2d it looks like unnecessary.
mesuvash
·4 か月前·議論
That's actually correct and intentional. TurboQuant applies the same rotation matrix to every vector. The key insight is that any unit vector, when multiplied by a random orthogonal matrix, produces coordinates with a known distribution (Beta/arcsine in 2D, near-Gaussian in high-d). The randomness is in the matrix itself (generated once from a seed), not per-vector. Since the distribution is the same regardless of the input vector, a single precomputed quantization grid works for everything. I've updated the description to make this clearer.
mesuvash
·4 か月前·議論
Yes. Great catch. I simplified the grid just for visualization purpose.

I've updated the visualization. The grid is actually not uniformly spaced. Each coordinate is quantized independently using optimal centroids for the known coordinate distribution. In 2D, unit-circle coordinates follow the arcsine distribution (concentrating near ±1), so the centroids cluster at the edges, not the center.
mesuvash
·4 か月前·議論
Author here. Sorry still working on refining the post. Will share once the post is ready.
mesuvash
·4 か月前·議論
TurboQuant explained with an easy to understand (no-math) animation https://mesuvash.github.io/blog/2026/turboquant-interactive/
mesuvash
·4 か月前·議論
I am glad you liked it :) You might like this https://mesuvash.github.io/blog/2026/rl_for_llm/ as well :)