foldl2022·เดือนที่แล้ว·discussGemma-4 E2B/E4B models reuses K-V cache from other layers, which do things in a "transposed" way: not reuse Q/K/V matrices within a single layer, but reuse across different layers.
foldl2022·10 เดือนที่ผ่านมา·discussA nice side-effect of "variables at the top": you keep your functions short.
foldl2022·2 ปีที่แล้ว·discussOff the topic: I am also in favor of minimalist.In the case of AlphaGeometry, I made AlphaGeometryRE to get rid of tensorflow/jax/flax, and 100+ Python packages.