Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels(nanduruganesh.github.io)

39 pointsby rawshil y a 20 heures5 comments

nanduruganesh.github.io

Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels

https://nanduruganesh.github.io/flash-msa/

5 comments

kamranjonil y a 11 heures

I’ve actually been really interested in Minimax M3 - seems like it flew under the radar but size wise might actually be runnable for local inference with a footprint somewhere between Deepseek V4 flash and pro.

Has anyone used the new Minimax M3 model? I’m curious how it compares with Deepseek V4 and GLM 5.2 and other larger open weights models.

throwa356262il y a 8 heures

I have used it a little bit (0.5B tokens) for agentic tasks and coding. It is pretty nice and a serious step up from M2.7. I prefer it to GLM 5.2 for simpler tasks because of the cheaper token plan.

[Update: their cheapest token plan has been removed, I guess its back to GLM now]

Right now M3 is not far behind DS4, but I belive DS4 will improve much more with each round of training. It simply has a bigger brain, it just needs to fill it with more information.

throwa356262il y a 8 heures

Speaking of minimax, has anyone used their video/music models? The demo page looks cool but I never heard of anyone actually using them in anything.

villgaxil y a 14 heures

World’s first?

Such lazy, much farming

https://github.com/fla-org/native-sparse-attention?utm_sourc...

kamranjonil y a 12 heures

The Minimax paper was published in June 2026 coinciding with the Minimax M3 release - I’m not sure how the repo you posted here could have been an implementation of Minimax sparse attention when it was updated over a year ago?