Snowflake's Arctic Long Sequence Training(techlife.blog)

https://techlife.blog/posts/snowflakes-arctic-long-sequence-training-how-to-train-llms-on-15-million-tokens-without-selling-a-kidney/