HackerTrans
トップ新着トレンドコメント過去質問紹介求人

haolu7

no profile record

投稿

8bit HippoAttention: Up to 3X Faster Compared to FlashAttentionV2

blog.hippoml.com
4 ポイント·投稿者 haolu7·2 年前·0 コメント

コメント

haolu7
·4 年前·議論
No, you don't need to train the model differently to use it with AITemplate. Here is an intro example to do inference with AITemplate with a very simple PyTorch model: https://facebookincubator.github.io/AITemplate/tutorial/how_.... For more advanced examples, check out https://github.com/facebookincubator/AITemplate/tree/main/ex...
haolu7
·4 年前·議論
AITemplate-PyTorch Stable Diffusion is the fastest stable diffusion inference solution by pushing image generation below one second on A100 (batch 1: 0.7s / 25 steps, 1.3s / 50 steps; batch 3: 1.6s / 25 steps, per image 0.55s; batch 16 7.9s / 25 steps, per image 0.49s) for the first time, 2.57X faster than Keras' XLA-based GPU compilation solution.

More benchmark numbers and repro at: https://github.com/facebookincubator/AITemplate/tree/main/ex...