HackerTrans
TopNewTrendsCommentsPastAskShowJobs

haolu7

no profile record

Submissions

8bit HippoAttention: Up to 3X Faster Compared to FlashAttentionV2

blog.hippoml.com
4 points·by haolu7·2 yıl önce·0 comments

comments

haolu7
·4 yıl önce·discuss
No, you don't need to train the model differently to use it with AITemplate. Here is an intro example to do inference with AITemplate with a very simple PyTorch model: https://facebookincubator.github.io/AITemplate/tutorial/how_.... For more advanced examples, check out https://github.com/facebookincubator/AITemplate/tree/main/ex...
haolu7
·4 yıl önce·discuss
AITemplate-PyTorch Stable Diffusion is the fastest stable diffusion inference solution by pushing image generation below one second on A100 (batch 1: 0.7s / 25 steps, 1.3s / 50 steps; batch 3: 1.6s / 25 steps, per image 0.55s; batch 16 7.9s / 25 steps, per image 0.49s) for the first time, 2.57X faster than Keras' XLA-based GPU compilation solution.

More benchmark numbers and repro at: https://github.com/facebookincubator/AITemplate/tree/main/ex...