PyTorch 2.2: FlashAttention-v2 integration, AOTInductor(pytorch.org)15 points·by egnehots·2 yıl önce·1 commentspytorch.orgPyTorch 2.2: FlashAttention-v2 integration, AOTInductorhttps://pytorch.org/blog/pytorch2-2/1 commentsPost comment[–]two_in_one·2 yıl öncereply> now supports FlashAttention-2, yielding around 2x speedups> torch.compile improvementsso far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does
> torch.compile improvements
so far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does