What we talk about when we talk about system designmaheshba.bitbucket.io259 points·by scv119·vor 3 Jahren·35 comments
scv119·vor 3 Jahren·discussThe tweet is referring to a paper that fine tunes Chinese dataset on english base model. I'm not surprised with LoRA's poor result in this setup.
scv119·vor 3 Jahren·discussI believe you can slightly change the flash attention kernel to implement the same kernel of this page attention, since both of them work on the key/value cache at block level.
scv119·vor 3 Jahren·discussPretty cool stuff and the results are amazing. Hoping we will see virtual memory get standardized in pytorch or cuda.