What we talk about when we talk about system designmaheshba.bitbucket.io259 points·by scv119·il y a 3 ans·35 comments
scv119·il y a 3 ans·discussThe tweet is referring to a paper that fine tunes Chinese dataset on english base model. I'm not surprised with LoRA's poor result in this setup.
scv119·il y a 3 ans·discussI believe you can slightly change the flash attention kernel to implement the same kernel of this page attention, since both of them work on the key/value cache at block level.
scv119·il y a 3 ans·discussPretty cool stuff and the results are amazing. Hoping we will see virtual memory get standardized in pytorch or cuda.