What we talk about when we talk about system designmaheshba.bitbucket.io259 points·by scv119·3 वर्ष पहले·35 comments
scv119·3 वर्ष पहले·discussThe tweet is referring to a paper that fine tunes Chinese dataset on english base model. I'm not surprised with LoRA's poor result in this setup.
scv119·3 वर्ष पहले·discussI believe you can slightly change the flash attention kernel to implement the same kernel of this page attention, since both of them work on the key/value cache at block level.
scv119·3 वर्ष पहले·discussPretty cool stuff and the results are amazing. Hoping we will see virtual memory get standardized in pytorch or cuda.