tylerekahn·3 ปีที่แล้ว·discussIt’s actually as low as 0.01% of the original weights.From the LoRa paper:>When the pre-trained model is GPT-3 175B, the number of train- able parameters |Θ| can be as small as 0.01% of |Φ0|.
tylerekahn·3 ปีที่แล้ว·discussLow Rank Adaption is a mathematical technique, it's not a technology standard