Show HN: LoRA can make distributed fine-tuning far faster(ericyu3.notion.site)
ericyu3.notion.site
Show HN: LoRA can make distributed fine-tuning far faster
https://ericyu3.notion.site/LoRA-makes-distributed-fine-tuning-far-faster-901ed36c5ba84d2ea7018dd2a36911af
3 comments
How is the model performance? I heard lora on LLM generally get inferior results comparing to no LORA finetuning
Generally good as long as you target all layers.
LoRA AWS swarms are 60x 1 GPU throughput for fine-tuning?? Kind of a big find