Hello everyone! I post-trained small LLM (Qwen Instruct 2.5 0.5B) to speak like GenZ using SFT+RL. Training was done in Google Colab using the cheapest GPU runtime.
Honestly, you could probably get better results with simple prompting of frontier models, as I wasn’t optimizing for results with the smallest model, cheapest GPU and synthetic data. But it was a good and fun learning exercise: how you can actually run RL to improve models without huge investments using the current advancements in AI, tooling and infrastructure.
Overall, it cost me <$2 using Colab's pay as you go plan to train the model, which was surprisingly less than I expected.
Notebook example is on Github, feel free to give it a try in your own free plan!
Fleevo | Founding Software Engineer | Hybrid in London, UK (NO visa sponsorship) | Full-time
Commercial fleets power the most essential operations in the physical world: trucks delivering food to our tables, vans repairing water pipes or installing power lines, construction equipment building our critical infrastructure, and much more. Fleets have a profound impact on our day-to-day lives, but they have historically been underserved by the tech industry.
Fleevo is changing that
We are transforming data into decisions for commercial fleets. Fleevo helps fleets to improve vehicle uptime, drive revenue, cut costs, optimise workflows, and operate more sustainably.
Honestly, you could probably get better results with simple prompting of frontier models, as I wasn’t optimizing for results with the smallest model, cheapest GPU and synthetic data. But it was a good and fun learning exercise: how you can actually run RL to improve models without huge investments using the current advancements in AI, tooling and infrastructure.
Overall, it cost me <$2 using Colab's pay as you go plan to train the model, which was surprisingly less than I expected.
Notebook example is on Github, feel free to give it a try in your own free plan!