We work with smaller data centers that have single node GPU capacity. So while there's a shortage for clouds that have capacity for clusters, we're focusing on aggregating single nodes behind a unified API. It works for our users too-- a single A100 is more than enough to fine-tune llama2 or Mistral7b on your own data.
Also, as you can imagine, the smaller data centers have a pretty rough experience for provisioning. Our API handles difficulties from those smaller data centers to make the experience consistent with provisioning from a major cloud service provider.
Hey! Co-founder of Brev.dev here and was in that video. We're actually a small team of 3-- makes it easy to maneuver when marketing and product are the same :)
We were building cloud dev environments in 2021, which I now feel strongly do not have PMF. Writing a fun post on that...
We were trying to make coding in the cloud as seamless as coding locally. Users started using that for GPUs ~1 year ago. A solid dev env experience makes sense for AI/ML developers. The infrastructure problems are more complicated & more expensive.
We've since reworked our product and aim to build AWS SageMaker but without the confines of any particular cloud. We're starting by sourcing cheap GPUs & having 1st class notebook experience for fine-tuning and training. Inference coming soon!
Hey, you could use a template on brev.dev to spin up a gpu box with the model and Jupyter notebook. Alternatively, the falcon 7b model should be small enough for colab
Finetuning a smaller model leading to better performance seems like a significant finding that'll lead to a lot of companies fine-tuning their own internal "ChatGPT"s
That makes sense, Brev.dev is a really simple way to run your code on a configured GPU without having to change your code. It'll also optimize your GPU to save money when possible.
Also, as you can imagine, the smaller data centers have a pretty rough experience for provisioning. Our API handles difficulties from those smaller data centers to make the experience consistent with provisioning from a major cloud service provider.