We work with smaller data centers that have single node GPU capacity. So while there's a shortage for clouds that have capacity for clusters, we're focusing on aggregating single nodes behind a unified API. It works for our users too-- a single A100 is more than enough to fine-tune llama2 or Mistral7b on your own data.
Also, as you can imagine, the smaller data centers have a pretty rough experience for provisioning. Our API handles difficulties from those smaller data centers to make the experience consistent with provisioning from a major cloud service provider.
Hey! Co-founder of Brev.dev here and was in that video. We're actually a small team of 3-- makes it easy to maneuver when marketing and product are the same :)
We were building cloud dev environments in 2021, which I now feel strongly do not have PMF. Writing a fun post on that...
We were trying to make coding in the cloud as seamless as coding locally. Users started using that for GPUs ~1 year ago. A solid dev env experience makes sense for AI/ML developers. The infrastructure problems are more complicated & more expensive.
We've since reworked our product and aim to build AWS SageMaker but without the confines of any particular cloud. We're starting by sourcing cheap GPUs & having 1st class notebook experience for fine-tuning and training. Inference coming soon!
Hey, you could use a template on brev.dev to spin up a gpu box with the model and Jupyter notebook. Alternatively, the falcon 7b model should be small enough for colab
Finetuning a smaller model leading to better performance seems like a significant finding that'll lead to a lot of companies fine-tuning their own internal "ChatGPT"s
That makes sense, Brev.dev is a really simple way to run your code on a configured GPU without having to change your code. It'll also optimize your GPU to save money when possible.
Hey, founder of Brev.dev here. Brev lets you suspend the instances when not in use, and also auto-stops it after 3 hours of inactivity to avoid expensive surprises. Would love for you to give it a shot
That would be a great follow on. I posted the repo if you want to leave any feedback or help me continue building it out: https://github.com/brevdev/dev-browser
100%, also just looking up dev docs for frameworks. Something like "write a post API in Golang's Gin framework" instead of looking up what that syntax was again
Totally! I found ChatGPT to be more helpful in this use case since it outputted full code snippets instead of generating it line by line. The context was also extremely valuable in making iterations (like "make it work for mac")
Also, as you can imagine, the smaller data centers have a pretty rough experience for provisioning. Our API handles difficulties from those smaller data centers to make the experience consistent with provisioning from a major cloud service provider.