> Maybe one of you startup inclined people can make an openllama startup that charges by request
I'm currently building www.lalamon.us specifically to provide a fully hosted open source model experience. One slight difference is that I'm providing a private chat instance for each user, so charging based on hours of active chat usage seemed to make more sense. Per-request charging seems more unpredictable for users, but I'd be interested in hearing the case either way.
Feel free to reach out with more questions if interested; my email is in my profile.
I’m building hosted large language models at www.lalamon.us*. Think ChatGPT, except that you’ll be using an open source model and running it on a private cloud instance only accessible by you. We’ll launch soon and would love to hear your thoughts. Sign up for waitlist at www.lalamon.us!
Some more background about the what and why:
Like many of you, I’ve been amazed by the rapid improvement of language models like ChatGPT in the past few months. However, there are potential concerns with directly sending sensitive information to ChatGPT. In response to these concerns, the community has developed a variety of open models. When I tried running these models on my laptop, I encountered a few major pain points:
- Larger models usually perform better, but they don’t always fit in memory
- My laptop doesn’t have a GPU
- I’m currently in a location with slower internet speeds, and downloading gigabytes of model weights takes hours
In response to all this, I decided to build my own solution, with the following key features:
- Pick the latest, best-performing open models
- Run the models on powerful cloud instances with newest-generation hardware
- Put user data privacy first. Chat sessions are strongly isolated from each other. Chat data is never used for training models or harvested for corporate gain.
*Disclaimer: I’m an infrastructure engineer by training, sorry about the basic web design!
I'm currently building www.lalamon.us specifically to provide a fully hosted open source model experience. One slight difference is that I'm providing a private chat instance for each user, so charging based on hours of active chat usage seemed to make more sense. Per-request charging seems more unpredictable for users, but I'd be interested in hearing the case either way.
Feel free to reach out with more questions if interested; my email is in my profile.