One thing that’s been bothering me about AI side projects is inference costs. With traditional software, a successful launch usually means higher profits. But with AI products, success can mean unexpectedly large bills.
This has pushed me toward cheaper, less capable models and made me hesitate to even explore certain ideas. I don’t want every side project to become another $20/month subscription, but I also can’t compete with VC-backed companies willing to subsidize inference costs.
Then I had this idea: what if users simply paid for their own inference?
This already happens in some apps through locally configured API keys, but could the model be extended? If users bring their own AI account, developers can build AI-powered products without taking on variable inference costs. How many AI applications shouldn’t be subscription businesses at all?
The challenge is that developers don’t want to handle user API keys, users don’t want to hand them out to every app they try, and nobody wants the friction of collecting payment methods just to pass through inference costs.
That’s the backstory to my latest side project, infer0.com.
It's a bit like SSO for AI inference. Users connect their AI providers once, and apps use auth tokens to request inference through infer0. Developers don’t manage API keys or pay model costs themselves, while users can bring the same AI accounts across multiple applications.
This may be a terrible idea, both because nobody will trust it and because I’m sure there are risks around handling user API credentials that I haven’t fully appreciated. But I felt the need to build it. So here’s a rough first pass.
This has pushed me toward cheaper, less capable models and made me hesitate to even explore certain ideas. I don’t want every side project to become another $20/month subscription, but I also can’t compete with VC-backed companies willing to subsidize inference costs.
Then I had this idea: what if users simply paid for their own inference?
This already happens in some apps through locally configured API keys, but could the model be extended? If users bring their own AI account, developers can build AI-powered products without taking on variable inference costs. How many AI applications shouldn’t be subscription businesses at all?
The challenge is that developers don’t want to handle user API keys, users don’t want to hand them out to every app they try, and nobody wants the friction of collecting payment methods just to pass through inference costs.
That’s the backstory to my latest side project, infer0.com.
It's a bit like SSO for AI inference. Users connect their AI providers once, and apps use auth tokens to request inference through infer0. Developers don’t manage API keys or pay model costs themselves, while users can bring the same AI accounts across multiple applications.
This may be a terrible idea, both because nobody will trust it and because I’m sure there are risks around handling user API credentials that I haven’t fully appreciated. But I felt the need to build it. So here’s a rough first pass.