Show HN: Serverless platform for running voice AI agents(blog.livekit.io)
blog.livekit.io
Show HN: Serverless platform for running voice AI agents
https://blog.livekit.io/deploy-and-scale-agents-on-livekit-cloud/
0 comments
—
https://blog.livekit.io/deploy-and-scale-agents-on-livekit-cloud/
—
- How do I size CPU/memory for unpredictable session lengths?
- What's the best way to autoscale without overprovisioning?
- How can I monitor and optimize performance across concurrent sessions?
A few months ago, we set out to build a serverless platform tailored for this—think Vercel for stateful AI agents, with low-latency cold starts and seamless scaling. Along the way, we tackled some fun engineering challenges:
- Container isolation to sandbox workloads and prevent noisy-neighbor issues
- Minimizing container startup times, to ensure proper autoscaling
- Custom load balancing to distribute sessions based on real-time resource utilization, not just round-robin—since session durations vary wildly
- Graceful draining during updates or scaling events
We've been dogfooding this internally and with early users, and it's handling thousands of concurrent voice sessions with minimal latency spikes. If you're building AI agents (voice or otherwise) and wrestling with infra, we'd love your feedback—does this solve pain points you've hit? What's missing?