Stresstest my AI chat – HAMMER IT
1 comments
It needs a spinning circle or dots to show that the AI is thinking. (It can be just an animated gif :) .)
Sometimes it doesn't reply until I add a second question like "????"
Sometimes it doesn't reply until I add a second question like "????"
It was so delightful to build thanks to jazz.tools.
Vercel's Fluid compute makes it run efficient and scale endlessly
How does it work? Each message is created locally and added to the chat. At the same time I "wake up" a Vercel Function which consumes the chat, runs AI and writes directly into Jazz
The client just renders the state from Jazz. No real streaming required.
Source: https://github.com/tobiaslins/jazz-ai-chat
Whats Fluid Compute? When you run Serverless Functions, usually one invocations serves one request. With Fluid, multiple requests can be served by one function invocation, which is super useful when using AI (low CPU usage, only waiting for upstreams)
Whats Jazz? It's a new way to build apps. It's persistence (like a graph database) with permissions and sync built in.
You can access/write Jazz even from backend workers like here from Vercel Functions.