Great question! We explored local LLMs (including llamafile-type solutions) in our early development, but found that the reasoning capabilities and consistency weren't quite there yet for our specific needs.
That's why we currently optimize for cloud AI models while implementing intelligent plan caching to significantly reduce API costs. This approach gives you the best of both worlds: high-quality execution plans with minimal API costs, plus much faster performance for similar actions.
The expectation: beautiful architecture diagrams where agents flow like poetry.
The reality: distributed chaos that made me question my life choices.
Who else has lived through:
• The "$$ - thinking!" messages multiplying faster than rabbits
• That one agent that decides to take a vacation mid-transaction
• The classic "rollback fail" that turns your elegant system into abstract art
• Users wondering why it's "so slow..." while you're debugging cascade failures
What started as "just three agents talking" somehow turned into material worthy of a distributed systems PhD thesis. It's wild how quickly "simple agent communication" evolves into a master class in distributed systems, eventual consistency, and prayer-driven development.
But here's the thing - if you've never had your multi-agent system burst into flames, are you even building something interesting?
What's your favourite "expectation vs reality" moment? Share your war stories!
Love: Ask people how they found you. Docs are the product. Be ruthlessly iterative.