Gitar runs multiple specialized AI agents on every code change. They review code, fix CI failures, execute custom repository rules as workflows, and respond to developer feedback in-thread. That's easily 50-100 LLM calls per PR, and complex ones can hit 500+. We tried swapping Claude for Kimi K2.5 at 1/5th the price over a weekend.
Three things bit us: finish_reason semantics differ between "compatible" providers, the model retried identical failing tool calls instead of adapting, and provider failover invalidated prompt caches on both sides.
We built our own harness from the ground up to account for this
Secrets come from aws secret manager and never injected into env directly.
Each part of the agentic workflow only gets the secrets it needs injected. Agent can see env var names but not the values (our harness masks them) . We also mask any attempts to output to stdout/files.
This keeps the agent architecture simple with env vars that all agents can operate on as it locally. Prompt injection attempts will only yield masked values