Show HN: Cost per Outcome for AI Workflows(github.com)
github.com
Show HN: Cost per Outcome for AI Workflows
https://github.com/botanu-ai/botanu-sdk-python
4 comments
“Cost per outcome” is the metric most teams actually need. In prod we saw totals look fine while cost/outcome drifted due to retries + fallback paths + context creep. Are you planning a before/after deploy comparison (prompt/version) to catch regressions, or anomaly alerts on cost/outcome slope?
Yes, we are building both
Nice — those two features tend to unlock the “why” behind drift. One thing we found especially useful was pairing cost/outcome alerts with a root-cause slice: when slope jumps, immediately show top contributing endpoint/feature + tenant/user + prompt version changes + retry ratio/context size trend.
For your event_id model: how do you handle partial outcomes (e.g., success after fallback/escalation) and do you keep pricing snapshots by timestamp so historical cost/outcome comparisons stay consistent across model price changes?
I started building this after repeatedly hitting the same problem on AI teams: we could see total LLM spend, but couldn’t answer “what did one successful outcome actually cost?”. In real systems, a single business event often requires multiple runs ex-retries, fallbacks, escalations, async workers etc., before it reaches a final outcome. Most tooling tracks individual calls, or at best single runs. That hides the true cost. botanu treats cost per outcome as the sum of all runs and attempts for an event, including failures.
How it works -An event represents business intent
-Each attempt is a run, with its own run_id
-All runs are linked via a shared event_id
-A single outcome (success / failure / partial) is emitted for the event
-Total cost = cost of all runs for that event
-Run context propagates across services using standard W3C Baggage (OpenTelemetry).
I’m building this as part of a broader effort around outcome-based pricing for AI systems and understanding true cost per outcome. If you’re thinking about similar problems, I’d love to chat and compare notes. Happy to answer technical questions or get critical feedback. Email- [email protected]