Ask HN: How do you know when your AI agent's output quality degrades?

1 pointsby ysimashkevych15 dni temu1 comments

Running an AI agent in production and curious how others handle this. Cost and latency monitoring is solved (Helicone, Langfuse, etc.) but I can't find a good answer for: how do you get alerted when the quality of your agent's responses drops before a customer tells you?

What are you doing today?

2 comments

kelrouter8 dni temu

heyyy,I'm building bitrouter (https://github.com/bitrouter/bitrouter), an open-source agentic LLM gateway & router that optimize production agentic loops in your way. Hope this can help solve your problem:)

yuto_makiharaprzedwczoraj