Anthropic: Demystifying Evals for AI Agents(anthropic.com)
anthropic.com
Anthropic: Demystifying Evals for AI Agents
https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
1 コメント
I work on Promptfoo (an open-source eval framework). Appreciate the mention here. This post captures a lot of the hard lessons around agent evals. In particular, task ambiguity and brittle graders are things we run into constantly.