Anyone else losing tokens to hallucinated MCP tool calls in production?
I have been building an agentic system on a custom internal platform and the llm keeps calling tools with identifiers that dont exist, wrong namespace, wrong handle, wrong enum. gets back an error, retries, still wrong. every bad call is tokens down the drain.
I ended up writing a big system prompt to fix it. took weeks of trial and error. its working but i still dont fully trust it.
I am curious if others are hitting this or if its just me. Especially if youre running MCP servers on internal platforms the llm has never seen before(the problem occurs most with less powerful models), do you guys have a go to solution for this? Thanks
5 comments
One recommendation would be to create a skill (name_of_your_internal_tool/SKILL.md) describing how the agent should use the tool(s) you're working with. This allows you to progressively disclose that context to the agent rather than filling its context (via its system prompt) with those instructions on every turn. If you use Claude Code, you can also try using it to interactively create the skill (it has a `skill-creator` skill). On the specific skills to make, you could align it with the specific tool (e.g., "Read this skill whenever the user asks you to use X tool...") or with a particular action (e.g., "Read this skill whenever the user asks you to [write, create, etc.]..."
The particular issue arises with the models from other providers, but thanks for the suggestion, it's helpful.
Yes, and that's one of the reasons I started working on this tool: to elicit desirable behavior from AIs before turn 1. AI is still non-deterministic and won't follow instructions 100% of the time, but with this tool, I intend to narrow that gap.
Would love to hear your thoughts on it: https://github.com/turnzero-ai/turnzero
Would love to hear your thoughts on it: https://github.com/turnzero-ai/turnzero
Hey, thanks for dropping the tool, will let you know about my thoughts soon.
Thank you Mahjabinbm, looking forward to it.