Ask HN: Anyone frustrated by AI tools hallucinating API calls?
4 comments
Just to clarify — I'm not trying to promote a product here. I'm genuinely trying to understand if others are running into this problem too.
I’ve talked to a few devs who said their AI tools confidently suggest incorrect SDK usage, especially when docs change frequently.
They also mentioned that they either manually copy and paste the relevant doc page (highly unoptimized) or keep the docs opened in parallel in a browser tab.
Would love to hear if that's happening to others, and if so, how you're working around it. Manual doc digging? Custom tools?
Also happy to go into how I’m approaching the solution if anyone’s interested.
I’ve talked to a few devs who said their AI tools confidently suggest incorrect SDK usage, especially when docs change frequently.
They also mentioned that they either manually copy and paste the relevant doc page (highly unoptimized) or keep the docs opened in parallel in a browser tab.
Would love to hear if that's happening to others, and if so, how you're working around it. Manual doc digging? Custom tools?
Also happy to go into how I’m approaching the solution if anyone’s interested.
Yesterday I had a very frustrating experience with Gemini flash 2.0.
I was lean an streamlit app that went from poc to production servers as a showcase....
Then came requirements for user registering,and someone tried to bolt in recaptcha by some magic unsafe markdown evaluation.
I haven't touched streamlit in a year so LLM looked like the right tool.
It kept saying some package existed in pipy, even after I said I googled, searched in pipy, followed their link (that get me to a 404) and posted the result of pip install failing.
In it's reasoning he kept saying he was right...
I was lean an streamlit app that went from poc to production servers as a showcase....
Then came requirements for user registering,and someone tried to bolt in recaptcha by some magic unsafe markdown evaluation.
I haven't touched streamlit in a year so LLM looked like the right tool.
It kept saying some package existed in pipy, even after I said I googled, searched in pipy, followed their link (that get me to a 404) and posted the result of pip install failing.
In it's reasoning he kept saying he was right...
Yeah that's exactly what I am talking about. Like I was working on this livekit project and I asked cursor, could you help me override this method? it very confidently gave me the incorrect answer and instead of speeding things up, I had to spend 2 hours just to figure out how do it by digging in their docs.
I am just curious here, would this be something you'd be willing to beta test? while I build it? I am dogfooding my own system and I wanna work with 5-10 people who have felt this pain and want to do something about it.
I am just curious here, would this be something you'd be willing to beta test? while I build it? I am dogfooding my own system and I wanna work with 5-10 people who have felt this pain and want to do something about it.
[deleted]
When integrating third-party APIs or SDKs, the AI often gives me outdated or completely wrong info. Sometimes it’s minor (wrong param), sometimes it burns hours.
I figured the root problem is docs go stale, but the AI still tries to be confident. So I’m building a tool (ChatVisible) that makes API/SDK docs AI-compatible and keeps them updated for AI tools.
I’m not here to pitch — I just want to hear:
- Has anyone else run into this problem?
- How are you solving it?
- Would something like this actually help?
Happy to share a link or walkthrough if anyone’s curious, but more interested in validating the problem.