Ask HN: How are you testing LLM integrations in CI?1 points·by akarshc·5 mesi fa·0 commentsWe’ve been integrating LLM features into our app and noticed something subtle.Our CI pipeline was calling live models during integration tests. That means every pull request consumes tokens.Mocking at the HTTP layer felt brittle. Hardcoded fixtures were unrealistic. Calling live models felt unnecessary just to validate application logic.Curious how others here are handling this.Are you stubbing SDK calls? Using deterministic fixtures? Or just running tests against live providers?Would love to hear what has worked in production.0 commentsPost comment—
Our CI pipeline was calling live models during integration tests. That means every pull request consumes tokens.
Mocking at the HTTP layer felt brittle. Hardcoded fixtures were unrealistic. Calling live models felt unnecessary just to validate application logic.
Curious how others here are handling this.
Are you stubbing SDK calls? Using deterministic fixtures? Or just running tests against live providers?
Would love to hear what has worked in production.