Show HN: SafeParse – schema validation and retries for AI pipelines(safeparse.com)
safeparse.com
Show HN: SafeParse – schema validation and retries for AI pipelines
https://safeparse.com
4 comments
This looks very cool, thanks for sharing.
One thing I’m trying to reason about with tools like this: how does SafeParse fit with provider‑side structured outputs (OpenAI’s structured output / tool calls, Anthropic tool use, etc.)?
Do you still run your own schema validation + retry loop on top of those APIs, or do you assume model‑enforced schemas and mostly focus on tracing / retries for transport errors and rate limits? Curious where you see the boundary between “let the model guarantee the schema” vs “treat the model as an untrusted JSON generator and validate/retry on our side.”
One thing I’m trying to reason about with tools like this: how does SafeParse fit with provider‑side structured outputs (OpenAI’s structured output / tool calls, Anthropic tool use, etc.)?
Do you still run your own schema validation + retry loop on top of those APIs, or do you assume model‑enforced schemas and mostly focus on tracing / retries for transport errors and rate limits? Curious where you see the boundary between “let the model guarantee the schema” vs “treat the model as an untrusted JSON generator and validate/retry on our side.”
Thanks, good question. I still treat the model as “mostly helpful but not fully trusted,” even when using structured outputs / tool calls.
Those APIs definitely help, but in practice I still run into things like required fields coming back empty, weird edge cases that technically pass the schema, or just timeouts / rate limits breaking the pipeline.
So SafeParse still validates the response again and retries if something looks wrong. It can also fall back to another model if the provider is having issues.
I mostly think of provider-side structured outputs as improving the baseline, and SafeParse handling the messy operational stuff once these pipelines run continuously in production.
Those APIs definitely help, but in practice I still run into things like required fields coming back empty, weird edge cases that technically pass the schema, or just timeouts / rate limits breaking the pipeline.
So SafeParse still validates the response again and retries if something looks wrong. It can also fall back to another model if the provider is having issues.
I mostly think of provider-side structured outputs as improving the baseline, and SafeParse handling the messy operational stuff once these pipelines run continuously in production.
I’ve been building a few automation pipelines that rely on LLMs, and I kept running into the same issue.
Everything works fine during testing, then breaks in production.
Typical things I saw:
– the JSON shape changes slightly – required fields come back empty – the model times out or rate-limits – downstream automations silently fail
Even with JSON mode or structured outputs, this still happens occasionally.
After dealing with this a few times I started building a small service that sits between the model and the rest of the pipeline.
The idea is simple:
define a schema send text to a webhook get validated JSON back
If the response doesn’t match the schema, SafeParse retries with context or falls back to another model. Every request is logged so failures can be replayed and debugged.
The goal is basically to make LLM-powered pipelines behave more like production infrastructure instead of fragile prompt wrappers.
I just put up a landing page + demo while I test whether this is a real problem for other people building AI pipelines.
https://safeparse.com
Curious if others here have run into similar reliability issues with LLM-based workflows.