HN 보기: Sqlsure - AI 생성 SQL에 대한 결정론적 의미 검사(github.com)
github.com
Show HN: Sqlsure – deterministic semantic checks for AI-generated SQL
https://github.com/sqlsure/sqlsure
11 comments
I don't know why but I really struggle comprehending AI written READMEs and comments. I understand the words but the way it's written is just distracting and unintelligible.
Fair point, I legit have a policy not to read anyone's "Thoughts" if they were AI -- hypocritical of me I went on with AI assisted documentation. I wanted a quick POC repo and I went the lazy way -- would love to redo any sections that didn't make sense.
How much of it is written by AI?
This was a passion weekend project started with only 60lines of code and idea to translate semantic and check sql against it. So definitely leveraged claude for documentation, testing, scaling.
However, personally I do want to redo a lot of sections that feel slop-esque. So if you do have any specific feedback on particular sections, would be happy to apply.
However, personally I do want to redo a lot of sections that feel slop-esque. So if you do have any specific feedback on particular sections, would be happy to apply.
Author here. sqlsure is a semantic inspector for SQL: it checks queries — human- or AI-written — against declared facts about your schema (grain, join cardinality, measure additivity) before execution. It never generates SQL, never reads your data, and runs in ~0.1 ms per check, fully offline. Apache-2.0.
The bug it targets: fan-out double-counting. Join orders to order_items and SUM(order_total), and every dollar is counted once per line item. The query runs fine, the dashboard renders, the number is silently wrong.
To test it, we ran it over the gold (expert-written) answer keys of the Spider and BIRD text-to-SQL benchmarks — 2,568 queries, using only the benchmarks' own PK/FK declarations as the rulebook. It raised 45 flags and zero spurious ones. For one BIRD query we executed the benchmark's own database and proved the official gold answer wrong by 8× (the fan-out factor). Filed upstream: https://github.com/bird-bench/mini_dev/issues/37
Three ways to use it: CLI (CI gate), MCP server (agents call check_sql before executing; rejections carry machine-actionable fix hints — in our benchmark, applying the hint verbatim fixed the query 10/10 times in one round), or as a Python library wrapping an existing text-to-SQL generator.
Rulebooks come from what you already have: dbt tests (unique / relationships), PK/FK declarations, or OSI semantic model YAML. Happy to answer anything about the method or the benchmark findings.
The bug it targets: fan-out double-counting. Join orders to order_items and SUM(order_total), and every dollar is counted once per line item. The query runs fine, the dashboard renders, the number is silently wrong.
To test it, we ran it over the gold (expert-written) answer keys of the Spider and BIRD text-to-SQL benchmarks — 2,568 queries, using only the benchmarks' own PK/FK declarations as the rulebook. It raised 45 flags and zero spurious ones. For one BIRD query we executed the benchmark's own database and proved the official gold answer wrong by 8× (the fan-out factor). Filed upstream: https://github.com/bird-bench/mini_dev/issues/37
Three ways to use it: CLI (CI gate), MCP server (agents call check_sql before executing; rejections carry machine-actionable fix hints — in our benchmark, applying the hint verbatim fixed the query 10/10 times in one round), or as a Python library wrapping an existing text-to-SQL generator.
Rulebooks come from what you already have: dbt tests (unique / relationships), PK/FK declarations, or OSI semantic model YAML. Happy to answer anything about the method or the benchmark findings.
But your example is not convincing this is a common enough problem to merit a library.
You have a table with cost that sums to 400. If you summed that table you'd get the same error. You don't need a fan-out JOIN to get the error...
That seems like bad database design, and creating a separate config file to mask the bad database design, rather than fixing the actual problem.
I think this could actually be useful, but I'd recommend a better example.
An OLAP table should be designed so that values can be summed if that's the purpose of the table. A relational table should be designed so that you don't have replicated bad data.
An OLAP table could have unintended many-to-ones, but it still shouldn't have this problem. Maybe I'm being naive, but I think the better solution is to fix the problem, not the queries.
You could get this problem, with a different example, but I'm not convinced this library is the best solution to that problem. Looker and DBT already mostly or completely solve it.