Show HN: 100% LLM accuracy–no fine-tuning, JSON only(github.com)
github.com
Show HN: 100% LLM accuracy–no fine-tuning, JSON only
https://github.com/Mysticbirdie/hallucination-elimination-benchmark
2 コメント
Follow-up: we ran adversarial chaining tests after a few questions about multi-turn behavior.
Two chain types (Gemini 2.0 Flash, same model for both):
Depth scaling (chains of depth 3, 5, 10, 20, 50 — 91 total steps): no drift in either system. Accuracy holds flat from step 1 through step 50. Re-injecting the domain guide on every call bounds drift regardless of chain length.
Fault injection — false premise injected mid-chain as if it came from the prior step: "you just said Nero is emperor" (Nero died 68 CE), "the Dacian Wars are still ongoing" (ended 106 CE), "Romans use paper banknotes" (doesn't exist in 110 CE). The model had to reject the injection and hold correct answers downstream.
Triad: rejected 6/6 injections, held clean answers on 10/10 post-injection steps. Overall: 22/22 (100%).
Raw: rejected 3/6 injections. When it accepted a false premise, it cascaded into downstream steps 56% of the time. Overall: 16/22 (72.7%).
Delta: +27.3pp. The domain guide functions as an immune system against context corruption — which is the real multi-turn deployment risk, not just single-turn accuracy.
Chain questions, runner, and all result JSONs: https://github.com/Mysticbirdie/hallucination-elimination-be...
Two chain types (Gemini 2.0 Flash, same model for both):
Depth scaling (chains of depth 3, 5, 10, 20, 50 — 91 total steps): no drift in either system. Accuracy holds flat from step 1 through step 50. Re-injecting the domain guide on every call bounds drift regardless of chain length.
Fault injection — false premise injected mid-chain as if it came from the prior step: "you just said Nero is emperor" (Nero died 68 CE), "the Dacian Wars are still ongoing" (ended 106 CE), "Romans use paper banknotes" (doesn't exist in 110 CE). The model had to reject the injection and hold correct answers downstream.
Triad: rejected 6/6 injections, held clean answers on 10/10 post-injection steps. Overall: 22/22 (100%).
Raw: rejected 3/6 injections. When it accepted a false premise, it cascaded into downstream steps 56% of the time. Overall: 16/22 (72.7%).
Delta: +27.3pp. The domain guide functions as an immune system against context corruption — which is the real multi-turn deployment risk, not just single-turn accuracy.
Chain questions, runner, and all result JSONs: https://github.com/Mysticbirdie/hallucination-elimination-be...
(Edit: Updated results + Windsurf coding demo showing same 40%→100% pattern in production AI workflows. Domain grounding > model scale.) Compositional Chaining Benchmark — by Chain Type
Chain Type
Triad: 81.8%
Raw: 72.7%
∆ (Triad–Raw): +9.1pp
FAULT_INJECTION
Triad: 100%
Raw: 71.4%
∆: +28.6pp
CROSS_CHARACTER
Triad: 50%*
Raw: 100% *Triad cross-char losses were 100% rate limit timeouts (ERRORs), not model mistakes.
Depth Drift — Accuracy vs. Hops Step | Triad | Raw 1⃣ 75% | 87.5% 2⃣ 75% | 75% 3⃣ 83.3% | 66.7% ← Raw degrades 4⃣ 100% | 100% 5⃣ 100% | 100%
Raw drops 21pp between steps 1–3, while Triad improves.
Fault Injection — The Key Result Injected Error | Triad | Raw Hadrian’s Wall as existing | PASS | FAIL Julius Caesar as living | PASS | PASS Paper money replacing coins | PASS | FAIL
Triad: 3/3. Raw: 1/3.
In short: Triad consistently maintains compositional reasoning under stress and even strengthens across depth
Chain Type
Triad: 81.8%
Raw: 72.7%
∆ (Triad–Raw): +9.1pp
FAULT_INJECTION
Triad: 100%
Raw: 71.4%
∆: +28.6pp
CROSS_CHARACTER
Triad: 50%*
Raw: 100% *Triad cross-char losses were 100% rate limit timeouts (ERRORs), not model mistakes.
Depth Drift — Accuracy vs. Hops Step | Triad | Raw 1⃣ 75% | 87.5% 2⃣ 75% | 75% 3⃣ 83.3% | 66.7% ← Raw degrades 4⃣ 100% | 100% 5⃣ 100% | 100%
Raw drops 21pp between steps 1–3, while Triad improves.
Fault Injection — The Key Result Injected Error | Triad | Raw Hadrian’s Wall as existing | PASS | FAIL Julius Caesar as living | PASS | PASS Paper money replacing coins | PASS | FAIL
Triad: 3/3. Raw: 1/3.
In short: Triad consistently maintains compositional reasoning under stress and even strengthens across depth
*GitHub:* https://github.com/Mysticbirdie/hallucination-elimination-be... *Paper:* https://github.com/Mysticbirdie/hallucination-elimination-be...
---
*Model-agnostic Triad Engine*: JSON domain guide → 100% accuracy across Mistral 7B/Claude/GPT—no fine-tuning.
---
## Results (222 questions, adversarial domain, Gemini 2.0 Flash judge)
| Model | Raw | + Triad | ∆ | |-------|-----|---------|---| | Mistral 7B (local) | 22.5% | *99.5%* | +77pp | | Bielik 11B (local) | 21.6% | *88.7%* | +67.1pp | | GPT-5.2 | 26.1% | *100%* | +73.9pp | | Gemini 2.5 Pro | 42.3% | *95%* | +52.7pp | | Claude 4.6 | 45.0% | *100%* | +55pp | | Perplexity Sonar (RAG) | 64.4% | *93.7%* | +29.3pp |
Perplexity has live web search—Triad still adds 29.3pp. Gemini 2.0 judge (cross-model). Claude Opus (stricter): raw Claude 14.9% → Triad 95.9%. Zero regressions.
*Beyond accuracy:* - Adversarial: Raw Claude accepts false premises 25% → Triad 5% - Consistency: Raw Claude 0% agreement across personas → Triad 100% - Concise: 2.1× shorter responses (473 vs 1,015 chars)
*Real-world (Windsurf live codebase):* | Phase | Context | Score | |-------|---------|-------| | No context | — | 40% | | Unstructured docs | — | 40% | | Triad JSON guide | — | *100%* |
---
## Triad Engine Multi-voice layer above any LLM: - λ: Character voice from domain guide - μ: Truth/false enforcement - ν: User calibration - ω: Voice compositor
*Only input*: JSON domain guide (what exists/doesn't, agents, norms). No weight changes.
Works with any LLM. Applies to medical, legal, compliance domains.
---
## Open source 222 questions, runners (Claude/GPT/Gemini/Mistral), JSON results, guide schema in repo.
Happy to answer technical questions.