Show HN: Anything To JSON – a language model for structured extraction(anythingtojson.com)
anythingtojson.com
Show HN: Anything To JSON – a language model for structured extraction
https://anythingtojson.com/
8 comments
Nice, makes sense to chain models so you don't waste the attention of the big smart model on the grunt work of JSON structure.
I bet there are some good post-processing heuristics you could also apply for hallucination with a flag for "this should be in the text verbatim" & then string matching whether the answer it outputted was a string from the text or not.
I bet there are some good post-processing heuristics you could also apply for hallucination with a flag for "this should be in the text verbatim" & then string matching whether the answer it outputted was a string from the text or not.
interesting idea! yeah next big challenge for us is hallucination in null fields (IE: if you ask for a "name" from text that doesn't have a name you usually get "John") so need to add more sampling heuristics to doublecheck against the input blob for a sense of if it's an accurate extraction
How does it differentiate from using json output with OpenAI?
I think it can be useful for some cases, but you’ll always have the limitation of the model size to compete against large closed source models.
I think it can be useful for some cases, but you’ll always have the limitation of the model size to compete against large closed source models.
openai json mode will ensure JSON, but not strictly the right JSON schema to match whatever object you're expecting to return. Keys could be missing, fields could be improperly typed. We found gpt-4 reasoning capabilities could, most of the time, make it work, but that's overkill for many
We’ve trained a small(ish) language model on structured extraction, and today we’re launching a playground for it at https://anythingtojson.com. Give it a try!
This model continues our work on structured generation, following last week’s launch of Fructose[1], a python client for strongly-typed LLM responses.
There seem to be two distinct halves of the problem intended to be solved by Fructose and structured generation:
1. the reasoning ability of the model, such as performing chain of thought, creative acts, and natural language tasks. In a way, the “business logic”.
2. the structured json response, to make sure the receiving code doesn't break
AnythingToJson is intended to solve the latter. No big-brain work, just find data in text and extract it. Constrained generation at inference time helps keep it on track, and our finetuning has improved the accuracy and consistency of extracted data.
There’s much more progress to be made (longer context window, hallucination to squash out, etc), but ship early and fast.
[1] https://news.ycombinator.com/item?id=39619053