Show HN: VLLM with JSON Guided Generation(outlines-dev.github.io)
outlines-dev.github.io
Show HN: VLLM with JSON Guided Generation
https://outlines-dev.github.io/outlines/reference/vllm/
3 comments
this is really cool probably the most wanted feature in LLM open source.
It would be nice to have regex support too?
do you plan to support regex?
do you plan to support regex?
We already support regex-guided generation in the library, and could easily make an API to serve this as well if that's a feature people want!
I need it yes. that would be amazing tbh.
My colleague, Rémi, created some patches that allow one to pass vLLM a JSON schema along with the prompt, which dramatically simplifies deployment of JSON-guided generation. He also added a new `serve` interface that puts it all together and makes serving such models a 2-3 line process.
Check it out and tell us what you think!