jeeceebees

·2년 전·discuss

You can just mask the output probabilities for each token based on which options are valid according to a grammar.

There are quite a few open source implementations of this e.g. https://github.com/outlines-dev/outlines