mvsin·hace 2 años·discussSomething like this does exist, production systems rarely use greedy search but have more holistic search algorithms.An example is Beam Search:https://www.width.ai/post/what-is-beam-searchEssentially we keep a window of probabilities of predicted tokens to improve the final quality of output.
An example is Beam Search:https://www.width.ai/post/what-is-beam-search
Essentially we keep a window of probabilities of predicted tokens to improve the final quality of output.