We use our own implementation of function calling orchestrated by chain-of-thought. The CoT allows us more granular control over the function calls, rather than zero-shotting and hoping the LLM selects the right functions.
This post was briefly flagged after that initial comment, which is what dioptre was referring to, not AdGuard. Phrasing was too ambiguous, hope that clears it up!
Agree with you, and we're definitely trying to thread the needle!
We're generating the SQL to answer natural language questions, so folks can just get answers and results tables if that's all they need, with the option for power users to fiddle with the SQL either directly or via a query editor GUI.
There's a ton of use cases for working with unstructured and semi-structured data and that's coming down the pipe!
Wherever possible, the chatbot output is deterministic, in that to answer a query, we're realtime generating and running code or SQL against your data. Our LLM orchestrates that, and finally evaluates whether the output correctly and adequately answers the question.
We also extensively use synthetic data and examples to guide and constrain our models.
Another way we're ensuring good-quality output is to ensure good-quality _input_ -- by enriching the detail and specificity of the user's question, and asking the user to disambiguate when we determine the question is too broad.
It does considerably more than (poorly) managing the context window. It also (poorly) enables persistent document storage, knowledge retrieval, function calling and code execution.
It's more about UX, to reduce the perceived delay. LLMs inherently stream their responses, but if you wait until the LLM has finished inference, the user is sitting around twiddling their thumbs.