rassibassi

hace 2 añosdiscuss

CFG (context free grammar), which is explained in the book, is used here together with LLMs: https://tree-diffusion.github.io

rassibassi

hace 2 añosdiscuss

In this context, RAG isn't what's being discussed. Instead, the reference is to a process similar to monte carlo tree search, such as that used in the AlphaGo algorithm.

Presently, a large language model (LLM) uses the same amount of computing resources for both simple and complex problems, which is seen as a drawback. Imagine if an LLM could adjust its computational effort based on the complexity of the task. During inference, it might then perform a sort of search across the solution space. The "search" mentioned in the article means just that, a method of dynamically managing computational resources at the time of testing, allowing for exploration of the solution space before beginning to "predict the next token."

At OpenAI Noam Brown is working on this, giving AI the ability to "ponder" (or "search"), see his twitter post: https://x.com/polynoamial/status/1676971503261454340

rassibassi

hace 2 añosdiscuss

The diskcache docs state:

""" Caching Libraries

    joblib.Memory provides caching functions and works by explicitly saving the inputs and outputs to files. It is designed to work with non-hashable and potentially large input and output data types such as numpy arrays.

""" From https://pypi.org/project/diskcache/

rassibassi

hace 2 añosdiscuss

What's the difference to using joblibs Memory class similar to this implementation:

https://github.com/stanfordnlp/dspy/blob/main/dsp/modules/ca...

comments