This is in fact what we do (with higher order abstractions now built on top of this). This builds self evolving interactive knowledge base and puts it into a QMD searchable index. The indexer is already open source: https://github.com/jibs/duffel
Aside 2:
Anecdotally we found that Pi performs more or less on par with native harnesses at lower cost on decently specified prompts. It is also phenomenal at context cacheing especially on Deepseek models (its hard to precisely attribute credit here are my understanding is this is a DS speciality).
But it fails much worse on poorly drafted prompts. I'm generalising but native harnesses seem to be better kind of flailing along on those.
We have an internal proxy (that I've been meaning to open source for ages) that routes all llm usage at our company, which allows us to see data in realtime. Its been fascinating how rapidly Pi has been adopted. Moreover since its pretty hackable, we've been able to automatically aggregate context from pi sessions, which has resulted in Pi efficacy being higher as more people use it, putting in place a interesting virtuous loop.
I didn't expect this outcome: for whatever reason I assumed proprietary harnesses fine tuned to work with a companies' models would work better?
ps/random aside: there is something slightly off about Pi's edit command, we are planning to investigate this further and patch this as we have quite a few session traces now..
We have switched approx 80% of our work to deepseek, and it works great. Our setup is a bit unconventional though, we upload all cot / sessions to shared storage and generate centralised project level context. We've found this is helpful in directing and working with these slightly less sota models and getting great value for ai spend.
I'm planning to open source all this infra soon, hopefully useful for others too.
Try llama.cpp it seems to be a lot more performant and a lot more hackable.
Also I'm surprised how substantial the impact of some of the inference configs (beyond just temp) can have, though this is much more model specific.
I love these lectures, his sonorous, lilting voice and surprisingly acute comedic timing, like a native english speaker. Never never quite realised it until you put it down like that, just how he weaves a whole cloth out of these cross cultural threads.
If there are any other hidden gems, these dialogues for example, please do share!