[untitled]
1 comments
Hi HN,
For a while, we've been exploring a core tension in agentic frameworks: you either get powerful but complex, code-heavy systems (like in my EAT project https://github.com/matiasmolinas/evolving-agents), or you get flexible, human-readable systems that are incredibly slow and expensive because the LLM is the runtime (like our LLMunix experiment https://github.com/EvolvingAgentsLabs/llmunix ).
We are starting a proposal for a third way: treating the LLM as a Just-in-Time (JIT) compiler.
The core idea is that the LLM generates a hybrid stream of text and functional logic encapsulated in special tokens (<FUNC_START>). A smaller, specialized LORA then "compiles" this logic into an optimized, executable, and cacheable function. This compiled code can then recursively call the LLM for high-level reasoning when needed.
It's an attempt to get the best of both worlds: the conceptual flexibility of LLMunix with the high-performance, low-cost execution of a compiled framework. The article is a deep dive into the architecture and the journey that led us here.
I'm looking for critical feedback on the core concept. Does this JIT/LORA compiler approach seem viable? What are the potential pitfalls we're not seeing?
We're planning to build a complete, open-source POC of this system, and we're actively looking for feedback and potential collaborators. Happy to answer any questions.