We used frameworks in the past, tried langchain, langgraph, and Openai's agents SDK pretty extensively. Now we roll our own, generally a much better and cleaner experience. We essentially built our own internal framework for our own use-case, we liked the graph approach of langgraph - so we took elements of that. We write everything async, and added nice handling for streaming.
You can see our framework [here](https://github.com/aurelio-labs/graphai). I don't necessarily recommend it as it's built for our use-cases and I make no guarantees for others, but it might be interesting to see what rolling your own might look like
Sharing my walkthrough on fine-tuning LLMs with LoRA using NVIDIA's NeMo microservices. The result is a llama-3.2-1b-instruct model fine-tuned to be really good at function-calling, making it ideal for agent-use.
It was a ton of fun to figure it out and it brought back some nostalgia from the days of training ML models, tweaking learning rates, dropout, and watching loss charts in W&B.
Final performance was way better than any 1-3B parameter LLM I tried with agentic workflows in the past.
Releasing this walkthrough on fine-tuning LLMs with LoRA using NVIDIA's NeMo Microservices (they sponsored the video, but with no reqs on what I do or say). We cover a ton on building prod AI applications, including:
- LoRA fine-tuning of Llama 3.2 1B
- Deploy of private AI agent systems
- Using NVIDIA NIMs to host our fine-tuned LLM
- Interacting with our LLM + streamed output
It was a ton of fun to figure this out, and it brought back some nostalgia from the good old days of training ML models, tweaking learning rates and dropout, and watching loss charts in W&B.
The result is a llama-3.2-1b-instruct fine-tuned to provide pretty good function-calling abilities (better than any other out-of-the-box 1-3B models I tried).
You can see our framework [here](https://github.com/aurelio-labs/graphai). I don't necessarily recommend it as it's built for our use-cases and I make no guarantees for others, but it might be interesting to see what rolling your own might look like