I'm Li Yin ([GitHub](https://github.com/liyin2015)), the author of AdalFlow and a former AI researcher at Meta AI.
AdalFlow was inspired by a viral [LinkedIn post](https://www.linkedin.com/posts/li-yin-ai_both-ai-research-an...) I made, discussing how the LLM ecosystem lacks a shared library that bridges the gap between research and product development—similar to how PyTorch has streamlined model training and adaptation.
I decided to build this library while working on my product, a conversational search engine called [Sylph](https://sylph.ai/). After trying out existing libraries and finding that I had to write everything myself, I ended up with a solution that was lighter, faster, and offered more control. However, managing the codebase soon became overwhelming.
AdalFlow is based on my vision for the future of LLM applications, which I see as a three-stage workflow:
- *V1*: Use the library to quickly build your initial task pipeline, getting you 70-80% of the way to production.
- *V2*: Auto-optimize the prompt to push an additional 10%, bringing your product to a near-ready state without the hassle of manual prompt iteration.
- *V3*: Leverage V2 to label more data. As more users interact with your product, the next step is to fine-tune the LLM, further optimizing for speed, accuracy, and cost-effectiveness.
We've completed V1 and V2. Our auto-optimizer can enhance GPT-3.5 performance to match that of GPT-4, making any task nearly production-ready. Our architecture is the most robust, lightweight, and modular, with our auto-optimizer being the most accurate—even when compared to Dspy and Text-Grad. We have three research papers coming out soon that will explain how we achieved this. This is the first time the library has been released ahead of the research papers.
It’s definitely worth checking out—you might be surprised by the results. We've had similar experiences using PyTorch and PyTorch Lightning.
I'm Li Yin ([GitHub](https://github.com/liyin2015)), the author of AdalFlow and a former AI researcher at Meta AI.
AdalFlow was inspired by a viral [LinkedIn post](https://www.linkedin.com/posts/li-yin-ai_both-ai-research-an...) I made, discussing how the LLM ecosystem lacks a shared library that bridges the gap between research and product development—similar to how PyTorch has streamlined model training and adaptation.
I decided to build this library while working on my product, a conversational search engine called [Sylph](https://sylph.ai/). After trying out existing libraries and finding that I had to write everything myself, I ended up with a solution that was lighter, faster, and offered more control. However, managing the codebase soon became overwhelming.
AdalFlow is based on my vision for the future of LLM applications, which I see as a three-stage workflow:
- *V1*: Use the library to quickly build your initial task pipeline, getting you 70-80% of the way to production. - *V2*: Auto-optimize the prompt to push an additional 10%, bringing your product to a near-ready state without the hassle of manual prompt iteration. - *V3*: Leverage V2 to label more data. As more users interact with your product, the next step is to fine-tune the LLM, further optimizing for speed, accuracy, and cost-effectiveness.
We've completed V1 and V2. Our auto-optimizer can enhance GPT-3.5 performance to match that of GPT-4, making any task nearly production-ready. Our architecture is the most robust, lightweight, and modular, with our auto-optimizer being the most accurate—even when compared to Dspy and Text-Grad. We have three research papers coming out soon that will explain how we achieved this. This is the first time the library has been released ahead of the research papers.
It’s definitely worth checking out—you might be surprised by the results. We've had similar experiences using PyTorch and PyTorch Lightning.
To learn more about our optimizer, visit: https://adalflow.sylph.ai/use_cases/classification.html.
Best,
Li