LangChain Is a Black Box
30 comments
Oh my god, I feel exactly the same way. I am not a programmer, but I'm excited by the potential of LLMs in my work. So, I've dusted off my python skills that I haven't used since I was in an fMRI lab, and experimenting to see what works.
First of all, every library/framework I've found is moving so fast that all the tutorials and printed material (O'Reilly books etc) are already out of date. Many of the changes are out of necessity, as it's a rapidly developing space, but sometimes it just feels like someone got high and decided to add 3 more layers of abstraction. Although for many tasks, AI coding assistants would be a benefit for noobs like me, the code base and documentation are too loose for me to get the expected benefits I would find in a more established code base.
LangChain seems to be where a lot of the action is with regard to modularity, and using different components in each part of the pipeline. That's important for me, because I need either local or HIPAA-compliant tools (Azure OpenAI works, Anthropic won't return my requests for a BAA, and I need a bigger GPU).
But using LangChain is a pretty horrible experience because, at least for my uses and as a noob, it's much too buried in abstractions to make quick iterations. The GUI-based stuff like flowise and langflow are too limited with regard to available components, and mostly they hide the problems so that errors are tough to address.
I'm thrilled that there has been so much work on adding JSON output and agent stuff at the LLM level, as hopefully it can bring some of these astronauts back to earth (or at least in a low orbit).
First of all, every library/framework I've found is moving so fast that all the tutorials and printed material (O'Reilly books etc) are already out of date. Many of the changes are out of necessity, as it's a rapidly developing space, but sometimes it just feels like someone got high and decided to add 3 more layers of abstraction. Although for many tasks, AI coding assistants would be a benefit for noobs like me, the code base and documentation are too loose for me to get the expected benefits I would find in a more established code base.
LangChain seems to be where a lot of the action is with regard to modularity, and using different components in each part of the pipeline. That's important for me, because I need either local or HIPAA-compliant tools (Azure OpenAI works, Anthropic won't return my requests for a BAA, and I need a bigger GPU).
But using LangChain is a pretty horrible experience because, at least for my uses and as a noob, it's much too buried in abstractions to make quick iterations. The GUI-based stuff like flowise and langflow are too limited with regard to available components, and mostly they hide the problems so that errors are tough to address.
I'm thrilled that there has been so much work on adding JSON output and agent stuff at the LLM level, as hopefully it can bring some of these astronauts back to earth (or at least in a low orbit).
Just in case anyone comes back to this, I found the following resource helpful:
Brandon Hancock’s (3 hour!!) YouTube video and accompanying GitHub repo: https://github.com/bhancockio/langchain-crash-course https://youtu.be/yF9kGESAi3M
I’m not in a position to say, but perhaps many people will ultimately go with directly using the python libraries and APIs of their components. But if you are messing around with different components until you settle on something, it’s useful.
Brandon Hancock’s (3 hour!!) YouTube video and accompanying GitHub repo: https://github.com/bhancockio/langchain-crash-course https://youtu.be/yF9kGESAi3M
I’m not in a position to say, but perhaps many people will ultimately go with directly using the python libraries and APIs of their components. But if you are messing around with different components until you settle on something, it’s useful.
Hi, CISO at Anthropic here. Sorry that we didn't respond to your BAA request. I am accountable for our response to BAA requests and I'd like to dig into what happened here. If you are comfortable, would you please reach out to me at [email protected] to let me know how you sent your request in?
We're moving our entire codebase away from Langchain and related products (langgraph, langsmith). I originally thought the library had potential and just needed some polishing which would come with time, but it's just been getting worse and worse. LCEL is the weirdest half-baked idea; it's a nice gimmick that makes your code impossible to debug in a step through debugger. Random stuff breaks with every other release. I deeply regret using it by now.
Are you replacing LangGraph with something else? I've luckily avoided LangChain exactly because it's a black box, but been on the lookout for some kind of DAG visualization of LLM flows. LangGraph pops up first on any related searches.
We're not sure what to do with langgraph yet, interested if anyone here has tried viable alternatives. We could implement a simplified execution graph in-house, but it feels like reinventing the wheel. Also thinking to keep the "graph" part of langgraph but replace all everything else (i.e. tool calls, retrievers, etc.) by non-langchain alternatives.
I haven't used LangGraph myself, but the latest magentic release is compatible with it if you'd like to check out the examples here https://github.com/jackmpcollins/magentic/issues/287
It is indeed a terrible library on multiple levels. Get rid of it. Just use the "native" Python SDK for the LLMs you are using for your POC.
I've built a lightweight package that provides a standard interface to the LLM providers, as well as taking care of boilerplate around structured outputs, function calling, and opentelemetry/tracing. It's hopefully a good compromise between ease-of-use and complexity.
https://github.com/jackmpcollins/magentic
https://github.com/jackmpcollins/magentic
Langchain has the odd nice component I use, I prefer their recursive text splitter to anything else I've seen or cobbled together, but I don't find the code readable or the abstractions particularly intuitive.
Langchain is a spaghetti rabbit hole. In our consultancy, we've basically switched entirely to Haystack, and keep the abstractions fairly light, really only use it to give the pipelines some structure that applies across projects. Very easy to add custom components with pretty straightforward interfaces. The way you assemble the components isn't everyone's cup of tea (string references), but it does make it very easy to understand at a glance what each pipeline is doing and when. Worth checking out if Langchain isn't doing it for you.
Last I checked Langchain was just basically a pile of helpers? There isn't really a coherent model of anything that's presented to the user, just some light wrappers around stuff. I played around for a bit, it was fine, but I opted to write my own helpers in the end.
Did I miss out on some major developments there? Because I don't see why it's a thing that's being talked about everywhere, when it's barely anything.
Did I miss out on some major developments there? Because I don't see why it's a thing that's being talked about everywhere, when it's barely anything.
Langchain is not just basically a pile of helpers. It is *just basically a pile of helpers with 35M in investment*.
https://venturebeat.com/ai/langchain-lands-25m-round-launche...
https://venturebeat.com/ai/langchain-lands-25m-round-launche...
Just in case anyone comes back to see this: I found the following (3 hour!!) YouTube video to be really helpful. It starts slow, but skip ahead and watch in 2x until you get to what you need.
https://youtu.be/yF9kGESAi3M?feature=shared
https://youtu.be/yF9kGESAi3M?feature=shared
Howdy! Erick from LangChain here. Would love to learn a bit more about your bug hunting process and see how we can improve the debugging process for everyone else! Could you email me at [email protected]?
Erick, please just read this entire thread.
Langchain is frequently crapped on here on HN. Do some searching and you'll see it's pretty much a universal sentiment that langchain is terrible and you shouldn't use it.
The reason I use pinecone without langchain and people call me crazy
I don't know if this will help you, but have you tried langroid?
Even if you don't want to use agents, it is still useful as a convenient library for calling an Open AI compatible endpoint.
https://langroid.github.io/langroid/quick-start/llm-interact...
Even if you don't want to use agents, it is still useful as a convenient library for calling an Open AI compatible endpoint.
https://langroid.github.io/langroid/quick-start/llm-interact...
Thanks for mentioning Langroid[1], I am the lead architect/dev. We are starting to see companies use it in production after evaluating alternative frameworks, whether agent-oriented or not.
We started building Langroid in early 2023 after finding existing frameworks lacking in terms of good dev-ex, extensibility and clarity of code. We prioritize code transparency, flexibility, stability, good test coverage. We designed it to be agent-oriented from the start, with an elegant agent orchestration mechanism loosely inspired by various paradigms such as Blackboard Architectures, Actor Model, and Production systems, and process calculi. But as you said, Langroid is useful even if you want to just use a single LLM conversation-state.
[1] https://github.com/langroid/langroid
We started building Langroid in early 2023 after finding existing frameworks lacking in terms of good dev-ex, extensibility and clarity of code. We prioritize code transparency, flexibility, stability, good test coverage. We designed it to be agent-oriented from the start, with an elegant agent orchestration mechanism loosely inspired by various paradigms such as Blackboard Architectures, Actor Model, and Production systems, and process calculi. But as you said, Langroid is useful even if you want to just use a single LLM conversation-state.
[1] https://github.com/langroid/langroid
Yeah, it's a meme by now. Langchain did not grow well with the new changes in AI. Generally avoid it where possible. It still has uses, but treat it as legacy code.
Ugh, I feel exactly the same way. In fact, I've spent the last 2 weeks refactoring our entire LLM pipeline to completely root out LangChain. I'm just implementing basic RAG from scratch, which 1) has me learning a lot more quickly and 2) feeling much more in control of the entire pipeline.
How are you running the pipeline exactly? I'm new to AI dev and when I hear pipeline I think either CI/CD or something like Apache Airflow or Kestra.
Remember than LangChain raised $10m a few months ago https://blog.langchain.dev/announcing-our-10m-seed-round-led... (wait, over a year ago!)
I tried using it for a simple POC and was very frustrated with it. Now we just use raw api calls, and a state machine to create our dag. Much simpler and very extensible.
Get rid of it, Langchain really doesn't do anything you can't easily do yourself.
I get the idea that things should be abstracted away and that's the point of a library but this feels like a little much.