Show HN: Bumpgen – upgrade NPM packages using AI(github.com)
github.com
Show HN: Bumpgen – upgrade NPM packages using AI
https://github.com/xeol-io/bumpgen
9 comments
This is the kind of gen AI that I can really get behind. Great repo and easy to understand with the attached video. Is such a thing possible for Python? Or does the AST parser and code planner work best with static typing?
Python is trickier because of the lack of strong typing support by default in Python. So we're left with projects using strict mypy or with very extensive test suites for validation.
It's possible, just probably not the next language to tackle. If you'd like Python support though, you can upvote this github issue: https://github.com/xeol-io/bumpgen/issues/61
It's possible, just probably not the next language to tackle. If you'd like Python support though, you can upvote this github issue: https://github.com/xeol-io/bumpgen/issues/61
I absolutely love this idea and how clever the implementation is. Thanks for sharing the details.
Appreciate the kind words!
Great product!
Just curious where you guys feel Rust sits in your language support priority?
Just curious where you guys feel Rust sits in your language support priority?
Good new is most strongly typed languages work well with our current architecture. Bad news is Rust is not currently on the roadmap but that I can see that changing depending on how popular this demand is.
Nice, would you guys consider open sourcing the adaptor portion of the product so others can contribute?
Whole thing is open source and will be, we absolutely welcome any contributions!
this is pretty cool!
There were some interesting challenges we encountered using an LLM to fix breaking changes:
[1] Finding the breaking changes → know how is a dependency used through the codebase
[2] Knowing how to fix the a breaking change → know how the dependency has changed from one version to another
[3] Understanding how the fix has modified existing behavior → know how the function the dependency is used in is used through the codebase
We addressed challenge [1] and [2] by choosing to support Typescript first. The strong typing helps us identify the breaking changes with a version bump pretty well. We can then pass the type errors into the LLM to increase its fix accuracy.
Challenge [3] is definitely the trickiest. We built an AST of the codebase that we then use to create a “plan graph” (https://huggingface.co/papers/2309.12499) of the type error, the function it’s used in, and the functions calling it. This plan graph should tell us how a fix would need to be perpetuated throughout the codebase.
At a high level bumpgen’s core loop is something like this:
- build an AST to understand your code’s relationships
- bump version and get type errors
- combine the above to create a plan graph (https://huggingface.co/papers/2309.12499) of how to fix the error and apply it to the rest of the codebase
- prompt the LLM to step through the plan graph and fix each breaking change
- validate the fixes with a rebuild
We also built our own benchmark suite (https://github.com/xeol-io/swe-bump-bench) to test bumpgen’s accuracy. It is a set of repos with human commits for version bumps. We would run bumpgen on the prior commit then compare bumpgen’s PR to that of the human commit to determine success. Our latest benchmark sits around ~50% accuracy.
Up next we want to better address challenge [2] by building embeddings for different dependency versions to give the LLM more context on the changes across versions. After that we will be releasing a GitHub app with some qol features such as configuring update cadences, etc before moving onto C# and Golang support.
We covered a lot of our architecture and thought process for bumpgen, we would love to hear your thoughts and feedback in the comments!