How to build a JavaScript package manager(yarnpkg.com)
yarnpkg.com
How to build a JavaScript package manager
https://yarnpkg.com/blog/2017/07/11/lets-dev-a-package-manager/
17 comments
It was making a lot of sense until the part about recursive dependencies. To my mind, once this happens it becomes a dependency graph instead of a tree, and instead of trying to satisfy dependencies "locally" I would work out all the constraints and then solve them. The "choose highest, check parent" algorithm seems to me like it would make results depend on which package in a cycle you tried to resolve first, and apparently with peerdependencies it sometimes is incorrect (https://github.com/yarnpkg/yarn/issues/422). Any insight as to why it was designed this way? Just the NP-completeness?
I've made some early work on using constraints when I started working on Yarn, but it ended up unpractical because of the sheer execution time it required. A huge number of constraints were required to satisfy the requirements, which was made even worse when you consider that a dependency version might have different sub-dependencies than its other versions.
That being said, I'm not an expert in SAT logic, and it's quite possible a better solution is possible! Maybe some kind of hybrid algorithm, possibly? In fact, one thing I'd really like to do in medium term would be to externalize the resolver part of Yarn - this way, it would be much easier to experiment with it to try to find algorithms that fits various requirements. If you want correctness you would use a slow but comprehensive algorithm, if you want speed you would use a naive algorithm like the one in the article, etc.
That being said, I'm not an expert in SAT logic, and it's quite possible a better solution is possible! Maybe some kind of hybrid algorithm, possibly? In fact, one thing I'd really like to do in medium term would be to externalize the resolver part of Yarn - this way, it would be much easier to experiment with it to try to find algorithms that fits various requirements. If you want correctness you would use a slow but comprehensive algorithm, if you want speed you would use a naive algorithm like the one in the article, etc.
Interesting, which packages did you try running it on for which it took too much execution time? Where was most of the time spent?
I unfortunately don't remember - possibly a combo of Babel + Webpack and their plugins, tho, they're usually my go-to choices to test install perfs.
Now that I think about it, it's quite possible that the process was apparently hanging on the same issue I describe in my article, where babel-core depends and babel-cli and vice versa. Maybe it would work better if I were to run the process a first time with a simple algorithm like the one exposed in the article, that would clear up any dependency loop, then a second more complex pass that wouldn't have to deal with this.
Now that I think about it, it's quite possible that the process was apparently hanging on the same issue I describe in my article, where babel-core depends and babel-cli and vice versa. Maybe it would work better if I were to run the process a first time with a simple algorithm like the one exposed in the article, that would clear up any dependency loop, then a second more complex pass that wouldn't have to deal with this.
Love this kind of format as a way to explain how Yarn works. Awesome article!
Oh no. Not just another package manager ... a package manager generator... runs :-)
Yet another package manager except it teaches you how to write one. Nice demo with explaining titles. Too bad I don't know JS and honestly I don't think the world needs another package manager just like it doesn't need another single page web app to read Hello World. But the post definitely gives a bird's eye view of things.
Author here! Our goal in this article wasn't to build another package manager (we've got our hands already full with Yarn :), but rather to give JavaScript developers some insight as to what's going on under the hood every time they install their dependencies.
We hope it will contribute to demystify package managers, and give people plenty of ideas to improve their favorite ones - Yarn is a community project, and we're always happy to see new contributors step in!
We hope it will contribute to demystify package managers, and give people plenty of ideas to improve their favorite ones - Yarn is a community project, and we're always happy to see new contributors step in!
>> (we've got our hands already full with Yarn :)
(we've got our hands already full with Yarn :\))
(we've got our hands already full with Yarn :\))
I don't understand these sorts of comments. The world probably doesn't need a lot of different things--another video game, another search engine, another social network--but strict need is not why we do things.
Why was it so important for you to leave this sort of dismissive comment?
Why was it so important for you to leave this sort of dismissive comment?
Agree with the parent. Regarding yarn, it was created because npm started showing weaknesses in many things, with large and complex dependency trees: install was slow, sometimes unreliable and not cross-platform portable, and the codebase was difficult to refactor due to legacy code and backward compatibility, so it made more sense to create a new package manager without that burden and try to iteratively align with npm to become a drop-in replacement.
It's easy to fall into the fatigue due to new languages, tools etc. coming out each week but please think twice before posting dismissive comments like that.
I observe a lot of "I don't read HN anymore" sentiment lately among top tech Twitter users, partially due to that.
There's a lot of awesome stuff being given out for free and people are whining. (Yes, I'm a bit tired of constantly following and evaluating things, but I keep it for myself.)
The situation came to the point where some guys are thinking about not opensourcing their tools anymore due to constant criticism and maintenance fatigue. Read the two links below (accidentally, the first one is from yarn co-author!):
https://twitter.com/sebmck/status/879282797915119616
https://nolanlawson.com/2017/03/05/what-it-feels-like-to-be-...
It's easy to fall into the fatigue due to new languages, tools etc. coming out each week but please think twice before posting dismissive comments like that.
I observe a lot of "I don't read HN anymore" sentiment lately among top tech Twitter users, partially due to that.
There's a lot of awesome stuff being given out for free and people are whining. (Yes, I'm a bit tired of constantly following and evaluating things, but I keep it for myself.)
The situation came to the point where some guys are thinking about not opensourcing their tools anymore due to constant criticism and maintenance fatigue. Read the two links below (accidentally, the first one is from yarn co-author!):
https://twitter.com/sebmck/status/879282797915119616
https://nolanlawson.com/2017/03/05/what-it-feels-like-to-be-...
I've seriously thought about it myself, shutting off and going back to closed software. I've had my software praised one week as innovative and awesome, to the next week being "yet another example of what is wrong with these young developers and their JavaScript" (I'm 35, I've been programming for 20 years). All here on HN. I won't even touch Reddit anymore.
Have fun writing it and using it and/or make money from it.
Nothing else really matters...
Nothing else really matters...
Exactly. Most CS curriculums have an OS and compiler course where you write one. Does the world need each student to write a new compiler or OS? No, but writing a toy version is incredibly helpful in learning how one works when you interact with it. By looking at the article title, then the url, I instantly knew it was about going over the internals instead of advocating for more package managers.
What's sad is that, from the title, I knew there'd be HN comments whining about "Ugh, why another package manager?"