Npm v5.0.0 released(blog.npmjs.org)
blog.npmjs.org
Npm v5.0.0 released
http://blog.npmjs.org/post/161081169345/v500
116 comments
When io.js merged back into node.js the following happened:
1. Node.js was abandoned and replaced with io.js.
2. Io.js was relabelled as node.js.
3. The node.js project was put under open governance via the creation of the Node Foundation.
This doesn't really compare to npm:
* The npm-cli's name is using npm Inc's trademark: giving it away would leave the company with no name or create unwanted ambiguity between npm Inc the company and npm-cli the (then) community-owned open source project unaffiliated with the company.
* Yarn does not share any source code directly with npm-cli, it's a complete reimplementation of a similar feature set. It's currently backed by a mirror of the npm registry but that's the extent of the overlap.
* The Node Foundation already exists and Yarn is already owned by the community. I could see Yarn joining the Node Foundation and the Node Foundation replacing its bundled dependency on the commercially owned npm-cli but even then npm Inc has nothing to contribute to the process.
Personally I would much rather see Yarn join the JavaScript Foundation because it speaks to the broader appeal of Yarn outside the traditional "Node community" and the politics involved in the latter and its close ties to npm Inc.
FWIW I consider the close ties between the Node project and npm Inc nothing more than a historical accident that has resulted in a conflict of interest that is overdue to be resolved by dropping npm-cli from the official releases.
Now that yarn is no longer distributed using npm (although it's still possible to install it that way) that seems more realistic than ever.
1. Node.js was abandoned and replaced with io.js.
2. Io.js was relabelled as node.js.
3. The node.js project was put under open governance via the creation of the Node Foundation.
This doesn't really compare to npm:
* The npm-cli's name is using npm Inc's trademark: giving it away would leave the company with no name or create unwanted ambiguity between npm Inc the company and npm-cli the (then) community-owned open source project unaffiliated with the company.
* Yarn does not share any source code directly with npm-cli, it's a complete reimplementation of a similar feature set. It's currently backed by a mirror of the npm registry but that's the extent of the overlap.
* The Node Foundation already exists and Yarn is already owned by the community. I could see Yarn joining the Node Foundation and the Node Foundation replacing its bundled dependency on the commercially owned npm-cli but even then npm Inc has nothing to contribute to the process.
Personally I would much rather see Yarn join the JavaScript Foundation because it speaks to the broader appeal of Yarn outside the traditional "Node community" and the politics involved in the latter and its close ties to npm Inc.
FWIW I consider the close ties between the Node project and npm Inc nothing more than a historical accident that has resulted in a conflict of interest that is overdue to be resolved by dropping npm-cli from the official releases.
Now that yarn is no longer distributed using npm (although it's still possible to install it that way) that seems more realistic than ever.
What replaces the npm registry, though? That's the reason that npm Inc. exists, not just stewardship of the CLI.
A decentralized repository of packages that anyone can help out maintain. Not entirely sure how, but I think that would be the ideal scenario.
Back when the npm registry was really struggling with reliability (I believe because it was secondary to profit for Joyent) I was saying we needed a decentralized registry solution.
But right when that got some upvotes on r/node, within a few days isaacs had npm inc. going. And within a few weeks or months, the npm registry was pretty rock solid. And I have not had a single registry issue in forever.
And npm 4 works well, and I'm sure npm@5 works even better.
I still think it makes sense to have a decentralized repository and I guess its nice to have a non-commercial alternative to npm. But for me its not important anymore because npm is working really great now.
But right when that got some upvotes on r/node, within a few days isaacs had npm inc. going. And within a few weeks or months, the npm registry was pretty rock solid. And I have not had a single registry issue in forever.
And npm 4 works well, and I'm sure npm@5 works even better.
I still think it makes sense to have a decentralized repository and I guess its nice to have a non-commercial alternative to npm. But for me its not important anymore because npm is working really great now.
That's great to hear!
Reliability sure was tough in those days. npm wasn't secondary or tertiary or any-ary to Joyent. It was my nights and weekends project the whole time I was there, and IrisCouch generously donated infrastructure, which we pushed to its very limit once Nodejitsu acquired them.
Reliability sure was tough in those days. npm wasn't secondary or tertiary or any-ary to Joyent. It was my nights and weekends project the whole time I was there, and IrisCouch generously donated infrastructure, which we pushed to its very limit once Nodejitsu acquired them.
Something like IPFS could perhaps be used for this. I think the biggest issue is probably access control. I.e. just releasing packages addressed by content isn't enough, because people generally depend on @whatever/left-pad and not 5b055a0b42f1c or whatever the hash may be. Real estate in a global name space means someone gets there first, so you'll need to consider how to deal with name squatters and the likes. With a centralized source like npm it's as easy as getting in touch with support (they're very good) but when no one knows the name space everyone owns the name space – makes it hard to make things "nice".
So I guess what I'm saying is that just storage isn't the tricky bit to distribute really, but having a global namespace is I guess.
So I guess what I'm saying is that just storage isn't the tricky bit to distribute really, but having a global namespace is I guess.
well, you could depend on left-pad@5b055a0b42f1c, and the name doesn't really matter. Normally you would include that as require('left-pad') but if you have multiple versions, I don't see why require('left-pad@5b055a0b42f1c') couldn't be used. There is a tool for doing this in Golang already[0] and I've also done some experiments to get this to work in JavaScript[1]
- [0] https://github.com/whyrusleeping/gx
- [1] http://everythingstays.com
- [0] https://github.com/whyrusleeping/gx
- [1] http://everythingstays.com
Go does something like this, by referencing packages through Git url. Of course, 99% of packages you install are through Github, but that's not Go's fault. It has downsides, but its really a great and extensible system.
You can do the same thing with npm and yarn as well by referencing the git+https URL and a tag/branch/commit instead of a version number.
Yarn does not run a mirror of the registry. registry.yarnpkg.com is a pass-through domain to the npm registry. It allows them to collect stats about yarn usage but is not a mirror.
>resolved by dropping npm-cli from the official releases.
But that would break 7 years of documentation and tutorials, likely bad for newcomers and thus the ecosystem
But that would break 7 years of documentation and tutorials, likely bad for newcomers and thus the ecosystem
so npm has created a huge vendor lock-in and we should live with it?!
As with any vendor lock-in situation, decisions should be made based on cost/benefit
Perhaps you can help come up with a strategy that gracefully handles breakages and eventually results in dropping npm-cli
Perhaps you can help come up with a strategy that gracefully handles breakages and eventually results in dropping npm-cli
Not to mention quite a few CI build pipelines.
FWIW it's trivial to install yarn already: https://yarnpkg.com/en/docs/install
NPM currently doesn't provide a standalone distribution as far as I can tell, but presumably they would offer one if they no longer had the luxury of being bundled with Node.
Since changes generally don't happen overnight I would expect a transitional period where Node still bundles npm-cli but npm Inc has the time to prepare a standalone distribution before yarn replaces npm-cli. Additionally downstream channels could decide to provide legacy packages containing both.
NPM currently doesn't provide a standalone distribution as far as I can tell, but presumably they would offer one if they no longer had the luxury of being bundled with Node.
Since changes generally don't happen overnight I would expect a transitional period where Node still bundles npm-cli but npm Inc has the time to prepare a standalone distribution before yarn replaces npm-cli. Additionally downstream channels could decide to provide legacy packages containing both.
They probably need to do something like docker did.
Answer 1: Yes but they're not exactly analogous and I don't see them merging. Yarn is a complete rewrite, and the compatibility "surface area" is much lower than Node itself so there's not much pressure for them to merge. Yarn has been working essentially flawlessly for me, so npm-cli is really going to have to surpass yarn in some way for me to switch back.
Answer 2: I don't use global packages often. For binaries I generally add a "scripts" entry to package.json for common tasks, which automatically adds "./node_modules/.bin" to the PATH. For one-off tasks I just run "./node_modules/.bin/executable"
Answer 2: I don't use global packages often. For binaries I generally add a "scripts" entry to package.json for common tasks, which automatically adds "./node_modules/.bin" to the PATH. For one-off tasks I just run "./node_modules/.bin/executable"
Git dependencies with semver support seems to pretty much leapfrog yarn. One of the few reasons for a private npm package is removed with this feature.
It's more accurate to say that npm v5 : npm :: io.js : node.
Same team, performing a big refactoring and significant improvement and modernization of some bits that were outdated and difficult to improve without a controlled demolition and rebuild.
The analogy breaks down, of course, because the governance didn't change, the project's relationship to its corporate backer wasn't standing in the way of progress, etc. (Except, I guess, that it might have gone faster if we'd had more money to hire more devs? But some of this was just a slog through a very old codebase with a lot of scar tissue, and it's hard to speed that up by putting more hands on it.)
Everything in npm 5 was literally planned years in advance. When we have this many people depending on a thing, we have to be careful about how we make drastic changes. Yarn was a strong signal from the community that we were on the right track, but it only seems like a "catalyst" when seen from the outside. Correlation is not causation, even when it lands first.
Same team, performing a big refactoring and significant improvement and modernization of some bits that were outdated and difficult to improve without a controlled demolition and rebuild.
The analogy breaks down, of course, because the governance didn't change, the project's relationship to its corporate backer wasn't standing in the way of progress, etc. (Except, I guess, that it might have gone faster if we'd had more money to hire more devs? But some of this was just a slog through a very old codebase with a lot of scar tissue, and it's hard to speed that up by putting more hands on it.)
Everything in npm 5 was literally planned years in advance. When we have this many people depending on a thing, we have to be careful about how we make drastic changes. Yarn was a strong signal from the community that we were on the right track, but it only seems like a "catalyst" when seen from the outside. Correlation is not causation, even when it lands first.
Thanks Isaac! Both for this insight and - more so - for your massive contributions to the community. Slainte! :)
"Deterministic builds by default are such an obvious (in retrospect) core requirement."
I agree... I like the theoretical concept that our dependencies, could be automatically managed/upgraded behind the scenes via a declarative versioning system, like semver. That, in my view, was the paradigm behind not having defaulted to dependency graph snapshot (lock/shrinkwrap) by default. In practice, people don't follow semver correctly and autoupdating patch+minor versions can cause you a lot of pain. It would be cool to make versioning more tied to actual API signature, so that changes to the signature could not actually be published as minor/patch changes... but would be forced to use a major version bump in publish. I don't know how this would work particularly in a language like javacscript w/out a typed interface... but I'm just theorizing...
I agree... I like the theoretical concept that our dependencies, could be automatically managed/upgraded behind the scenes via a declarative versioning system, like semver. That, in my view, was the paradigm behind not having defaulted to dependency graph snapshot (lock/shrinkwrap) by default. In practice, people don't follow semver correctly and autoupdating patch+minor versions can cause you a lot of pain. It would be cool to make versioning more tied to actual API signature, so that changes to the signature could not actually be published as minor/patch changes... but would be forced to use a major version bump in publish. I don't know how this would work particularly in a language like javacscript w/out a typed interface... but I'm just theorizing...
As @pluma mentioned, a "merge" seems unlikely (for one thing they don't share much, if any, code).
However, npm could certainly compete with more of yarn's features (fast local cache[0], shorter syntax for running scripts like `npm start`, etc)
[0] turns out they did that: https://twitter.com/maybekatz/status/865393382260056064 wow!
However, npm could certainly compete with more of yarn's features (fast local cache[0], shorter syntax for running scripts like `npm start`, etc)
[0] turns out they did that: https://twitter.com/maybekatz/status/865393382260056064 wow!
Yeah, I think that's a good comparison, except the upsides of yarn seemed much more obvious to the community than was the case with IOJS. There seemed a lot less controversy than with IOJS.
What a time those days were, feels like a million years ago already.
What a time those days were, feels like a million years ago already.
npm would need to add the --flat option for it to work for me
> Thank you, yarn, for helping the community see the naked emperor. Deterministic builds by default are such an obvious (in retrospect) core requirement.
https://docs.npmjs.com/cli/shrinkwrap provides deterministic builds and has been around far longer than yarn. Since Oct 2014 npm v3 started automatically updating shrinkwrap whenever '--save' was used. See https://github.com/npm/npm/pull/4918#issuecomment-61344871
Calling npm 'the naked emperor' for not implementing something it did, infact, implement is uncalled for.
Edit: added links in response to unexplained downmods. The parent is simply, provably wrong.
https://docs.npmjs.com/cli/shrinkwrap provides deterministic builds and has been around far longer than yarn. Since Oct 2014 npm v3 started automatically updating shrinkwrap whenever '--save' was used. See https://github.com/npm/npm/pull/4918#issuecomment-61344871
Calling npm 'the naked emperor' for not implementing something it did, infact, implement is uncalled for.
Edit: added links in response to unexplained downmods. The parent is simply, provably wrong.
Even with shrinkwrap, npm install is not deterministic, install order still matters. To get a deterministic install, you need to use shrinkwrap and do an `rm -rf node_modules` before every install. And it's still not completely deterministic if any dependency or sub-dependency has optionalDependencies.
https://docs.npmjs.com/how-npm-works/npm3-nondet
https://docs.npmjs.com/how-npm-works/npm3-nondet
That, and the fact that it's not the default behavior in npm gives yarn a pretty big advantage in my opinion. Especially for devs who are newer to Node.js.
I've read the https://docs.npmjs.com/how-npm-works/npm3-nondet but it make no mention of shrinkwrap. Since shrinkwrap captures versions for the entire tree, install order wouldn't matter.
I personally had invalid dependencies generated with shrinkwrap which did not work with a "npm install" after and had to edit the file manually so I would say it's not yet there.
Shrinkwrap is a partially successful attempt to solve the problem, but still has obvious flaws. For example, instead of the relatively neat version specifications normally used in package.json, with a shrinkwrapped project you wind up with https://registry.npmjs.org/... paths instead, but only sometimes. This gets messy if you want to both install stable versions of all your packages and use a local registry to supply them for reliability, which is not an unlikely combination.
We dropped npm in favour of yarn a while back, but I imagine we'll switch back once the stable release is out. Our needs are perhaps simpler than most, but the key issues we were having with npm at the time were directly solved by yarn:
1. Dependency install was taking too long.
2. Inconsistent builds between devs because of no lock file.
3. Could not search the registry from the command line in a timely manner.
Isn't npm shrinkwrap the npm "lockfile"?
Kinda. Shrinkwrap has been around for a while.
The new lockfile is basically shrinkwrap 2.0 using what's been learned since
The new lockfile is basically shrinkwrap 2.0 using what's been learned since
Just tried on a couple of projects with a lot of dependencies, we moved to yarn a while ago due to performance issues and it seems to be resolved.
On cold cache: Yarn: 20.94 seconds NPM5: 21.11 seconds
With cache: Yarn: 10.35 seconds NPM5: 15.20 seconds
For some reason, when node_modules folder is still there, yarn exits in a couple hundres milliseconds but npm5 does something for around 5 seconds.
Haven't checked lock file / installation consistency stuff. Yarn has been great on that too so we have no intention to go back but this is a decent release.
On cold cache: Yarn: 20.94 seconds NPM5: 21.11 seconds
With cache: Yarn: 10.35 seconds NPM5: 15.20 seconds
For some reason, when node_modules folder is still there, yarn exits in a couple hundres milliseconds but npm5 does something for around 5 seconds.
Haven't checked lock file / installation consistency stuff. Yarn has been great on that too so we have no intention to go back but this is a decent release.
Also ran some numbers here (with warm cache. node 6.10.3).
Competition is wonderful though. I'll check npm again in 6-12 months and see what the community thinks of it before switching again.
* yarn: 25s
* npm@5: 28s
* npm@4: 63s
* npm@3: 68s
Competition is wonderful though. I'll check npm again in 6-12 months and see what the community thinks of it before switching again.
amen
If I recall correctly, npm5 with an existing node_modules folder still goes out to the internet and checks some data whereas yarn does not.
Yup, tho it's configurable and you can set --prefer-offline to true to avoid it
I'd be curious to see the warmed cache with --prefer-offline as I'm betting that's probably the time suck there.
my test:
npm@5 28.274s vs 11.60s Yarn
@Windows10, SSD, warn cache
npm@5 28.274s vs 11.60s Yarn
@Windows10, SSD, warn cache
So happy with the --save by default. Someone at work kept installing new dependencies without save (they didn't knew about it, somehow). We then had an unusable package.json. I had to manually find directories in node_modules and install them on production -_-.
You should implement basic ci with travis et al. That commit should never be able to make it into master. If you are using Github you can force green tests before a pull request is mergeable.
And I think this is a complete bs. I have to try the module and then make a conscious decision to use (save) it, not kind of save it first and then hope that it actually does what I want.
Like wtf, who thought that saving something you download maybe for the first time as a dependency is a good idea?
Like wtf, who thought that saving something you download maybe for the first time as a dependency is a good idea?
It is perfectly reasonable and avoids mistakes. It's better to have a module that's unused than to break your whole build.
If you want to try something install it and then remove it. That's the more conscious decision. I don't want to have to remember to save the module if I end up liking it, but I will remember to remove it when tearing it down from the codebase, and if I don't, nothing bad happens.
If you want to try something install it and then remove it. That's the more conscious decision. I don't want to have to remember to save the module if I end up liking it, but I will remember to remove it when tearing it down from the codebase, and if I don't, nothing bad happens.
Is it that hard to `npm uninstall` if you decide you don't want it?
Most of the time when I do npm install it's for stuff I know I will need. If you want to check something out you can always just --no-save it.
Does uninstall default to --save too? Because if so, then your workflow doesn't even have to change gear. You just install it to check it out, you like it then do nothing, you don't like it then you can just `npm uninstall`, something you'd hopefully do anyway.
First time I hear about '--save' for uninstall, but it doesn't make a difference in my case because I always install many things, test them all at once, then '--save' what I need and 'rm -rf' the 'node_modules'. After that, 'npm i' gives me everything I need.
I still think this change is beyond stupid.
I still think this change is beyond stupid.
When you commit your changes, it's similarly easy to inspect package.json changes, and only commit the lines you want [1].
This change increases the chance that what you have working locally is reproducible from "source". IMHO that's a very important goal, while debate over convenience of omitting --save vs --no-save is superficial. Obviously there are 2 groups and one will be unhappy either way, but does that justify calling it "beyond stupid"? (I could maybe see that if you had actual data that 90% people want --no-save...)
Also, seems you can config old default by `npm config set save false` (with the risk one day you'll work on another machine and be surprised by uncustomized defaults). https://twitter.com/maybekatz/status/859193277894991872
Docs are lacking, following up on https://github.com/npm/npm/issues/5108
[1] If your commit flow doesn't make this a pleasant experience — e.g. command-line git IMHO doesn't — you deserve better tools. I can recommend `git citool` for start — old and ugly UI, but fast, portable, has keyboard shortcuts, also very convenient for amending.
This change increases the chance that what you have working locally is reproducible from "source". IMHO that's a very important goal, while debate over convenience of omitting --save vs --no-save is superficial. Obviously there are 2 groups and one will be unhappy either way, but does that justify calling it "beyond stupid"? (I could maybe see that if you had actual data that 90% people want --no-save...)
Also, seems you can config old default by `npm config set save false` (with the risk one day you'll work on another machine and be surprised by uncustomized defaults). https://twitter.com/maybekatz/status/859193277894991872
Docs are lacking, following up on https://github.com/npm/npm/issues/5108
[1] If your commit flow doesn't make this a pleasant experience — e.g. command-line git IMHO doesn't — you deserve better tools. I can recommend `git citool` for start — old and ugly UI, but fast, portable, has keyboard shortcuts, also very convenient for amending.
If you're on Windows and want to give this thing a test drive, use `npm-windows-upgrade`:
Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force
npm i -g npm-windows-upgrade
npm-windows-upgrade --npm-version latest
Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force
npm i -g npm-windows-upgrade
npm-windows-upgrade --npm-version latest
> A new, standardised lockfile feature meant for cross-package-manager compatibility (package-lock.json)
Surely it would be much better to follow standard lock file naming conventions and name the file package.lock
Surely it would be much better to follow standard lock file naming conventions and name the file package.lock
I'd rather my editor recognize a .json file and treat as such. I don't have a .lock convention
Or even package.lock.json
I had originally been for package.lock, but after reading right through the issue I now agree with your excellent suggestion of package.lock.json.
Rather than just talk here, I've posted against the issue suggesting this approach basically asking for up/down votes. https://github.com/npm/npm/pull/16441#issuecomment-304458746
Rather than just talk here, I've posted against the issue suggesting this approach basically asking for up/down votes. https://github.com/npm/npm/pull/16441#issuecomment-304458746
It bothers me somewhat that they say this is standardized, but not by whom. Is the idea that Yarn should fall in line or is it a case of https://xkcd.com/927/ ?
[deleted]
I still want npm-search improved.
I want to be able to search by tag, constrained by license and with packages that use dependencies I already have installed floated to the top. There are so many ways this could be nicer.
I want to be able to search by tag, constrained by license and with packages that use dependencies I already have installed floated to the top. There are so many ways this could be nicer.
Its still not as fast as yarn FYI.
Yarn doesn't support all of npm's features. At least not as of a couple months ago when it 1) didn't support one of our needs at all, and 2) broke a couple NPM packages on install. This may or may not be part of why it's faster: it does less.
what are these npm's features that yarn doesn't support?
I do support and these issues come up all the time with yarn:
- native packages are not well-supported by yarn yet, notably node-sass since it's so widely-used
- no private module support
- native packages are not well-supported by yarn yet, notably node-sass since it's so widely-used
- no private module support
I also had a weird issue where for certain packages yarn would pick up a version which was older than what npm would. Not sure if this had anything to do with the fact that we have an internal npm mirror (that being said, yarn claims it picks up the .npmrc information, so even if it did, that would be a yarn bug).
In addition, yarn seems to increasingly be prioritizing its own default configs over what may be saved in my ~/.npmrc. Had several proxy issues after upgrading to a newer version of yarn that I didn't have earlier.
The reason I switched to yarn wholesale was because my existing npm based infrastructure worked seamlessly (at least to the extent that I was using npm features). But I've noticed that newer versions of yarn seem to have regressed in terms of npmrc support, which is a huge problem.
In addition, yarn seems to increasingly be prioritizing its own default configs over what may be saved in my ~/.npmrc. Had several proxy issues after upgrading to a newer version of yarn that I didn't have earlier.
The reason I switched to yarn wholesale was because my existing npm based infrastructure worked seamlessly (at least to the extent that I was using npm features). But I've noticed that newer versions of yarn seem to have regressed in terms of npmrc support, which is a huge problem.
A look through their issues on Github will reveal quite a few "npm supports this feature, yarn doesn't, plz add" entries. There are even more "this works in npm but not yarn, and I'm not sure why", many of which are also effectively a result of features/cases that npm supports and yarn doesn't. I don't have an exhaustive list or anything, but it's right there to see if you're interested enough to dig for a minute or two. It's pretty clearly not a drop-in replacement for npm—at least not yet.
My experience a couple months ago was that almost anything slightly off the path of the most basic use case rapidly entered "here be dragons" territory, such that every problem became "are we doing something wrong, or is yarn broken again?", which wasn't worth any benefits it provided. If you're just installing packages from NPM, and those packages don't do anything even slightly weird, it's probably fine.
[EDIT] a running theme of the issues is that a lot of correctness-checking and edge-case handling is missing or incomplete, in addition to some "strange" npm features not being supported. Anyway, here are a couple examples of what I'm talking about:
https://github.com/yarnpkg/yarn/issues/3433
https://github.com/yarnpkg/yarn/issues/3507
https://github.com/yarnpkg/yarn/issues/2090
(last one has a lovely "why are you doing this?" as its first response, from the repo owner for bonus LULZ)
FAR from the only ones. IIRC one of our issues had something to do with Yarn totally rewriting the package.json of a package installed from a local directory, removing the "main" entry entirely, which broke it (obviously). Maybe it'll be something I use later, but right now I don't really want to have to wonder whether any given problem is due to some obscure breakage in my package manager when I can... just use npm and not worry about it.
My experience a couple months ago was that almost anything slightly off the path of the most basic use case rapidly entered "here be dragons" territory, such that every problem became "are we doing something wrong, or is yarn broken again?", which wasn't worth any benefits it provided. If you're just installing packages from NPM, and those packages don't do anything even slightly weird, it's probably fine.
[EDIT] a running theme of the issues is that a lot of correctness-checking and edge-case handling is missing or incomplete, in addition to some "strange" npm features not being supported. Anyway, here are a couple examples of what I'm talking about:
https://github.com/yarnpkg/yarn/issues/3433
https://github.com/yarnpkg/yarn/issues/3507
https://github.com/yarnpkg/yarn/issues/2090
(last one has a lovely "why are you doing this?" as its first response, from the repo owner for bonus LULZ)
FAR from the only ones. IIRC one of our issues had something to do with Yarn totally rewriting the package.json of a package installed from a local directory, removing the "main" entry entirely, which broke it (obviously). Maybe it'll be something I use later, but right now I don't really want to have to wonder whether any given problem is due to some obscure breakage in my package manager when I can... just use npm and not worry about it.
This might be a totally stupid question, but what does that mean?
Edit: nevermind, I found it --> https://github.com/npm/npm/releases this is the 'prerelease' release.
npm install npm@latest -g
/usr/local/bin/npm -> /usr/local/lib/node_modules/npm/bin/npm-cli.js
/usr/local/lib
└── [email protected]
Edit: nevermind, I found it --> https://github.com/npm/npm/releases this is the 'prerelease' release.
Right now latest still points to v4. You can install 5 with:
npm install -g npm@5
npm -g ls npm
+-- [email protected] npm install npm@next -g
This seems to be very similar to Gradle vs Buck (or Bazel).
Generally speaking the main tool is slow, or otherwise suboptimal but does a lot. A different tool is built with an emphasis on speed. Eventually, the main stream tools picks up the speed features of the other tool and then "wins" because it does more. (Side note: I have no idea when Gradle came out in comparison to Buck/Bazel) For context, Gradle in the 4.0 version is starting to really support build cache support.
Edit: this comment is a reaction to all the other comments I see here about yarn vs npm.
Generally speaking the main tool is slow, or otherwise suboptimal but does a lot. A different tool is built with an emphasis on speed. Eventually, the main stream tools picks up the speed features of the other tool and then "wins" because it does more. (Side note: I have no idea when Gradle came out in comparison to Buck/Bazel) For context, Gradle in the 4.0 version is starting to really support build cache support.
Edit: this comment is a reaction to all the other comments I see here about yarn vs npm.
How does this stack up against yarn now?
I built a project to compare yarn and npm. See https://github.com/thomaschaaf/npm-vs-yarn
The different shows how much better npm@5 is :)
It installs two node.js projects (react & ghost) and shows how long it takes to do under multiple scenarios (cold cache, installed and lockfile). It is automatically run each day. It also creates an average if one version is run multiple times.
The different shows how much better npm@5 is :)
It installs two node.js projects (react & ghost) and shows how long it takes to do under multiple scenarios (cold cache, installed and lockfile). It is automatically run each day. It also creates an average if one version is run multiple times.
That's really smart. I love how you're using Travis matrix builds for this, and publishing the results via the Google Sheets API.
I never used yarn, but I just did some research about the changes in comparison to yarn.
Well, actually they are more similar now, obviously (saving by default for example). In terms of speed, npm has improved as well (https://twitter.com/passcod/status/860437637764571140). Even though npm still seems to be slower than yarn (https://docs.google.com/presentation/d/1re-AGrTOf4zmnXXDMTG1...). At the end, I'd say it's about personal preference, they both get the job done.
Glad to know they addressed the issues why I switched to yarn (speed, save by default and lock file).
Definitely will give this a try once it is moved out from pre-release.
Definitely will give this a try once it is moved out from pre-release.
>All installs will be saved by default
>since npm@3, npm will automatically update npm-shrinkwrap.json when you save
All installs update shrinkwrap then? doesn't that start to make it redundant to package.json in the first place now. Does this make git tracking package and shrinkwrap mandatory in-case you want to install, test but then roll back the version, as shrinkwrap will already have changed.
edit: Oh I'm supposed to --no-save now for that described flow?
>since npm@3, npm will automatically update npm-shrinkwrap.json when you save
All installs update shrinkwrap then? doesn't that start to make it redundant to package.json in the first place now. Does this make git tracking package and shrinkwrap mandatory in-case you want to install, test but then roll back the version, as shrinkwrap will already have changed.
edit: Oh I'm supposed to --no-save now for that described flow?
Seems like their catching up to Yarn, and finally they are there. I'm happy to ditch Yarn and go back to NPM once it's as good.
Why go back if you already migrated?
A ton of things either don't work or are harder with Yarn.
It's great at what it does, and great for the simple case, but it's only a subset of functionality.
It's great at what it does, and great for the simple case, but it's only a subset of functionality.
- Global dependencies do not work on Yarn if you install node with brew.
- NPM comes pre-installed with Node.
- NPM comes pre-installed with Node.
last I check certain features like bundledDeps didn't work on yarn
Someone told they now use hashes for versioning, like Nix, is this true?
Is it finally save to install 2 times and get 100% the same packages?
Is it finally save to install 2 times and get 100% the same packages?
It's been that way for years, since shrinkwrap was invented.
Shrinkwrap did not use hashes and did not guarantee deterministic installs. Subdependencies could still end up as different versions.
Even the shrinkwrap file itself contained a lot of trash and generated massive diff noise. I had a file full of scripts to make the shrinkwrap file usable and even then we had (production!!!) issues due to changing subdeps. We reworked our build processes to zip & deploy the exact code that passed the tests to work around that, but it was still a massive pain that installs at different points in time would yield different trees.
When yarn came out, I deleted a folder full of hacky scripts, cut my install time by 60% and finally got deterministic installs. Needless to say, I was ecstatic.
Even the shrinkwrap file itself contained a lot of trash and generated massive diff noise. I had a file full of scripts to make the shrinkwrap file usable and even then we had (production!!!) issues due to changing subdeps. We reworked our build processes to zip & deploy the exact code that passed the tests to work around that, but it was still a massive pain that installs at different points in time would yield different trees.
When yarn came out, I deleted a folder full of hacky scripts, cut my install time by 60% and finally got deterministic installs. Needless to say, I was ecstatic.
Shrink wrap versions subdependencies. Why would they end up as different versions?
npm used to have issues that if you had a shrinkwrap and a pre-existing node_modules directory and ran `npm install`, then npm would often report success but silently fail to make the node_modules directory actually match the shrinkwrap. ... After our build system ran into this issue once and built and deployed code to production with fatally mismatched dependencies, I wrote a hacky wrapper script which would double-check that node_modules really did match the shrinkwrap, and if not it would remove the directory entirely and re-run `npm install`... Thankfully this was fixed in npm v4.
OK that's a bug, but it's fixed for more than a year. Nobody in this entire thread has given a reason why a captured versions of the entire tree wouldn't produce deterministic output.
Yeah, it should only be bugs that cause subdependency versions to not match the shrinkwrap. Your question seemed valid. I can only guess others have run into similar issues as me and hadn't known if they were fixed.
Really? Does shrinkwrap work for transitive deps?
npm desperately needs a flat install option.
This is probably a naive question, but the only use case I've come across of needing a flat-install option was to use Polymer.js web components (which is why so much of the polymer project relies on bower).
Aside from that, what is the use case that flat installs solve?
Aside from that, what is the use case that flat installs solve?
Not naive, and that's pretty much the primary reason. But for all web components in general, not just the Polymer flavor.
gotcha, that makes sense. My only expose to web components was polymer. Thanks for the clarification
What does "flat install" mean?
> A new npm cache verify command that will garbage collect your cache, reducing disk usage for things you don’t need (-handwave-), and will do full integrity verification on both the index and the content
Nice.
Nice.
> Running npm while offline will no longer insist on retrying network requests. npm will now immediately fall back to cache if possible, or fail. (#15666)
Finally! :-)
Finally! :-)
"package-lock.json" can someone please define what it is and does, I've heard of shrink wrap, google doesn't reveal anything about this file, I have a vauge idea that it has something to do with shrinkwrap but don't know why it exists or if it's supposed to replace shrinkwrap or whatever. splainit!
I'm really excited about links. Links will make things much easier for monorepos.
Can someone explain the git semver thing ? How does it work?
How is the package manager for a language still iterating through major versions and making breaking changes nearly a decade after launch? Node is bizarre in that you have to follow its package manager so closely.
npm has to deal with scale: it's now the largest programming language package manager, and most project dependency trees have exploded in size as a result. Optimizing that led to yarn and to npm@5.
npm has to deal with scale because every single npm install goes to npm instead of a local cache of certain versions.
Bundler led to yarn. The JS community should start looking what other ecosystems do and have done (right and wrong).
Such a waste of time these past 20 years and it happens over and over.
Bundler led to yarn. The JS community should start looking what other ecosystems do and have done (right and wrong).
Such a waste of time these past 20 years and it happens over and over.
I could be incorrect, but I don't believe Bundler was the first to lock and manage dependencies? I seem to recall using such tools before 2010. This isn't to take away from your main point by any stretch; only recently has JavaScript gone from "downright terrible" to "tolerable" and most of it is through doing what others already did, better, many years before. It still feels like a lot of the community doesn't have sufficient exposure to stuff outside their closure and people who work with that community's stuff are the worse for it.
This statement is false and has been since the very first work done on the npm cli. npm has, in every version of it that has ever existed, since before there was even a package registry, installed preferentially from its local cache of packages where possible.
Couple questions:
Question 1: Does anyone else who's been around more than a couple years share my view that Yarn : NPM :: IO.JS : Node?
IOW: healthy competition, catalyst for necessary change, ultimately a bridge or stopgap.
Question 2: Any good comprehensive writeups on best practices?Committing lockfiles is a no-brainer. But what about globals? Per-node-version seems logical, but there are also semantic problems with "global" packages vs concept of executable binaries. I'd love to see a strong writeup outlining and defending a standard approach.