Npm Audit: broken by design?(overreacted.io)
overreacted.io
Npm Audit: broken by design?
https://overreacted.io/npm-audit-broken-by-design/
264 comments
I'm a maintainer of a few of the larger packages on npm. This is generally pretty accurate. Snyk Security seems only to find regex DoS bugs and I'm a bit disappointed in them being classified as high severity, and they're the only ones submitting reports right now.
They seem pretty adamant on filing CVEs despite what the owner says (It's normally fine but these DoS vulns require very large input to be handed into the function by untrusted sources, which given how these libraries work isn't going to be very common).
Now, I have people yelling at me about dependent packages not being updated because they don't understand version ranges, or because some audit states they are high vulns, or whatever.
Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly trusting what corporations say.
They seem pretty adamant on filing CVEs despite what the owner says (It's normally fine but these DoS vulns require very large input to be handed into the function by untrusted sources, which given how these libraries work isn't going to be very common).
Now, I have people yelling at me about dependent packages not being updated because they don't understand version ranges, or because some audit states they are high vulns, or whatever.
Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly trusting what corporations say.
> Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly trusting what corporations say.
Because this isn't true. Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. These Regex issues should be fixed, libraries should update to safe versions, things should advance and any incentive we have we should use to make this happen.
Because this isn't true. Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. These Regex issues should be fixed, libraries should update to safe versions, things should advance and any incentive we have we should use to make this happen.
I think it helps to inform the developer about possible issues, but I think in most cases depending on the software this is plainly not relevant and can be ignored. I wouldn't classify it has high severity. Also, It might just not be trivial to develop a regex library that cannot be DDOSed or the mechanism that was declared a vulnerability.
Might be nice to be able to tag libraries that should be ignored in audits. Perhaps there is such a function, not really a NPM expert. But if your projects has too many of these "high severity" problems, you probably stop doing them.
Still, I think the availability of such audits from the package manager is quite neat. As an embedded dev I think these are certainly luxury problems.
Might be nice to be able to tag libraries that should be ignored in audits. Perhaps there is such a function, not really a NPM expert. But if your projects has too many of these "high severity" problems, you probably stop doing them.
Still, I think the availability of such audits from the package manager is quite neat. As an embedded dev I think these are certainly luxury problems.
krageon(1)
> Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken.
I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally, and knowing under which circumstances package lock files were implemented.
They are broken.
> These Regex issues should be fixed
They do, pretty much immediately after they're reported.
> libraries should update to safe versions
I check all the version ranges of dependent libraries when I push a patch with vuln fixes. They get pulled just fine without needing to update every single package. This is what version ranges are for.
> things should advance
Yes but this is nebulous and vague and aside from the point.
> and any incentive we have we should use to make this happen.
I don't see where the disagreement is. This is exactly what happens all the time, nothing is the problem here. I don't get your point.
---
Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time) since they promised it for the v5 release. They were on a time crunch because they were trying to keep with Node.js's next major release timeline, which operates independently of npm's (at least, that's how it was conveyed to me).
So this change got pushed out, had an absolute mountain of bugs that took ages to fix (e.g. at one point adding a new dependency would delete your entire node_modules folder), and promised added security when in reality they do nothing of the sort.
Instead, they cause subtle caching-related bugs, they add an artifact to source control (which is always code smell in my book), crap up diffs/PRs, cause headaches across platforms, and do very little to help... anything, really.
They're super, super broken by design. Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
It was a shit show. They add absolutely nothing to the industry.
I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally, and knowing under which circumstances package lock files were implemented.
They are broken.
> These Regex issues should be fixed
They do, pretty much immediately after they're reported.
> libraries should update to safe versions
I check all the version ranges of dependent libraries when I push a patch with vuln fixes. They get pulled just fine without needing to update every single package. This is what version ranges are for.
> things should advance
Yes but this is nebulous and vague and aside from the point.
> and any incentive we have we should use to make this happen.
I don't see where the disagreement is. This is exactly what happens all the time, nothing is the problem here. I don't get your point.
---
Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time) since they promised it for the v5 release. They were on a time crunch because they were trying to keep with Node.js's next major release timeline, which operates independently of npm's (at least, that's how it was conveyed to me).
So this change got pushed out, had an absolute mountain of bugs that took ages to fix (e.g. at one point adding a new dependency would delete your entire node_modules folder), and promised added security when in reality they do nothing of the sort.
Instead, they cause subtle caching-related bugs, they add an artifact to source control (which is always code smell in my book), crap up diffs/PRs, cause headaches across platforms, and do very little to help... anything, really.
They're super, super broken by design. Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
It was a shit show. They add absolutely nothing to the industry.
Hope do you make your builds hermetic and reproducible without package locks?
Your builds are not reproducible with anything related to npm. Neither npm nor any bundler that I'm aware of guarantees that.
Unless we're not talking about the same reproducibility property. Also I don't know what "hermetic" means in this context but I doubt it's anything that npm solves correctly.
Unless we're not talking about the same reproducibility property. Also I don't know what "hermetic" means in this context but I doubt it's anything that npm solves correctly.
There is a way, but it's troublesome. Create a docker image with installed node modules. Save it, and from then onwards you have frozen node modules. If you need a new dependency/updated version you need to create a new image and npm i.
That's absolutely no different than just installing and not re-installing. Docker adds nothing in this case.
Not re-building doesn't make your build reproducible. It just means that you're... not building. If I save the result of a single iteration of an RNG, I can't claim that the RNG always produces the same result because I saved the result somewhere...
Not re-building doesn't make your build reproducible. It just means that you're... not building. If I save the result of a single iteration of an RNG, I can't claim that the RNG always produces the same result because I saved the result somewhere...
Where did they say they’re not building? Building your app does not mean you install the modules every time. Some apps are so large they have to be split into chunks / layers anyway. In golang this used to be the way you’d add deps, check the entire source into your version control.
Building the same project even without re-installing node modules has absolutely no guarantee of reproducibility.
Oh please explain what’s different. The entire code chunk is the same. How is that not reproducible?
The code being the same != reproducible. Build tools can incorporate e.g. build timestamps into the built artifacts, or randomize the output for e.g. pattern scanning/patch deterrence.
The input is irrelevant. I think you should have a look at what reproducible builds really are before evangelizing them.
The input is irrelevant. I think you should have a look at what reproducible builds really are before evangelizing them.
> I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally
So when you say "npm staff at the time", do you mean at the time of node 0.10?
> and knowing under which circumstances package lock files were implemented.
> Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time)
The amusing thing about your comment here is the parts which are accidentally correct.
`package-lock.json` files use the same file format as `npm-shrinkwrap.json` files. Always have, although of course the format of this file has changed significantly over the years, most dramatically with npm 7.
The "design" of the shrinkwrap/package-lock file was done rather quickly, since it was initially just a JSON dump of (most of) the data structure that npm was already using for dependency tree building. However, as far as I know, the days were the standard length of 24 hours, so while that may be "short", certainly shorter than I'd often prefer, they were (as far as I know) no shorter than any other days.
This was indeed shipped without any review by even a single "npm employee", which should not surprising, as "npm" was not at that time a legal entity capable of hiring employees. The initial work was done by Dave Pacheco, and reviewed by npm's author (at that time its sole committer and entire development staff), both of whom were Joyent employees at the time.
The use of a shrinkwrap as a non-published normal-use way to snapshot the tree at build time and produce reproducible builds across machines and time was not implemented by default until npm v5, but there wasn't really much to rush, on that particular feature. You could argue that npm 5 itself was rushed, and that's probably a fair claim, since there was some urgency to ship it along with node version 8, so as not to wait a year or more to go out with node v10.
> So this change got pushed out, had an absolute mountain of bugs that took ages to fix...
Idk, I think calling it a "mountain" is relative, actually ;)
> They're super, super broken by design.
I know you're using this phrase "broken by design" in the same sense as the author of the OP means it, but... has language just changed on me here, and I didn't notice?
As I've always heard the term used, something is "broken by design" when the actual intent is for a system to fail in some way, to achieve some goal. For example, a legislative or administrative process that is intentionally slow-moving and unable to accomplish its goals in a reasonable time frame, with the hope that this leaves room for independent innovation. Or a product that requires some minor upgrade or repair to continue working, so that the seller can keep tabs on their customers more easily. That kind of thing.
I think what you mean is not that it's "broken by design", but rather it's "a broken design". Unless this is like "begging the question", and I should just accept that I'm gradually coming to speak a language of the past, while the future moves on. It's certainly not intended to cause problems, as far as I'm aware.
If you really do mean "broken by design" (in the sense of a tail light that goes out after 50k miles so that you will visit the dealership and they can sell you more stuff), I'm super curious what you think npm gets out of it.
> Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
As of npm v7, there's no longer any practical reason why it can only audit the lockfile, rather than the actual tree on disk. Just haven't gotten around to implementing that functionality. If you want it changed, I suggest posting an issue https://github.com/npm/cli/issues. There's some question as to whether to prioritize the virtual tree or the actual tree, since prioritizing the actual tree would be a breaking change, but no reason why it can't fall back to that if there's no lockfile present.
But even approaching build reproducibility is impossible without lockfiles. If a new version of a transitive dependency is published between my install and yours, we'll get different package trees. If we both install from the same lockfile, we'll get the same package tree. (Not necessarily the same bytes on disk, since install scripts can change things, but at least we'll fetch the same bytes, or the build will fail.)
> So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
You do not recall correctly, sorry. (Or maybe you correctly recall an incorrect explanation?) The answer is reproducible builds. Using a lockfile does reduce network utilization in builds, but not very significantly.
> Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
I hope my tone is civil and playful enough in this message to not consider my response "yelling".
So when you say "npm staff at the time", do you mean at the time of node 0.10?
> and knowing under which circumstances package lock files were implemented.
> Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time)
The amusing thing about your comment here is the parts which are accidentally correct.
`package-lock.json` files use the same file format as `npm-shrinkwrap.json` files. Always have, although of course the format of this file has changed significantly over the years, most dramatically with npm 7.
The "design" of the shrinkwrap/package-lock file was done rather quickly, since it was initially just a JSON dump of (most of) the data structure that npm was already using for dependency tree building. However, as far as I know, the days were the standard length of 24 hours, so while that may be "short", certainly shorter than I'd often prefer, they were (as far as I know) no shorter than any other days.
This was indeed shipped without any review by even a single "npm employee", which should not surprising, as "npm" was not at that time a legal entity capable of hiring employees. The initial work was done by Dave Pacheco, and reviewed by npm's author (at that time its sole committer and entire development staff), both of whom were Joyent employees at the time.
The use of a shrinkwrap as a non-published normal-use way to snapshot the tree at build time and produce reproducible builds across machines and time was not implemented by default until npm v5, but there wasn't really much to rush, on that particular feature. You could argue that npm 5 itself was rushed, and that's probably a fair claim, since there was some urgency to ship it along with node version 8, so as not to wait a year or more to go out with node v10.
> So this change got pushed out, had an absolute mountain of bugs that took ages to fix...
Idk, I think calling it a "mountain" is relative, actually ;)
> They're super, super broken by design.
I know you're using this phrase "broken by design" in the same sense as the author of the OP means it, but... has language just changed on me here, and I didn't notice?
As I've always heard the term used, something is "broken by design" when the actual intent is for a system to fail in some way, to achieve some goal. For example, a legislative or administrative process that is intentionally slow-moving and unable to accomplish its goals in a reasonable time frame, with the hope that this leaves room for independent innovation. Or a product that requires some minor upgrade or repair to continue working, so that the seller can keep tabs on their customers more easily. That kind of thing.
I think what you mean is not that it's "broken by design", but rather it's "a broken design". Unless this is like "begging the question", and I should just accept that I'm gradually coming to speak a language of the past, while the future moves on. It's certainly not intended to cause problems, as far as I'm aware.
If you really do mean "broken by design" (in the sense of a tail light that goes out after 50k miles so that you will visit the dealership and they can sell you more stuff), I'm super curious what you think npm gets out of it.
> Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit).
As of npm v7, there's no longer any practical reason why it can only audit the lockfile, rather than the actual tree on disk. Just haven't gotten around to implementing that functionality. If you want it changed, I suggest posting an issue https://github.com/npm/cli/issues. There's some question as to whether to prioritize the virtual tree or the actual tree, since prioritizing the actual tree would be a breaking change, but no reason why it can't fall back to that if there's no lockfile present.
But even approaching build reproducibility is impossible without lockfiles. If a new version of a transitive dependency is published between my install and yours, we'll get different package trees. If we both install from the same lockfile, we'll get the same package tree. (Not necessarily the same bytes on disk, since install scripts can change things, but at least we'll fetch the same bytes, or the build will fail.)
> So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency.
You do not recall correctly, sorry. (Or maybe you correctly recall an incorrect explanation?) The answer is reproducible builds. Using a lockfile does reduce network utilization in builds, but not very significantly.
> Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something.
I hope my tone is civil and playful enough in this message to not consider my response "yelling".
Read as: tough, that company says you need to do the work so I demand you do the work.
A regex "denial of service" "vulnerability" could be important, if it shows up in code that processes untrusted input from end users.
But NPM Audit has no idea of context-- a "critical" bug in `browserlist`, which, in this context, is never used outside the development process and never takes input outside of what's in my package.json, gets the same prominence (or more so, since it's early in alphabetical order) as a "critical" bug in Express, potentially allowing my server to be compromised.
I'm not really sure what the solution is here; NPM's just a package manager and doesn't know how you're using a given package. A simple heuristic distinguishing development dependencies and runtime dependencies in NPM Audit might be a start, but that doesn't help with situations like create-react-app's react-scripts where everything, runtime or dev dependency, is a transitive dependency of one package declared as a runtime dependency.
But NPM Audit has no idea of context-- a "critical" bug in `browserlist`, which, in this context, is never used outside the development process and never takes input outside of what's in my package.json, gets the same prominence (or more so, since it's early in alphabetical order) as a "critical" bug in Express, potentially allowing my server to be compromised.
I'm not really sure what the solution is here; NPM's just a package manager and doesn't know how you're using a given package. A simple heuristic distinguishing development dependencies and runtime dependencies in NPM Audit might be a start, but that doesn't help with situations like create-react-app's react-scripts where everything, runtime or dev dependency, is a transitive dependency of one package declared as a runtime dependency.
Agreed!
A “Critical” bug in a dev context should mean something very different from a “Critical” bug in a prod context. A “Critical” devDependency bug should be either a direct threat to the developer’s context, either by infecting the dev machine or by injecting a supply-chain problem, worming it’s way into downstream contexts.
npm audit is just not granular OR careful enough to address these issues appropriately.
A “Critical” bug in a dev context should mean something very different from a “Critical” bug in a prod context. A “Critical” devDependency bug should be either a direct threat to the developer’s context, either by infecting the dev machine or by injecting a supply-chain problem, worming it’s way into downstream contexts.
npm audit is just not granular OR careful enough to address these issues appropriately.
Agree 100%
Would be nice if package.json had a flag to indicate the runtime would be either Node.js or a browser. So many of these "bugs" have no bearing in a browser context.
The package.json should be able to actively ignore vulnerability id's. As id's disappear with audits the npm audit could just remove those, eg a "prune"
[deleted]
IMHO one solution would be to categorize vulnerabilities separately for prod dependencies and dev dependencies, and bubble that categorization up.
For example, a RegEx DDoS vulnerability in Express would show up as high severity, while the same would not show in the bundler you use, or any package that your bundler has in its dependency tree.
For example, a RegEx DDoS vulnerability in Express would show up as high severity, while the same would not show in the bundler you use, or any package that your bundler has in its dependency tree.
Other developers have no idea of context either. Unless you have a way of enforcing that certain code is never exposed to user input (and I agree that a build-time-only dependency does solve that), sooner or later it will be.
Accepting regexes from user input is a really insidious class of bug that can go undetected for years. I've seen real outages caused by it, so it's absolutely worth doing something proactive about.
Accepting regexes from user input is a really insidious class of bug that can go undetected for years. I've seen real outages caused by it, so it's absolutely worth doing something proactive about.
True story, the npm registry was once taken down (not maliciously, just by accident) by a ReDOS in node-semver. That was extra fun to debug because the failure happened inside of CouchDB.
> A regex "denial of service" "vulnerability" could be important, if it shows up in code that processes untrusted input from end users.
But in this context what's the end result? Chrome locking up on the end user's (attacker's) machine? Again, an "attacker" doesn't have access to the source code for distribution. By inputting bad regexp data they're only DOSin themselves, no?
But in this context what's the end result? Chrome locking up on the end user's (attacker's) machine? Again, an "attacker" doesn't have access to the source code for distribution. By inputting bad regexp data they're only DOSin themselves, no?
Could be in a service on a server, which in this case a RegEx DOS could lock the server for all users.
Or if a JS frontend takes in input that comes from other users-- something like forum post titles or content.
That's "just" a browser freeze for end users, but still a potential DOS vulnerability if it's in the application's critical path.
That's "just" a browser freeze for end users, but still a potential DOS vulnerability if it's in the application's critical path.
Right, but that's why I wrote "context," and seems to be the primary complaint in this article.
[deleted]
StackOverflow and Cloudflare have both self-DoSed themselves with such flaws, causing downtime.
This kind of nonsense really goes back to the broken CVE process. https://opensourcesecurity.io/2021/03/30/its-time-to-fix-cve...
Linux kernel maintainer Greg Kroah-Hartman has a similar opinion. https://github.com/gregkh/presentation-cve-is-dead/blob/mast...
Edit: LWN mention https://lwn.net/Articles/801157/
Linux kernel maintainer Greg Kroah-Hartman has a similar opinion. https://github.com/gregkh/presentation-cve-is-dead/blob/mast...
Edit: LWN mention https://lwn.net/Articles/801157/
The view of SQLite developers on CVEs is also dim: https://www.sqlite.org/cves.html
Beautifully succinct. This quote: "Grey-hat hackers are rewarded based on the number and severity of CVEs that they write. This results in a proliferation of CVEs that have minor impact, or no impact at all, but which make exaggerated impact claims." Alignment of incentives is messed up. Goodhart-Strathern's and Campbell's laws apply.
Sounds like academic research publications. Sure, that will totally be a key step toward cancer therapy or better biofuels (realistically, the PI gets his jollies by shoving aldehyde groups onto random molecules)
Oh, you mean like the guys who tried to inject vulnerabilities into the linux kernel and got their entire university on Greg Kroah-Hartman's shit list? https://news.ycombinator.com/item?id=26887670
That sounds better than a PI that gets their jollies by shoving nitro groups into unsuspecting organic molecules.
Although I still have a deep admiration for the Klapötke "Energetic Materials" group at Munich Uni.
Although I still have a deep admiration for the Klapötke "Energetic Materials" group at Munich Uni.
Maybe then writing and submitting a CVE should cost some money that’s payed back together with the reward if the vulnerability is found to be „reasonable“ upon review?
I'm always suspicious of just throwing money at a problem, particularly in things like open source where money isn't always the motivator and can often be a corrupting influence. In some cases this will reduce the ability of genuinely well-intentioned people to participate, simply because they don't have the money up front, and for well-funded organizations the money would have to be quite a lot.
I'd like to ask what, other than money directly motivates people? Is it prestige? A line on their resume? A requirement for a bootcamp class? In addition, we should re-evaluate the difficulty of submitting a CVE. Is it too easy? The story about a mass of "hey your regex parser could choke on this weird expression[1]" reports suggest that perhaps so. What can we do to make it so that CVEs and equivalents are truly meaningful? Also, just the fact that CVE reports are given a great deal of respect could be the problem, although at this point that seems to be self-correcting.
[1] Some classes of regex parsers are known to be vulnerable by nature, those that do backtracking for example, because their worst-case runtime grows exponentially and can run in unbounded time. This has been known since at least 2009. There are other implementations with better worst-case runtimes, but worse performance in typical cases. The fact that it's trivially easy to look at a regex parser to see if it does backtracking and construct an "evil" expression that breaks it means it's trivially easy to file a DOS report against any such parser.
I'd like to ask what, other than money directly motivates people? Is it prestige? A line on their resume? A requirement for a bootcamp class? In addition, we should re-evaluate the difficulty of submitting a CVE. Is it too easy? The story about a mass of "hey your regex parser could choke on this weird expression[1]" reports suggest that perhaps so. What can we do to make it so that CVEs and equivalents are truly meaningful? Also, just the fact that CVE reports are given a great deal of respect could be the problem, although at this point that seems to be self-correcting.
[1] Some classes of regex parsers are known to be vulnerable by nature, those that do backtracking for example, because their worst-case runtime grows exponentially and can run in unbounded time. This has been known since at least 2009. There are other implementations with better worst-case runtimes, but worse performance in typical cases. The fact that it's trivially easy to look at a regex parser to see if it does backtracking and construct an "evil" expression that breaks it means it's trivially easy to file a DOS report against any such parser.
AFAIK MITRE has a process for an organization to register as vendor, and then it would accept CVEs for their products only from the vendor, not from random people. Of course this has an opposite failure mode that may have unscrupulous vendors hide issues or just be lazy in issuing CVEs for existing bugs, but it eliminates the problem of random people issuing a ton of CVEs for non-issue bugs.
I'm pretty sure CVEs and the like came about because vendors were choosing to hide or deny security vulnerabilities. Vulnerability disclosure policies are a whole different kettle of worms.
Maybe, but you risk swinging too far in the opposite direction into under-reporting of vulnerabilities.
Believe me, 90% of people who find bugs for a living are perfectly content with keeping them to themselves and/or selling them privately.
I had a researcher contact me about a "vuln" in an OSS effort of mine once. The vuln made no sense w/ how the tool was used, but they published and I earned a CVE scarlet letter nonetheless. I finally "fixed" it, but IMHO, nothing was ever broken or vulnerable.
I wouldn't call a CVE a scarlet letter. Given the current state of software engineering, it's more like "my project is valuable enough to be used by someone that cares about security". You fixed it, one less bug to worry about. No doubt there are many less popular products with many worse vulnerabilities that don't have a CVE.
Even OpenBSD had to change their tagline to "Only two remote holes in the default install, in a heck of a long time!" (from "Five years without a remote hole in the default install!") Still a pretty impressive track record.
Even OpenBSD had to change their tagline to "Only two remote holes in the default install, in a heck of a long time!" (from "Five years without a remote hole in the default install!") Still a pretty impressive track record.
> You fixed it, one less bug to worry about.
Those "bugs" can be features though - or the work involved to fix the bug meant that high-impact feature work - or other bugfixes, had to be postponed or even cancelled.
Our SaaS frequently gets security "researchers" (read: people running online scanners) submitting emails through our contact-form informing us about click-jacking attacks on our login-page - the problem for us is that we have a lot of second-party and third-party integrations on unbounded origins that offer access to our application, and by extension our login-screen through an <iframe> on their own origin, which is sometimes even an on-prem LAN web-server accessed through embedded devices where we can't use popups to do it properly - let alone switch to a more robust OIDC system - so there is no easy solution that makes the "I ran a tool, gimme $100" people go-away without causing a much bigger problem to now exist.
Those "bugs" can be features though - or the work involved to fix the bug meant that high-impact feature work - or other bugfixes, had to be postponed or even cancelled.
Our SaaS frequently gets security "researchers" (read: people running online scanners) submitting emails through our contact-form informing us about click-jacking attacks on our login-page - the problem for us is that we have a lot of second-party and third-party integrations on unbounded origins that offer access to our application, and by extension our login-screen through an <iframe> on their own origin, which is sometimes even an on-prem LAN web-server accessed through embedded devices where we can't use popups to do it properly - let alone switch to a more robust OIDC system - so there is no easy solution that makes the "I ran a tool, gimme $100" people go-away without causing a much bigger problem to now exist.
> there is no easy solution that makes the "I ran a tool, gimme $100" people go-away
I use the "mark as spam" button :)
I use the "mark as spam" button :)
> so there is no easy solution that makes the "I ran a tool, gimme $100" people go-away without causing a much bigger problem to now exist.
Maybe consider setting up a free-tier HackerOne bounty program? I think they triage to some degree on your behalf.
Maybe consider setting up a free-tier HackerOne bounty program? I think they triage to some degree on your behalf.
In the free tier? Triage is a paid service.
Here's the video of the talk that goes with Greg Kroah-Hartman's slides:
https://www.youtube.com/watch?v=HeeoTE9jLjM
CVEs can be used to try and extort OSS maintainers into expensive consulting gigs: https://www.patreon.com/posts/unfixed-security-21250652
I wouldn't take Greg's opinions on security too seriously.
Spender has a much more nuanced, informed view. I think it covers the issues of the CVE process well, but doesn't make the same mistakes that Greg does.
https://www.grsecurity.com/reports_of_cves_death_greatly_exa...
Spender has a much more nuanced, informed view. I think it covers the issues of the CVE process well, but doesn't make the same mistakes that Greg does.
https://www.grsecurity.com/reports_of_cves_death_greatly_exa...
The more I work with parsing, parser combinators and writing grammars for little languages, the less often I find myself using or wanting to use any regex at all. When I do, I always feel like there should be a better way, perhaps a type safe way of accessing the info I need and so on. It feels "Ugh, there should be a better way to do this." Especially in JavaScript, regexes blow in comparison to languages with named matching groups and all that. In JS regex really feels horrible, even more cryptic than in other languages.
I think regexes are often used as a quick and dirty solution to problems, which should be solved differently. But once the regex "works" and is in place, others begin to rely on that output. Over time cruft begins to accumulate and the regex is forgotten or at least never replaced with anything more appropriate.
I think regexes are often used as a quick and dirty solution to problems, which should be solved differently. But once the regex "works" and is in place, others begin to rely on that output. Over time cruft begins to accumulate and the regex is forgotten or at least never replaced with anything more appropriate.
> The more I work with parsing, parser combinators and writing grammars for little languages, the less often I find myself using or wanting to use any regex at all.
Surprise: The most common parser combinator libraries do backtracking. That's exactly the problem. Any solution as widely used (if not overused) as regular expressions ends up exposing a number of dark corners where the design isn't as clean and tight as you would want it. There are lots of better ways, but most of them are specialized and are totally unsuited for significant areas where people need something.
That said: yes I've used LR(1) parsing (not LALR) using a library that uses parser combinators with a good interface, and it's more powerful than regex and worth it for the right usecase.
Surprise: The most common parser combinator libraries do backtracking. That's exactly the problem. Any solution as widely used (if not overused) as regular expressions ends up exposing a number of dark corners where the design isn't as clean and tight as you would want it. There are lots of better ways, but most of them are specialized and are totally unsuited for significant areas where people need something.
That said: yes I've used LR(1) parsing (not LALR) using a library that uses parser combinators with a good interface, and it's more powerful than regex and worth it for the right usecase.
> Especially in JavaScript, regexes blow in comparison to languages with named matching groups and all that
Good news (well, probably). JavaScript (ECMAScript 2018+) now supports named matching groups.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...
Good news (well, probably). JavaScript (ECMAScript 2018+) now supports named matching groups.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...
Add prototype pollution and you've covered 90% of all "vulnerabilities"
We got bit by this last week; our scans were suddenly all red, and nobody could deploy to production. We had to write an analysis of why this wasn't actually dangerous to us in order to get security to suppress the findings.
Hmm. This actually sounds really plausible. I wonder if there's any way to check that.
Having 20 supposeddly high risk issues about possible DoS when all come from a build dev-dependency is just totally useless. If only I could add whitelists, then it would be bearable. Like "I don't care about such kind of issue in a dev dependency".
Isn't this an area where gamification and machine learning could actually be useful, if applied carefully?
If people are competing for CVEs, then why not work out a way to better differentiate them them through scoring and make this visible. The goal would be for attention to shift to the scoring instead of only a CVE count. Offer both views of the world, so tools could still fall back on the problematic listings they get today.
Apply machine learning to classify CVEs based on the reputation of the reporter, blast radius, or other criteria. Use that to drive community review and scoring.
I would not see this a panacea because it brings a lot of challenges (a la StackOverflow), but it would be much better than what we have today.
If people are competing for CVEs, then why not work out a way to better differentiate them them through scoring and make this visible. The goal would be for attention to shift to the scoring instead of only a CVE count. Offer both views of the world, so tools could still fall back on the problematic listings they get today.
Apply machine learning to classify CVEs based on the reputation of the reporter, blast radius, or other criteria. Use that to drive community review and scoring.
I would not see this a panacea because it brings a lot of challenges (a la StackOverflow), but it would be much better than what we have today.
We're kind of already doing scoring in that CVEs are usually graded on severity, but researchers are motivated to inflate the severity of CVEs they find. So the question you'd need to tackle is how does one apply a universal standard to measure the real impact of a CVE?
I suspect it's an impossible challenge, but I only dip into this domain casually so maybe someone has better ideas.
I suspect it's an impossible challenge, but I only dip into this domain casually so maybe someone has better ideas.
I'm not making the claim it's a universal standard, but there are likely indications that some researchers are a different pedigree from others. A researcher reporting the same kind of low grade vulnerability probably shouldn't carry the same reputation score as other researchers.
I don't think there is a perfect way to do this, and I don't think there is an absolute standard that can be applied. It will be unfair to some people, but the system should have options for resolution when there are egregious mistakes. I'm not making the claim either, that the views of the data you are interested in are the ones I might be interested in. A good system would provide some different levels which itself is an incentive towards better research that would break through.
I don't think there is a perfect way to do this, and I don't think there is an absolute standard that can be applied. It will be unfair to some people, but the system should have options for resolution when there are egregious mistakes. I'm not making the claim either, that the views of the data you are interested in are the ones I might be interested in. A good system would provide some different levels which itself is an incentive towards better research that would break through.
I'm more inclined to think the better solution would be to stop issuing CVEs for trivial "exploits" like Regex DOS, unless there's actual demonstrated uses of the exploit.
Honestly, this hit me.
I'm not a react developer, I was experimenting with it for a new project. I finished the tic-tac-toe tutorial, then tried to throw bootstrap on top to build from there. It told me there was 97 vulnerabilities (85 moderate, 12 high)...
I just deleted the directory and went back to vanilla JS. This is a fun side project, I don't need that.
My tweet about it: https://twitter.com/preinheimer/status/1402785757962592256
I'm not a react developer, I was experimenting with it for a new project. I finished the tic-tac-toe tutorial, then tried to throw bootstrap on top to build from there. It told me there was 97 vulnerabilities (85 moderate, 12 high)...
I just deleted the directory and went back to vanilla JS. This is a fun side project, I don't need that.
My tweet about it: https://twitter.com/preinheimer/status/1402785757962592256
That is exactly the point of the article. Every JS developer knows that these numbers are stupid and doesn't look at them.
However a beginner that doesn't know what impact they have of course is scared if when installing a library it tells you that there are all that vulnerabilities.
However a beginner that doesn't know what impact they have of course is scared if when installing a library it tells you that there are all that vulnerabilities.
As a beginner _just to npm_ I can imagine getting totally freaked out and worried that my whole system was _potentially_ compromised after seeing a “Critical” vulnerability reported as installed on my system.
After all, npm can execute any script with the users permissions on install…except often (compared to bash) it’s less easily inspected due to the common use of nested dependencies!
I, too, would delete my node_modules, and if I even wanted to move forward at that point, would probably waste at least half a day looking up the Critical vulns and discovering that they are probably not at all critical in my particular scenario. Like not at all for the 99.99% use case.
After experiencing something like that, it’s just like the article says. “The boy who called wolf.” Really terrible use of the labels “Critical” and “High”. The labels are fine, but the way they are applied is just stupid.
After all, npm can execute any script with the users permissions on install…except often (compared to bash) it’s less easily inspected due to the common use of nested dependencies!
I, too, would delete my node_modules, and if I even wanted to move forward at that point, would probably waste at least half a day looking up the Critical vulns and discovering that they are probably not at all critical in my particular scenario. Like not at all for the 99.99% use case.
After experiencing something like that, it’s just like the article says. “The boy who called wolf.” Really terrible use of the labels “Critical” and “High”. The labels are fine, but the way they are applied is just stupid.
Who doesn't run an npm based app in a jail/vm/etc and as a regular user with any more than the bare minimum access needed to get it's job done?
I would imagine installing directly as a regular user is the _typical_ approach, and even more-so for beginners.
I don’t see any recommendation in the nodejs or npm docs for any other approach.
It may be commonsense and obvious to you, but I would be really surprised if commonsense and common practice overlap significantly in scenarios like this for all but the most security conscious.
I don’t see any recommendation in the nodejs or npm docs for any other approach.
It may be commonsense and obvious to you, but I would be really surprised if commonsense and common practice overlap significantly in scenarios like this for all but the most security conscious.
FYI, it's probably not react-bootstrap or bootstrap or any of their dependencies being flagged. It's some of the other ~2K packages already installed.
I guess we could have a documentary series!
Next up, npm link: broken by design
Synopsis of the chapter: A command with broken behavior that has been reported since as early as 2015, but that "got lost" every time the winds changed and the project decided to change where to manage bugs. What will happen in the latest attempt from an affected user? Tune in and be ready for an exciting ride!
https://github.com/npm/cli/issues/2372
Spoiler: bugs are not sentient beings that solve themselves just by closing the issue (or the whole issue tracker, for that matter).
EDIT to clarify: Sorry for the snarkyness. I just find it funny in a sarcastic way that up until I reported the issue in 2020, the issue had been reported repeatedly but "lost" in the way because the project closed or ignored the issue every time it changed issue tracker. Which happened twice since 2017! so go figure the amount of reports that had gone to waste. On the flip side, this time they haven't changed platforms (yet), although the issue has been closed prematurely anyway.
Next up, npm link: broken by design
Synopsis of the chapter: A command with broken behavior that has been reported since as early as 2015, but that "got lost" every time the winds changed and the project decided to change where to manage bugs. What will happen in the latest attempt from an affected user? Tune in and be ready for an exciting ride!
https://github.com/npm/cli/issues/2372
Spoiler: bugs are not sentient beings that solve themselves just by closing the issue (or the whole issue tracker, for that matter).
EDIT to clarify: Sorry for the snarkyness. I just find it funny in a sarcastic way that up until I reported the issue in 2020, the issue had been reported repeatedly but "lost" in the way because the project closed or ignored the issue every time it changed issue tracker. Which happened twice since 2017! so go figure the amount of reports that had gone to waste. On the flip side, this time they haven't changed platforms (yet), although the issue has been closed prematurely anyway.
npm has been buggy for so long that it is actively driving me away from NodeJS.
I would like to wait to see if the rearchitecture for npm 7 actually allows them to test for regressions more productively, but at this point I don't know if I have the stamina to wait for my company to migrate to node 16.
Someone offers me a job doing Elixir or non-webapp stuff and I'm out. Probably permanently.
I would like to wait to see if the rearchitecture for npm 7 actually allows them to test for regressions more productively, but at this point I don't know if I have the stamina to wait for my company to migrate to node 16.
Someone offers me a job doing Elixir or non-webapp stuff and I'm out. Probably permanently.
Node is the new IE.
It reminds me, that exact issue is why I switched from `npm` to `yarn` a few years ago. Sad to see that it has not been solved after all this time..
> Inline all dependencies during publish… From a maintainer’s point of view, the upsides are clear: you get faster boot time, smaller downloads, and — as a nice bonus — no bogus vulnerability reports from your users.
And when you inlined a version that later really does have a vulnerability, it is not easily flagged or fixed by your consumers.
The tension between "upgrades (especially of indirect dependencies) might break" and "upgrades (especially of indirect dependencies) might be necessary to fix bugs or patch security vulnerabilities" is real. There is no magic bullet to get around it. There are practices to try to balance it -- which generally involve ecosystem-wide commitment to backwards compatibility, reflected in semantic versioning (and minimizing major releases).
That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem. It makes the dependency tree even more insane, and it also accomodates a false belief that it's fine to lock to very specific versions of dependencies -- because it won't prevent other dependencies from co-existing in tree with other conflicting requirements after all. Which then accomodates a false belief that dependency maintainers don't need to worry too much about backwards compatibility, after all consumers can just lock to specific working versions... and now we wind up with insane dependency trees which depend on vulnerable versions in ways that require a whole bunch of releases to resolve.
And when you inlined a version that later really does have a vulnerability, it is not easily flagged or fixed by your consumers.
The tension between "upgrades (especially of indirect dependencies) might break" and "upgrades (especially of indirect dependencies) might be necessary to fix bugs or patch security vulnerabilities" is real. There is no magic bullet to get around it. There are practices to try to balance it -- which generally involve ecosystem-wide commitment to backwards compatibility, reflected in semantic versioning (and minimizing major releases).
That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem. It makes the dependency tree even more insane, and it also accomodates a false belief that it's fine to lock to very specific versions of dependencies -- because it won't prevent other dependencies from co-existing in tree with other conflicting requirements after all. Which then accomodates a false belief that dependency maintainers don't need to worry too much about backwards compatibility, after all consumers can just lock to specific working versions... and now we wind up with insane dependency trees which depend on vulnerable versions in ways that require a whole bunch of releases to resolve.
> I'm not totally sure why they are so insane
I think a big part of it is that due to much stronger pressure on bundle size than most other environments, each library tends to be small, so there have to be more to carry the same amount of functionality.
Duplicates are certainly a contributing factor as well, and small bundles compound with allowed-duplication to further increase the tree size. I think that small package size also probably makes it harder to require a single version for each dep, since there are going to be more edges in the graph and therefore more relations for library maintainers to keep track of (including what would in other languages be intra-package requirements), so you're more likely to get version incompatibilities.
I think a big part of it is that due to much stronger pressure on bundle size than most other environments, each library tends to be small, so there have to be more to carry the same amount of functionality.
Duplicates are certainly a contributing factor as well, and small bundles compound with allowed-duplication to further increase the tree size. I think that small package size also probably makes it harder to require a single version for each dep, since there are going to be more edges in the graph and therefore more relations for library maintainers to keep track of (including what would in other languages be intra-package requirements), so you're more likely to get version incompatibilities.
I agree with all of this. Also JavaScript's "standard library" is nearly nonexistent (or at least was when Node first got big). That built a culture of people assuming they needed to reach for third-party dependencies for nearly everything (see: leftpad).
Slightly related to the lack of a standard library is that a lot of these 3rd party packages come from random people in the community. It’s great that people are so willing and able to share code, but it also means that as a community we put a lot of trust into code that may not be vetted or funded properly. I think we assume that because these packages are open source that someone is making sure they are safe to consume, but because there’s so many of them it’s hard to verify them.
> That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem.
Aren't npm dependency trees so large because JavaScript doesn't have much of a standard library? And also, similarly, the community is so large and has been moving so fast that even de facto standards have difficulty forming and surviving at a large scale across the community.
Aren't npm dependency trees so large because JavaScript doesn't have much of a standard library? And also, similarly, the community is so large and has been moving so fast that even de facto standards have difficulty forming and surviving at a large scale across the community.
The lack of static typing (in base JS, at least) also makes it hard for tools to automatically spot very basic brokenness in dependencies without (repeatedly) running & testing the code. This makes even "safe" version bumps less trustworthy and harder to audit, and makes it harder for developers to notice if they've accidentally changed an interface on one of their libraries that they marked as a minor patch (i.e. the errors are both harder to check for, and more likely to occur, basically because they're harder to check for), so it's tempting to stick to old versions longer.
Add to that everything else—the fast pace of changes, javascript "culture", the weak standard library, the tendency to patch in what ought to either be basic language features or else avoided in favor of more-vanilla idioms, often in competing and incompatible ways—and all that is how you end up with 20 slightly-different copies of the same damn library in your dependency tree, and then 20 other copies of another library that does the same thing.
Add to that everything else—the fast pace of changes, javascript "culture", the weak standard library, the tendency to patch in what ought to either be basic language features or else avoided in favor of more-vanilla idioms, often in competing and incompatible ways—and all that is how you end up with 20 slightly-different copies of the same damn library in your dependency tree, and then 20 other copies of another library that does the same thing.
Big part of the insanity is also how projects introduce dependencies for very simple things like padding a number or a string.
https://www.npmjs.com/package/is-odd
447,211 weekly downloads for what can be done in vanilla JS with foo % 2 === 1;
447,211 weekly downloads for what can be done in vanilla JS with foo % 2 === 1;
You can thank Jon Schlinkert for that: https://github.com/jonschlinkert?tab=repositories
He abandoned the projects, but:
* https://github.com/jonschlinkert/is-odd
* https://github.com/jonschlinkert/is-even
Of course that doesn't stop them from getting ~500k downloads/week.
He abandoned the projects, but:
* https://github.com/jonschlinkert/is-odd
* https://github.com/jonschlinkert/is-even
Of course that doesn't stop them from getting ~500k downloads/week.
Which itself depends on is-number package.
You and GP have shown me new even more scary depths here. Not sure whether I should thank you for that. (When does the next flight off this rock go?)
It's not really that crazy. This package is a small amount of code, but it's important code (the same goes for this package's dependency is-number). This package shows up in the dependency trees of some popular packages, which is probably where most of the weekly downloads come from.
If you're writing straightforward application code where you already know you have a valid number, then this package probably isn't for you. You can just do num % 2 === 1.
If you're writing straightforward application code where you already know you have a valid number, then this package probably isn't for you. You can just do num % 2 === 1.
I get it, JS is a weird language, but simple things like "is number" are still easy enough to do in JS, especially, when ints and floats are all just "number" in JS:
function is_number(val) { return typeof val === "number"; }
With a function that easily written, no one should have any excuse to depend on a third-party dependency for it.
function is_number(val) { return typeof val === "number"; }
With a function that easily written, no one should have any excuse to depend on a third-party dependency for it.
But the code you posted is not what the code in this package does. If all you need is the code that you posted, then you should absolutely use that.
> not what the code in this package does
Here's `is-number` (https://github.com/jonschlinkert/is-number/blob/master/index...):
Here's `is-number` (https://github.com/jonschlinkert/is-number/blob/master/index...):
module.exports = function(num) {
if (typeof num === 'number') {
return num - num === 0;
}
if (typeof num === 'string' && num.trim() !== '') {
return Number.isFinite ? Number.isFinite(+num) : isFinite(+num);
}
return false;
};
Note that there’s a difference between “needs to be a package” and “needs to be used by everyone.” Nothing needs to be a package, because it’s always possible to copy paste code, regardless of how many lines it is.
leftpad was a small amount of code too. If it’s a small amount of code that’s design stable and downloaded often, it’s an extremely strong candidate for inclusion in the standard library.
It was added to the standard library: padStart
Well said.
I would add that I think the devDependencies solution is underrated. Not using it is a bad practice. npm has a nice feature, `npm prune --production` which will remove all the dev dependencies for you resulting in a clean build of the program. You can easily have things set up so that none of the development dependencies that have all these audit issues are ever present on your production machines if you do things right.
Furthermore, if you have a project with end users, devDependencies allows you to make clear in your issue template that audit issues that don't show up in a production build are very likely to be false positives and will probably be closed without comment. If you aren't properly isolating your dependencies, you can't take advantage of this.
At the end of the day, as you say, the issue is largely with Node projects having too many dependencies and a large number of relatively minor issues that affect very specific use cases get reported. That's a hard ecosystem problem to solve, not necessarily something that indicates an inherent problem with npm audit; but if someone isn't using devDependencies, that alone could constitute an enormous improvement in their workflow.
I would add that I think the devDependencies solution is underrated. Not using it is a bad practice. npm has a nice feature, `npm prune --production` which will remove all the dev dependencies for you resulting in a clean build of the program. You can easily have things set up so that none of the development dependencies that have all these audit issues are ever present on your production machines if you do things right.
Furthermore, if you have a project with end users, devDependencies allows you to make clear in your issue template that audit issues that don't show up in a production build are very likely to be false positives and will probably be closed without comment. If you aren't properly isolating your dependencies, you can't take advantage of this.
At the end of the day, as you say, the issue is largely with Node projects having too many dependencies and a large number of relatively minor issues that affect very specific use cases get reported. That's a hard ecosystem problem to solve, not necessarily something that indicates an inherent problem with npm audit; but if someone isn't using devDependencies, that alone could constitute an enormous improvement in their workflow.
If there's a vulnerability in Webpack (a devDependency) that injects malicious code into your bundle, `npm prune --production` won't save you.
This is not a vulnerability (ie. security bug) it's an attack (ie. malicious).
It doesn't really matter how you call it; the problem is that there could be CVE's in your devDependencies that affect your production build, and pruning those dependencies after using them to create that build doesn't remove the risk.
I'm currently using 'npm ci --prod', which means that they never get installed in the first place.
However it's incompatible with optionalPackages, so I'm carrying around some tertiary dependencies that are a small but noticeable fraction of the entire archive size.
However it's incompatible with optionalPackages, so I'm carrying around some tertiary dependencies that are a small but noticeable fraction of the entire archive size.
> That npm dependency trees are often insane
For my hundreds of repos (Java, Scala, JS, Typescript, Python...), Snyk flags 99% of the CVEs for the JS repos. Shocking how I've only seen a few dozen or so Java based CVEs flagged over the last few years.
Perhaps it's because my NPM based repos have ~10K more dependencies? That and the Java stdlib handling most needs w/ the vanilla lang.
For my hundreds of repos (Java, Scala, JS, Typescript, Python...), Snyk flags 99% of the CVEs for the JS repos. Shocking how I've only seen a few dozen or so Java based CVEs flagged over the last few years.
Perhaps it's because my NPM based repos have ~10K more dependencies? That and the Java stdlib handling most needs w/ the vanilla lang.
There is no magic bullet, no, but a lot of people are confused about how much less sense npm lockfiles make today than ruby lockfiles made when npm was still new.
Some of us are considerably salty about it. Especially the design-by-PR aspects of the whole thing that have resulted in confusing gyrations from one version to another.
Some of us are considerably salty about it. Especially the design-by-PR aspects of the whole thing that have resulted in confusing gyrations from one version to another.
Dan isn't the first person to notice:
https://www.voitanos.io/blog/don-t-be-alarmed-by-vulnerabili...
We disable [1] audit entirely because it's not a good default behavior within a monorepo. It spams the hundreds of developers with the list of "vulnerabilities" on every install, but only a few folks should really be upgrading packages.
We then run audit in non-blocking CI and track the total number of issues and mostly focus on critical ones.
[1] https://docs.npmjs.com/cli/v7/using-npm/config#audit
https://www.voitanos.io/blog/don-t-be-alarmed-by-vulnerabili...
We disable [1] audit entirely because it's not a good default behavior within a monorepo. It spams the hundreds of developers with the list of "vulnerabilities" on every install, but only a few folks should really be upgrading packages.
We then run audit in non-blocking CI and track the total number of issues and mostly focus on critical ones.
[1] https://docs.npmjs.com/cli/v7/using-npm/config#audit
We also wanted to use npm audit in our CI process so, instead of humans being careful, we could assert any known CVE stops a staging or prod deploy. Very annoyingly, npm audit doesn't have ignore functionality, at least when I was last forced to use it. I had to hack something together with bash scripts.
For vulnerabilities that we determined weren't an issue ever (vuln in frontend framework we didn't use), or weren't high priority enough to P0 through, we needed some way to ignore either permanently or temporarily specific vulnerabilities.
Given the enormous dependency sets eg react create, you'd think the tools would be better at managing them.
For vulnerabilities that we determined weren't an issue ever (vuln in frontend framework we didn't use), or weren't high priority enough to P0 through, we needed some way to ignore either permanently or temporarily specific vulnerabilities.
Given the enormous dependency sets eg react create, you'd think the tools would be better at managing them.
> Very annoyingly, npm audit doesn't have ignore functionality
I can't believe no one here has mentioned the fantastic tool "better-npm-audit" which can be included as an npm dependency[0] and lets you add specific vulnerabilities to an ignore list.
The ignore list is actually a JSON config file stored alongside package.json in the repo, so only one developer ever needs to see the npm audit warning and can mute it for everyone else (after getting their PR approved).
Even better, the config file lets you specify an expiry date for each entry in the ignore list, and provide a note, such as a link to the upstream issue being worked on, so that you can periodically be reminded to go back and check if a new version is available which can give more confidence that your code really isn't affected.
I think that developers might have to be instructed to use the "--no-audit" option to "npm install" if they don't want to see the (false positive) warnings that the default behaviour produces, and that's a bad habit to learn if not all projects they work on are using "better-npm-audit". I don't know if there is a way to make that option the default on a per-project basis.
[0] https://www.npmjs.com/package/better-npm-audit
I can't believe no one here has mentioned the fantastic tool "better-npm-audit" which can be included as an npm dependency[0] and lets you add specific vulnerabilities to an ignore list.
The ignore list is actually a JSON config file stored alongside package.json in the repo, so only one developer ever needs to see the npm audit warning and can mute it for everyone else (after getting their PR approved).
Even better, the config file lets you specify an expiry date for each entry in the ignore list, and provide a note, such as a link to the upstream issue being worked on, so that you can periodically be reminded to go back and check if a new version is available which can give more confidence that your code really isn't affected.
I think that developers might have to be instructed to use the "--no-audit" option to "npm install" if they don't want to see the (false positive) warnings that the default behaviour produces, and that's a bad habit to learn if not all projects they work on are using "better-npm-audit". I don't know if there is a way to make that option the default on a per-project basis.
[0] https://www.npmjs.com/package/better-npm-audit
Thank you!
edit: we're going to move to this. My implementation is a is a file with cve to ignore | reason | jira ticket but I think we can slam all that in there. And our expiry is manually checked monthly, so this is a big improvement.
edit: we're going to move to this. My implementation is a is a file with cve to ignore | reason | jira ticket but I think we can slam all that in there. And our expiry is manually checked monthly, so this is a big improvement.
Wouldn't you want to only stop a deploy if the commit introduced the vulnerability (i.e. the deploy changed the dependency tree).
From my experience most audit flags happen because a new vulnerability is discovered, which means stopping a deploy doesn't actually do anything helpful.
From my experience most audit flags happen because a new vulnerability is discovered, which means stopping a deploy doesn't actually do anything helpful.
we do daily deploys, so we're mostly using this as a way to check cves across rails and react in a way that fails loudly and doesn't require anyone to step outside their standard workflow. Regardless of whether the new commits introduced the cve or not.
Having hundreds of developers work on the same code repository seems insanely complicated. What are the advantages? Where can I read more about monorepos?
They do not use an usual git repository. Companies using monorepos have tools to restrict people's access to parts of it, and filter log noise.
Or, in other words, it's not that insanely complicated, because they have tools that make it look a lot like multiple repositories. And the large companies using multiple repositories have tools that make them look just like those monoreps. And HN has all those interesting threads full of people saying why one is better than the other.
Or, in other words, it's not that insanely complicated, because they have tools that make it look a lot like multiple repositories. And the large companies using multiple repositories have tools that make them look just like those monoreps. And HN has all those interesting threads full of people saying why one is better than the other.
The feature I missed about Subversion was, if you could figure out how to treat your code like a proper tree, individual teams would check out one, maybe two directories, and only the leads and operational people ever had the whole thing.
I never entirely understood what it was in subversion's internals that prevented it from being used as a DCVS. We could figure out version numbers with branches and multiple repos. If not then, certainly now.
There's a space between subversion and Git that could be occupied with something that sheds the worst behaviors of each and makes something better.
I never entirely understood what it was in subversion's internals that prevented it from being used as a DCVS. We could figure out version numbers with branches and multiple repos. If not then, certainly now.
There's a space between subversion and Git that could be occupied with something that sheds the worst behaviors of each and makes something better.
Revisions in Subversion are identified by a monotonically increasing number. How can you distribute that?
Versions are a tree structure. The implementation may have required the next revision number to be n + 1, but the data structure does not necessarily have to do that. If myself and someone else push commits, they have to be rebased and renumbered anyway, right?
We are just 2 devs at the moment and we've setup Lowdefy with a monorepo and yarn 2 and I can add that even for small projects a monorepo is just bliss especially with yarn 2 and lerna!
We come from separate repos on past versions of the code and the monorepo setup has sped dev up dramatically, even for a team of 2. Having the docs next to our code and autogen some of it from the source code, all is the same repo is just one advantage.
It's a small project but you can view the setup here: https://github.com/lowdefy/lowdefy
We come from separate repos on past versions of the code and the monorepo setup has sped dev up dramatically, even for a team of 2. Having the docs next to our code and autogen some of it from the source code, all is the same repo is just one advantage.
It's a small project but you can view the setup here: https://github.com/lowdefy/lowdefy
Google is probably the most (in?)famous example of monorepo. All of their code is in a single monorepo.
You can read a bit about the approach’s benefits and drawbacks from someone at Google here: https://medium.com/@Jakeherringbone/you-too-can-love-the-mon...
You can read a bit about the approach’s benefits and drawbacks from someone at Google here: https://medium.com/@Jakeherringbone/you-too-can-love-the-mon...
Does monorepo mean that if I checkout Google Wave, I also checkout the GMail source too?
Sort of, the google3 monorepo is so large that all checkouts are sparse checkouts. So you have a virtual checkout of everything but specify the subset you actually need.
Once you have the tooling to do this you can use the monorepo to store all kinds of interesting things, like built artifacts or the contents of the CDN, since it's basically just a giant hierarchical KV store with a global version.
Once you have the tooling to do this you can use the monorepo to store all kinds of interesting things, like built artifacts or the contents of the CDN, since it's basically just a giant hierarchical KV store with a global version.
Yes. And if you improve something used by both of them and have to refactor across the source tree you can do that in a single commit.
https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
yes
There's lots of writing about monorepos these days, and they're quite common in public projects though usually at a smaller scale (ex. React is a monorepo, so is Babel and VSCode).
While complicated, in my experience the tooling to get the same productivity from hundreds of separate repos is at least as complicated and generally discourages folks from contributing to codebases outside their own.
Our monorepo encourages a collective velocity culture where everyone is pushing every project forward. If you upgrade React or Node, fix a security issue, or implement an optimization, the hundreds of apps are all improved at the same time. It's harder for one team to "leap forward" in terms of tech stack quickly, but at the same time it's far less likely we end up with hundreds of outdated or abandoned codebases since everyone is collectively improving the repo together.
For example I've rarely seen a single engineer upgrade a thousand separate projects across a thousand git repos with complex nested dependencies very successfully, but that happens every day in our monorepo.
While complicated, in my experience the tooling to get the same productivity from hundreds of separate repos is at least as complicated and generally discourages folks from contributing to codebases outside their own.
Our monorepo encourages a collective velocity culture where everyone is pushing every project forward. If you upgrade React or Node, fix a security issue, or implement an optimization, the hundreds of apps are all improved at the same time. It's harder for one team to "leap forward" in terms of tech stack quickly, but at the same time it's far less likely we end up with hundreds of outdated or abandoned codebases since everyone is collectively improving the repo together.
For example I've rarely seen a single engineer upgrade a thousand separate projects across a thousand git repos with complex nested dependencies very successfully, but that happens every day in our monorepo.
Monorepos are pretty common nowadays, arguably popularized by Google
https://research.google/pubs/pub45424/
That said (1) “you are not Google” as the saying goes (2) Google built their VCS from scratch in house for scale.
However it’s not tops hard to do this with just git and some rules/patterns for organization.
https://research.google/pubs/pub45424/
That said (1) “you are not Google” as the saying goes (2) Google built their VCS from scratch in house for scale.
However it’s not tops hard to do this with just git and some rules/patterns for organization.
> Google built their VCS from scratch in house for scale
It's more complicated than that. Initially, there was Perforce. Then, Google grew client-side tooling on top of that ("g4", rather than "p4"). Then, someone grew client-side Git on top of that (for reasons, good or bad). And at some point, the Perforce backend simply stopped scaling, and a new backend was written (named "piper", but not the "piper" most non-Googlers think of). And once that was in place, assorted more "make it scale" tooling could be built on top of that (and, at that point, I stopped using the client-side Git, because CitC was simply That Good).
So, "from scratch" is a bit of a stretch.
It's more complicated than that. Initially, there was Perforce. Then, Google grew client-side tooling on top of that ("g4", rather than "p4"). Then, someone grew client-side Git on top of that (for reasons, good or bad). And at some point, the Perforce backend simply stopped scaling, and a new backend was written (named "piper", but not the "piper" most non-Googlers think of). And once that was in place, assorted more "make it scale" tooling could be built on top of that (and, at that point, I stopped using the client-side Git, because CitC was simply That Good).
So, "from scratch" is a bit of a stretch.
> this “vulnerability” is absurd in this context
Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities.
If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense.
I do agree with the author's note about providing a better way to provide feedback on severity reviews.
npm audit is better than no npm audit...telling people it's broken by design is going to discourage them from using it completely. smh.
Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities.
If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense.
I do agree with the author's note about providing a better way to provide feedback on severity reviews.
npm audit is better than no npm audit...telling people it's broken by design is going to discourage them from using it completely. smh.
I do think he makes a valid point towards the end though. create-react-app, and projects like it, get a lot of bug reports about these vulnerabilities. It's a lot of energy dedicated to something that ultimately isn't that important if you understand the context. I can understand the frustration.
Exactly. Why does it matter if it's absurd in this context? Just upgrade it to be on the safe side. Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea, even more so when you consider that the line between static and dynamic have gone blurry long ago.
>Just upgrade it to be on the safe side.
A big part of the problem is there is no reliably way to "just upgrade it" today in npm:
- `npm audit fix --force`, which is supposed to do that, is buggy and doesn't work
- There is no way to override a transitive dependency with npm (there is with Yarn though, so hopefully this feature will come to npm soon)
- Sometimes the fix in transitive dependency _also_ includes breaking changes (e.g. because it wasn't backported), and so updating it subtly breaks the logic
>Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea
I don't think databases can do it, but what I'd like to be able to do is to be able to provide advisory that the way _my package_ uses a concrete transitive dependency is not affected by that vulnerability. Because as the package owner I _do_ have that context. I understand there may be significant issues with this approach though!
A big part of the problem is there is no reliably way to "just upgrade it" today in npm:
- `npm audit fix --force`, which is supposed to do that, is buggy and doesn't work
- There is no way to override a transitive dependency with npm (there is with Yarn though, so hopefully this feature will come to npm soon)
- Sometimes the fix in transitive dependency _also_ includes breaking changes (e.g. because it wasn't backported), and so updating it subtly breaks the logic
>Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea
I don't think databases can do it, but what I'd like to be able to do is to be able to provide advisory that the way _my package_ uses a concrete transitive dependency is not affected by that vulnerability. Because as the package owner I _do_ have that context. I understand there may be significant issues with this approach though!
> * - There is no way to override a transitive dependency with npm (there is with Yarn though, so hopefully this feature will come to npm soon)*
I do not think this is a good idea as it allows consumer's of a library to utilize that dependency with untested, and unspecified transitive decencies What happens when a transitive dependency breaks the 1st level dependency? What a PITA that would be to try and fix.
I do not think this is a good idea as it allows consumer's of a library to utilize that dependency with untested, and unspecified transitive decencies What happens when a transitive dependency breaks the 1st level dependency? What a PITA that would be to try and fix.
> Because as the package owner I _do_ have that context. I understand there may be significant issues with this approach though!
But what if one of your contributors slips in a merge that uses the vulnerable code path of your dependency... Does this "not affected" marker still exist, and now you have vulnerable code? Does it disappear with each version?
What if someone maliciously adds a "not affected" marker? To a package they intend to exploit?
Edit: Again why the heck am I being downvoted?
But what if one of your contributors slips in a merge that uses the vulnerable code path of your dependency... Does this "not affected" marker still exist, and now you have vulnerable code? Does it disappear with each version?
What if someone maliciously adds a "not affected" marker? To a package they intend to exploit?
Edit: Again why the heck am I being downvoted?
> what if one of your contributors slips in a merge that uses the vulnerable code path of your dependency
If your threat model is a contributor submitting malicious code, your problem is not something npm audit will help with either way.
If a malicious actor is able to add the "not affected" marker, you have bigger problems.
The threat model you're talking about neither seems realistic, nor like something npm audit can help with. The attack vector of a contributor sneaking in malicious code is dealt with by only giving the commit bit to trusted people, and reviewing code yourself.
If your threat model is a contributor submitting malicious code, your problem is not something npm audit will help with either way.
If a malicious actor is able to add the "not affected" marker, you have bigger problems.
The threat model you're talking about neither seems realistic, nor like something npm audit can help with. The attack vector of a contributor sneaking in malicious code is dealt with by only giving the commit bit to trusted people, and reviewing code yourself.
> The threat model you're talking about neither seems realistic, nor like something npm audit can help with. The attack vector of a contributor sneaking in malicious code is dealt with by only giving the commit bit to trusted people, and reviewing code yourself.
How is this not realistic when it has already been seen in the npm ecosystem multiple times. For an example of this in the wild see the event-stream (crypto-mining trojan) fiasco: https://github.com/dominictarr/event-stream/issues/116
Dependencies are a target for exploit, your package may be safe now and then become unsafe in the future, either intentionally or unintentionally.
How is this not realistic when it has already been seen in the npm ecosystem multiple times. For an example of this in the wild see the event-stream (crypto-mining trojan) fiasco: https://github.com/dominictarr/event-stream/issues/116
Dependencies are a target for exploit, your package may be safe now and then become unsafe in the future, either intentionally or unintentionally.
The problem is, if a security vulnerability snaked past the maintainers of a project, what hope do I have as someone who consumes the package to a) catch it b) know how to fix it?
That's exactly the issue that npm-audit seeks to ameliorate. It's not perfect, but it's better than nothing.
I don't think anyone should be blindly upgrading anything. Doing so brings about its set of problems and in many cases makes things worse. This is particularly true in the case of Create React App where just upgrading dependencies will often break the application.
The last Create React App I worked on (~3 months ago) had over 500 "vulnerabilities" reported by npm/yarn audit. Most of the reported vulnerabilities were obviously junk. As the author noted, there's no need to report vulnerabilities in the same dependency in every path through the dependency graph. The noise made it very difficult to sift through the output for anything useful. Even then, I have my doubts about how applicable the results are because with tree shaking of an SPA, it's quite possible the vulnerable part of a dependency is never even used.
The last Create React App I worked on (~3 months ago) had over 500 "vulnerabilities" reported by npm/yarn audit. Most of the reported vulnerabilities were obviously junk. As the author noted, there's no need to report vulnerabilities in the same dependency in every path through the dependency graph. The noise made it very difficult to sift through the output for anything useful. Even then, I have my doubts about how applicable the results are because with tree shaking of an SPA, it's quite possible the vulnerable part of a dependency is never even used.
> Just upgrade it to be on the safe side
Upgrading a dependency can go anywhere from trivial to absolute nightmare. Usually somewhere in the middle where it takes time and effort to do right. A typical JS app nowadays has hundreds if not thousands of dependencies. I'd love to see a world where "just upgrade" is reasonable advice, but we are not there.
Upgrading a dependency can go anywhere from trivial to absolute nightmare. Usually somewhere in the middle where it takes time and effort to do right. A typical JS app nowadays has hundreds if not thousands of dependencies. I'd love to see a world where "just upgrade" is reasonable advice, but we are not there.
Or even undoable because it is a dependency of dependency, that vulnerable major version is no longer maintained, and your dependency heavily relies on specific feature of that major version.
At this point, what should you do now?
The author of dependency of dependency is probably not going to touch it because it is fixed in the new major version. The author of dependency is probably also not going to fix it instantly because it requires major rewrite.
At this point, what should you do now?
The author of dependency of dependency is probably not going to touch it because it is fixed in the new major version. The author of dependency is probably also not going to fix it instantly because it requires major rewrite.
Fork the repo and fix it yourself, ideally basing the change off the fix on the new major version if possible.
Yes, you now don't get vuln. notifications for the original repo, which is an issue in itself. It would be nice to mark a CVE as mitigated in your package.json and to also mark a resolution to still pick up CVEs from the original package. E.g
{
That is how we handle such issues in our team, we also review any forks at the start of every sprint to see if it has been resolved and we can remove the fork dependency.
If the vuln is valid and exploitable in your system, what other choice do you have? It's the pitfalls on depending on 3rd party packages. If you are using an old major version that now has a vuln that is only fixed in a newer version, NPM doesn't make that situation worse.
Caveat: These are my 5 minutes thoughts, I could probably do a better, more thorough write up.
Yes, you now don't get vuln. notifications for the original repo, which is an issue in itself. It would be nice to mark a CVE as mitigated in your package.json and to also mark a resolution to still pick up CVEs from the original package. E.g
{
"dependencies": {
badRepo: "1.0.0" // has a dependency that is vulnerable
},
"resolutions": {
"badRepo/**/dependency": "git://github.com/org/dependency" // fixed, but now won't report new cves from the original badRepo/**/dependency package. Would be good to specify that we still want reports for the original repo
That is how we handle such issues in our team, we also review any forks at the start of every sprint to see if it has been resolved and we can remove the fork dependency.
If the vuln is valid and exploitable in your system, what other choice do you have? It's the pitfalls on depending on 3rd party packages. If you are using an old major version that now has a vuln that is only fixed in a newer version, NPM doesn't make that situation worse.
Caveat: These are my 5 minutes thoughts, I could probably do a better, more thorough write up.
> Why does it matter if it's absurd in this context?
If I wanted shitty false alarms about bogus security issues, I'd get a PCI-DSS audit.
If I wanted shitty false alarms about bogus security issues, I'd get a PCI-DSS audit.
Have you ever worked with a complex project? Upgrading isn't easy if you have a moderate number of dependencies, also you risk introducing bugs for fixing a problem that really doesn't impact you. It doesn't make much sense to me...
This is a general problem with many security scanning tools, and when a security team is empowered to give deadlines to fix any issue they report, leads to much frustration and poor relations in teams.
Imagine if you had 3 days to fix the regex DoS issue shown there, screw your release freeze and your current sprint plans, and you have the real working environment in some companies.
I've also heard reports of people trying to claim bug bounties for similar reports, or security vendors that run automated tools that detect for issues of similar (lack of) value.
Imagine if you had 3 days to fix the regex DoS issue shown there, screw your release freeze and your current sprint plans, and you have the real working environment in some companies.
I've also heard reports of people trying to claim bug bounties for similar reports, or security vendors that run automated tools that detect for issues of similar (lack of) value.
Among other things, it illustrates the insufficiency of a single numeric value for assessing 'badness', which in turn masks a management issue.
CVEs try to supplement with flags for remotely exploitable, etc. but it still intentionally leaves a lot of space for interpretation, which is necessary for any normal enterprise.
The problem comes in when analysts (or their managers) interpret inflexibly, without appropriate technical context, or without understanding business impact and tradeoffs.
If you look at the workflow, it is hard to close the loop from engineering or IT back to security. We need a set of controls for secops departments' output relevance and departmental interoperation.
CVEs try to supplement with flags for remotely exploitable, etc. but it still intentionally leaves a lot of space for interpretation, which is necessary for any normal enterprise.
The problem comes in when analysts (or their managers) interpret inflexibly, without appropriate technical context, or without understanding business impact and tradeoffs.
If you look at the workflow, it is hard to close the loop from engineering or IT back to security. We need a set of controls for secops departments' output relevance and departmental interoperation.
This is actively going on where I work. Granted, it’s a financial company, so they take security pretty serious. During our last release, we had to go through 3 different teams, all doing different security scans.
One if them is scanning all your dependencies, and its so frustrating. Because that team obviously has no idea what any of the dependencies do or how they’re being used. All they see is a red flag, and tell you to fix it. Good luck when they tell you this days before a release, and a week after the code is frozen. They’ll just block your release without a second thought.
Funnily, in our last release, some of our NPM packages were flagged as a risk, obviously without explanation. The thing was, these packages where dependencies of another package. Obviously we can’t go around updating open source code, just because the security team in our company told us.
One if them is scanning all your dependencies, and its so frustrating. Because that team obviously has no idea what any of the dependencies do or how they’re being used. All they see is a red flag, and tell you to fix it. Good luck when they tell you this days before a release, and a week after the code is frozen. They’ll just block your release without a second thought.
Funnily, in our last release, some of our NPM packages were flagged as a risk, obviously without explanation. The thing was, these packages where dependencies of another package. Obviously we can’t go around updating open source code, just because the security team in our company told us.
>Obviously we can’t go around updating open source code, just because the security team in our company told us.
This isn't obvious to me. Most open source projects accept contribution from others.
This isn't obvious to me. Most open source projects accept contribution from others.
> This isn't obvious to me. Most open source projects accept contribution from others.
Of course they do, and I'm more than happy to help with open source projects. My point was that, we can't do it, just because a security review at my company says so. It's not just as simple as updating the version of the affected package, there's also testing involved, potentially fixing issues due to using a later version. This would almost be a full-time job.
Of course they do, and I'm more than happy to help with open source projects. My point was that, we can't do it, just because a security review at my company says so. It's not just as simple as updating the version of the affected package, there's also testing involved, potentially fixing issues due to using a later version. This would almost be a full-time job.
Who says it's important to that maintainer that their project used as a build time dependency has a vulnerability if provided untrusted user input?
What if it requires major upgrades of their framework or toolchain they don't want someone doing drive by?
What if they require a CLA that your legal team won't let you sign?
What if it requires major upgrades of their framework or toolchain they don't want someone doing drive by?
What if they require a CLA that your legal team won't let you sign?
Writing my comments in the snarky tone of the article.
So the article boils down to “a bunch of these vulnerabilities aren’t applicable to my app which is built using a specific NPM package”.
Congratulations. Welcome to the world of practical information security.
As a security engineer, we’re lucky if your favorite package manager even associates vulnerability information with your packages. Never mind that you’re pulling in code at build time from who-only-knows-where that almost certainly wasn’t security reviewed. But that’s for another post.
Now you have a package manager that is kind enough to tell you that there might be a vulnerability, and you’re upset because NPM did not have specific logic to understand the mechanics of one of the packages it manages? And the upshot is that you have to apply judgment and attention to each notification? Is that a tear in my eye- no, wait, it’s just an eyelash.
How many packages are there? I’m sure the NPM guys have nothing better to do than to build context awareness for every package in their repository.
In all seriousness, I would love to see context awareness in vulnerability reporting. But expecting a package manager to understand that because of your specific choice of framework, that the DoS could only be conducted by an admin of your app, seems unreasonable to me.
So the article boils down to “a bunch of these vulnerabilities aren’t applicable to my app which is built using a specific NPM package”.
Congratulations. Welcome to the world of practical information security.
As a security engineer, we’re lucky if your favorite package manager even associates vulnerability information with your packages. Never mind that you’re pulling in code at build time from who-only-knows-where that almost certainly wasn’t security reviewed. But that’s for another post.
Now you have a package manager that is kind enough to tell you that there might be a vulnerability, and you’re upset because NPM did not have specific logic to understand the mechanics of one of the packages it manages? And the upshot is that you have to apply judgment and attention to each notification? Is that a tear in my eye- no, wait, it’s just an eyelash.
How many packages are there? I’m sure the NPM guys have nothing better to do than to build context awareness for every package in their repository.
In all seriousness, I would love to see context awareness in vulnerability reporting. But expecting a package manager to understand that because of your specific choice of framework, that the DoS could only be conducted by an admin of your app, seems unreasonable to me.
Typical security engineer opinion. This "feature" is costing the industry tens, perhaps hundreds of million of dollars in wasted salary hours but it's worth it to you in the off-chance .1% are even potentially affected. Because only your job matters.
The point is that if the feature is going to constantly produce false positives, it's useless. I concur with the author, I never check those warnings anymore.
It’s not a false positive, as described. It’s “mitigated by environment”. The vulnerability is real. The severity is arguably too high.
People throw around “false positive” as a catch-all for “I don’t care about this”. But there are a number of distinct reasons one might not care:
- the scanner is wrong (e.g. there’s a code bug in the scanner like detecting “printed” instead of “sprintf”.
- the output is wrong because the vulnerability isn’t a vulnerability anytime, anywhere under any circumstances
- the scanner is correct, but environment or mitigation’s mean it doesn’t apply to me or the severity is wrong in my environment (this is the case here)
- the scanner is correct, but is giving me output I don’t care about (eg I want to filter for only high/critical but I can’t)
- there is so much output that I can’t pay attention to all of it; it’s so overwhelming that I can’t stand to look at it
Many security products have problems with output that is too verbose. This seems like a trivial problem to work around here; after you’ve triaged that a particular vulnerability doesn’t apply to a particular project, then filter it out with grep -v (our put a bunch of such lines in a bash script and always pipe npm audit output to the script.
Also, I sympathize with concerns that the vulnerability reporter perhaps scored the vulnerability too high. But there’s no perfect solution for that, and I’d rather be aware of a vuln and choose to ignore it, than not be aware at all.
People throw around “false positive” as a catch-all for “I don’t care about this”. But there are a number of distinct reasons one might not care:
- the scanner is wrong (e.g. there’s a code bug in the scanner like detecting “printed” instead of “sprintf”.
- the output is wrong because the vulnerability isn’t a vulnerability anytime, anywhere under any circumstances
- the scanner is correct, but environment or mitigation’s mean it doesn’t apply to me or the severity is wrong in my environment (this is the case here)
- the scanner is correct, but is giving me output I don’t care about (eg I want to filter for only high/critical but I can’t)
- there is so much output that I can’t pay attention to all of it; it’s so overwhelming that I can’t stand to look at it
Many security products have problems with output that is too verbose. This seems like a trivial problem to work around here; after you’ve triaged that a particular vulnerability doesn’t apply to a particular project, then filter it out with grep -v (our put a bunch of such lines in a bash script and always pipe npm audit output to the script.
Also, I sympathize with concerns that the vulnerability reporter perhaps scored the vulnerability too high. But there’s no perfect solution for that, and I’d rather be aware of a vuln and choose to ignore it, than not be aware at all.
[deleted]
In my opinion, most of these vulnerabilities are completely useless. A React developer performing a DoS on themselves or their build servers with a complex regex is theoretically a problem, but in practice I'd rate it below "low" importance. If a developer wants to ruin a company with huge building costs, why not add a cryptominer to the build process? Why not just spawn hundreds of build tasks? Why bother with generating complex regexes as an attack vessel when you have access to the code?
I'm not even sure if I'd classify it as a security bug at this point, I'd just classify these examples as configuration options to be aware of. Document and ignore.
With the explosion in Javascript packages to implement trivial behaviours, the NPM dependency hell and all the other cruft that "modern" frontend development requires, there are more important security issues to monitor.
Many real issues exist, but their classification is ridiculously flawed. Every security researcher tries to mark their vulnerability up to be the next Eternal Blue of Javascript development, but these vulnerabilities rarely matter.
I think a much bigger problem from a security standpoint is not necessarily the quality of these bug reports, but the sheer number of dependencies even a basic React project has these days. Supply chain attacks are real and the javascript world can do with some dependency purging. Any of the 200 single-line Javascript libraries can be compromised at any point and infect developers all over the world the next time they update their dependencies.
The way people "just" seem to add new dependencies to projects terrifies me. left-pad hit the frontend world and changed nothing. twilio-npm infected developer machines, and create-react-app imports over 1500 libraries as if it's the most normal thing in the world. It's absolute madness.
I have no idea how to fix all of this. It's only a matter of time before someone replicates the research people have done in uploading packages with typos in the name (which have reached into big, famous companies like Apple and Facebook) and start doing some serious damage. Maybe it's already happening: a package having been taken over for scraps, the owners lying in wait while everyone downloads and updates their React/Svelte/etc. packages, ready with their cryptolockers to strike frontend devs and build servers the world over with the push of a button. I wouldn't be surprised, not in the slightest.
I'm not even sure if I'd classify it as a security bug at this point, I'd just classify these examples as configuration options to be aware of. Document and ignore.
With the explosion in Javascript packages to implement trivial behaviours, the NPM dependency hell and all the other cruft that "modern" frontend development requires, there are more important security issues to monitor.
Many real issues exist, but their classification is ridiculously flawed. Every security researcher tries to mark their vulnerability up to be the next Eternal Blue of Javascript development, but these vulnerabilities rarely matter.
I think a much bigger problem from a security standpoint is not necessarily the quality of these bug reports, but the sheer number of dependencies even a basic React project has these days. Supply chain attacks are real and the javascript world can do with some dependency purging. Any of the 200 single-line Javascript libraries can be compromised at any point and infect developers all over the world the next time they update their dependencies.
The way people "just" seem to add new dependencies to projects terrifies me. left-pad hit the frontend world and changed nothing. twilio-npm infected developer machines, and create-react-app imports over 1500 libraries as if it's the most normal thing in the world. It's absolute madness.
I have no idea how to fix all of this. It's only a matter of time before someone replicates the research people have done in uploading packages with typos in the name (which have reached into big, famous companies like Apple and Facebook) and start doing some serious damage. Maybe it's already happening: a package having been taken over for scraps, the owners lying in wait while everyone downloads and updates their React/Svelte/etc. packages, ready with their cryptolockers to strike frontend devs and build servers the world over with the push of a button. I wouldn't be surprised, not in the slightest.
As a security engineer, do you feel like the addition of npm audit is a net positive for the security of the npm ecosystem?
Net positive for awareness. Not sure I like the “force update with dependency that might break me”. NPM audit needs the ability to filter by severity and the ability for users to tune vuln severities per project.
It's a tricky problem to solve.
Ideally you'd want to show only relevant alerts, but... how? You'd need to know which kind of errors are relevant for a particular project, but that'd require solving the halting problem. This is made much worse by that it's JS.
Some libraries have an enormous complexity and attack surface. Take a database interface -- there probably is a vulnerability in some obscure corner the typical person may not even know exists.
I think though at the very least some improvement could be made by better priorization and categorization. DoS by exploiting a regex parser isn't that big of a deal if your project is just getting started, but an exploit allowing arbitrary code execution would still be.
Ideally you'd want to show only relevant alerts, but... how? You'd need to know which kind of errors are relevant for a particular project, but that'd require solving the halting problem. This is made much worse by that it's JS.
Some libraries have an enormous complexity and attack surface. Take a database interface -- there probably is a vulnerability in some obscure corner the typical person may not even know exists.
I think though at the very least some improvement could be made by better priorization and categorization. DoS by exploiting a regex parser isn't that big of a deal if your project is just getting started, but an exploit allowing arbitrary code execution would still be.
You don't have to go all out doing exhaustive dynamic analysis. Data flow analysis gets rid of 99% of the most popular bugs (injection, validation, defaults, etc.). GitHub CodeQL can do this, and produces much better results than any static analysis tool I've ever used. Feeding this data back into npm (owned by GH) is just the next step.
You just need a way for a package maintainer to flag a vulnerability in a dependency as a non-issue that does not affect that package's use of the dependency.
In Dan's twitter thread, he calls this out as a viable solution.
In Dan's twitter thread, he calls this out as a viable solution.
We basically run it in CI... and then allow it to fail without failing the build. ¯\_(ツ)_/¯
It seems like a lot of this has been designed for Node (backend) development, whilst ignoring the fact that NPM is probably used more heavily for front-end development at this point.
It seems like a lot of this has been designed for Node (backend) development, whilst ignoring the fact that NPM is probably used more heavily for front-end development at this point.
Yep, some vulnerable package isn't even in the compile output. How a dev server that only binds to 127.0.0.1 a serious DOS problem? Who on the earth will want to DOS that?
[deleted]
For those hoping to run npm audit in your CI/CD pipeline, I recommend this tool from IBM: https://github.com/IBM/audit-ci
In highly regulated industries, shipping code flagged as having a vuln without a manual approval could be a liability.
This wrapper around npm takes an allowlist argument, and our procedure is for an engineer to review the failing build, determine if the vulnerability (ugh, usually regex ddos or prototype pollution) is present in code that runs only at build time with trusted inputs, only on the client which is by definition untrusted, or in our webserver which takes in untrusted input.
As long as it's either of the first two, we document it in a commit and comment and redeploy. It's annoying, but it's far better than npm audit forcing a fix.
In highly regulated industries, shipping code flagged as having a vuln without a manual approval could be a liability.
This wrapper around npm takes an allowlist argument, and our procedure is for an engineer to review the failing build, determine if the vulnerability (ugh, usually regex ddos or prototype pollution) is present in code that runs only at build time with trusted inputs, only on the client which is by definition untrusted, or in our webserver which takes in untrusted input.
As long as it's either of the first two, we document it in a commit and comment and redeploy. It's annoying, but it's far better than npm audit forcing a fix.
Compare with npm-audit-resolver in terms of how the ignores are defined. It's important to not be too vague when ignoring things.
Let me plug this as it contains a lot of references https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Meanwhile I'll try to get someone from IBM involved in the OpenJSF collab space
Let me plug this as it contains a lot of references https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Meanwhile I'll try to get someone from IBM involved in the OpenJSF collab space
Indeed, npm is not aware of the context of the vulnerabilities. That does not invalidate them, however, and mean they should be hidden. I've worked in offensive security for quite long enough (been a dev for 10+ years before) to tell you that your context, or the most common one, isn't all that exist and someone's going use it the way it makes the app vulnerable. Based on the article's example, someone's going to build an app that builds an app.
A vulnerability is a vulnerability, whether it applies to your context or not. A metaphor might be: "The passenger door is broken on my car, but I'm the only one to use it". Seriously, who, in their right mind, is going to argue that the door isn't broken?
- If your dependencies have security vulnerabilities, apply the updates.
- If you cannot update because there's no fix available, let your org or you assess the risk and go from there.
- If you cannot update because it breaks your app, {find a replacement, fix it yourself, let your org or you assess the risk and from from there}.
A sensible org has a process that freezes releases until known security issues are fixed. Freezes can also be opposed by devs and are evaluated on a case-by-case basis (because sometimes they are not relevant to the product, or someone steps up to take the blame for incidents and the org agrees).
We might not like it because it disturbs the "flow", but it's just part of the engineering process. More to the point though, why not take this opportunity to teach newcomers how to code properly, pick well-engineered and -written programs, and handle this vulnerability management process altogether? In any case, I hope newcoming-dev is not going to push to prod anytime soon. ;)
edit: formatting
A vulnerability is a vulnerability, whether it applies to your context or not. A metaphor might be: "The passenger door is broken on my car, but I'm the only one to use it". Seriously, who, in their right mind, is going to argue that the door isn't broken?
- If your dependencies have security vulnerabilities, apply the updates.
- If you cannot update because there's no fix available, let your org or you assess the risk and go from there.
- If you cannot update because it breaks your app, {find a replacement, fix it yourself, let your org or you assess the risk and from from there}.
A sensible org has a process that freezes releases until known security issues are fixed. Freezes can also be opposed by devs and are evaluated on a case-by-case basis (because sometimes they are not relevant to the product, or someone steps up to take the blame for incidents and the org agrees).
We might not like it because it disturbs the "flow", but it's just part of the engineering process. More to the point though, why not take this opportunity to teach newcomers how to code properly, pick well-engineered and -written programs, and handle this vulnerability management process altogether? In any case, I hope newcoming-dev is not going to push to prod anytime soon. ;)
edit: formatting
> A metaphor might be: "The passenger door is broken on my car, but I'm the only one to use it". Seriously, who, in their right mind, is going to argue that the door isn't broken?
I think a better metaphor might be, the button on your key fob that opens the trunk doesn't work, so you have to open the trunk using a physical lever. Every time you start the car, a loud warning siren sounds, and a red message appears on the dashboard to tell you that there is a "high impact" problem with your car, and you need to take it to be serviced. If you were merely the owner of the car, and other people also had to drive it, you might understandably be the target of several complaints about this "high impact" problem.
I think a better metaphor might be, the button on your key fob that opens the trunk doesn't work, so you have to open the trunk using a physical lever. Every time you start the car, a loud warning siren sounds, and a red message appears on the dashboard to tell you that there is a "high impact" problem with your car, and you need to take it to be serviced. If you were merely the owner of the car, and other people also had to drive it, you might understandably be the target of several complaints about this "high impact" problem.
A lot of these cases that npm reports are denial of service vulnerabilities (and marked high risk!). I just tried it on a project I have, and 11 out of 15 are DOS vulnerabilities in code that I run locally. When the normal user is using a project only locally, and the issue is DOS, it's hard to argue "but maybe someone will eventually put it online" and therefore I need to drop what I'm doing and patch my dependencies. (Yes, sometimes that would be the only way to satisfy npm, since the semver rules prevent it from fixing things automatically.)
I agree.
Ever seen apps which are basically impossible to patch because devs have ignored patching for so long that it’s basically impossible to version bump things sanely? I have.
Ever seen apps which are basically impossible to patch because devs have ignored patching for so long that it’s basically impossible to version bump things sanely? I have.
I've been saying this for years, especially to useless "prototype polution" notices being reported by npm audit. In a project where this "pollution" only happens on the nodejs side in our build tools, they are meaningless if our output is a browser JS bundle.
It sounds like npm needs a mitigated and irrelevant flag, these flags should include an explanation field.
Security teams would also have to accept this as a solved/fix status.
For projects you own you'd have to flag each dependency path though, because for example, one dependency may not have the input for the regex exposed to the end user, while another dependency could.
Maintainers of libraries should also flag the security issues, and an issue with these two statuses on them wouldn't be raised by default. Options should be available to list them though for auditing.
For more security critical teams/projects, a per project setting to alert about any issues the maintainers have flagged irrelevant or mitigated and you'd have to accept them before it it would stop alerting about them.
For projects you own you'd have to flag each dependency path though, because for example, one dependency may not have the input for the regex exposed to the end user, while another dependency could.
Maintainers of libraries should also flag the security issues, and an issue with these two statuses on them wouldn't be raised by default. Options should be available to list them though for auditing.
For more security critical teams/projects, a per project setting to alert about any issues the maintainers have flagged irrelevant or mitigated and you'd have to accept them before it it would stop alerting about them.
As someone who only had just gotten into front end programming after years of backend work, npm has been a nightmare. I haven't experienced the same level of frustration with other package managers (pip, cargo, go mod, etc) as I have with npm.
Is yarn the better option? What is our path forward?
Is yarn the better option? What is our path forward?
I think the problem is not really NPM it's the way that mainstream frontend development requires very wide and deep dependencies. If you stick to using smaller frameworks and libraries without heavy build tools then it's not much of a problem.
I basically do no frontend development, partly due to tools npm and the current frameworks, I simply cannot wrap my head around it. I do help run a few modern javascript application however. Even a minimal app will pull in 1000+ dependecies, and I think that’s the problem.
It simply don’t happen in Python, Go or Rust (or even Java) because the languages comes with a rich standard library. Javascript comes with just the basics, everything else is a dependency. It’s not uncommon for people to audit their dependencies in Python or Go, but you pull in maybe 10 or 20. A basic Javascript app easily pull in 100 times that, how are you suppose to deal with that?
It simply don’t happen in Python, Go or Rust (or even Java) because the languages comes with a rich standard library. Javascript comes with just the basics, everything else is a dependency. It’s not uncommon for people to audit their dependencies in Python or Go, but you pull in maybe 10 or 20. A basic Javascript app easily pull in 100 times that, how are you suppose to deal with that?
> Rust
This has not been my experience at all. Most Rust developers unfortunately seem quite happy to adopt modern programming bad practices. (There are many exceptions, of course.)
Example: I went with the first Rust program I could find installed on my computer, tealdeer. It's a dead simple program: run `tldr progname` and it will print a handful of examples of how to run `progname` in your terminal. Run `tldr --update` and it will download the latest version of the database containing these examples from a web server.
To build this extremely basic CLI program, I need ONE HUNDRED AND NINETEEN distinct crates.
This has not been my experience at all. Most Rust developers unfortunately seem quite happy to adopt modern programming bad practices. (There are many exceptions, of course.)
Example: I went with the first Rust program I could find installed on my computer, tealdeer. It's a dead simple program: run `tldr progname` and it will print a handful of examples of how to run `progname` in your terminal. Run `tldr --update` and it will download the latest version of the database containing these examples from a web server.
To build this extremely basic CLI program, I need ONE HUNDRED AND NINETEEN distinct crates.
I've been having a good experience with pnpm lately.
It's not a silver bullet but it addresses some pain points with npm/yarn.
same xd, npm even managed to make me appreciate maven and gradle
I think that there's difference between Java and JS landscapes.
Java has rich standard library. Some would say - too rich. But anyway - very few people would need customized collections, standard library covers all the needs. And where it does not cover all the needs, there are few commonly accepted additions like apache-commons or google guava. So that one solves `leftpad`-like issues.
Another difference is that Java is old. Most needs were covered by many libraries and few libraries survived which are good enough. It's again some commonly accepted wisdom, so you don't really need to search for many options. You have one good enough option that you'll use and move on.
So it's not about maven/gradle vs npm, it's more about ecosystems. I don't think that porting maven to JS world would change anything.
Java has rich standard library. Some would say - too rich. But anyway - very few people would need customized collections, standard library covers all the needs. And where it does not cover all the needs, there are few commonly accepted additions like apache-commons or google guava. So that one solves `leftpad`-like issues.
Another difference is that Java is old. Most needs were covered by many libraries and few libraries survived which are good enough. It's again some commonly accepted wisdom, so you don't really need to search for many options. You have one good enough option that you'll use and move on.
So it's not about maven/gradle vs npm, it's more about ecosystems. I don't think that porting maven to JS world would change anything.
PNPM[1] has became a must-use for me in every node project I work on now.
[1]: https://pnpm.io/
[1]: https://pnpm.io/
It doesn't solve everything, but we use npm-audit-resolver[0] and it's.. workable. It presents you vulnerabilities, offers to fix (upgrade the nested dependency) if a version exists that meets all the constraints, and gives you an option to ignore for a week/month/forever if no fix exists. Those decisions (including fixes, which is a bit silly) get recorded in a JSON file in source control. For each group of ignores we add a link to the relevant Github issue where it's been reported, so if the ignore time we chose expires we can quickly go and see what the status is.
There are still problems:
- The decisions file gets unweildy, mainly because every time it fixes something it writes to the file. You probably only care about ignores. It's also append only, though you could manually clear it down sometimes.
- It always defaults to fixing at the deepest level, which is.. not ideal for NPM. On my machine (a not very old Macbook Pro) NPM simply can't update a dependency 20 layers deep in the tree, ie `npm update nested-dependency-from-hell --depth 20` will eventually time out and won't fix anything. So you have to manually crawl up tree yourself and find the thing that can be updated - or just ignore it until the thing right at the top of the tree gets updated.
I'm not surprised to see Dan posting this though. I agree with everything he said, so I don't mean this as an attack, but a lot of the time the thing at the top of the tree we're waiting for an update on is create-react-app. It must be incredibly annoying how many Github issues get opened on that repo every time there's a new NPM advisory on some 20-dependencies-deep parser it uses for something or other.
I do like the suggested fix that a maintainer can use their knowledge of the specific usage to say the vulnerability doesn't apply. Often in these threads there's a perfectly good explanation of why it isn't a real issue, and then people come back with "Okay but can you please update it anyway because I'm forced to audit and my security team/CI are yelling at me".
[0] https://www.npmjs.com/package/npm-audit-resolver
There are still problems:
- The decisions file gets unweildy, mainly because every time it fixes something it writes to the file. You probably only care about ignores. It's also append only, though you could manually clear it down sometimes.
- It always defaults to fixing at the deepest level, which is.. not ideal for NPM. On my machine (a not very old Macbook Pro) NPM simply can't update a dependency 20 layers deep in the tree, ie `npm update nested-dependency-from-hell --depth 20` will eventually time out and won't fix anything. So you have to manually crawl up tree yourself and find the thing that can be updated - or just ignore it until the thing right at the top of the tree gets updated.
I'm not surprised to see Dan posting this though. I agree with everything he said, so I don't mean this as an attack, but a lot of the time the thing at the top of the tree we're waiting for an update on is create-react-app. It must be incredibly annoying how many Github issues get opened on that repo every time there's a new NPM advisory on some 20-dependencies-deep parser it uses for something or other.
I do like the suggested fix that a maintainer can use their knowledge of the specific usage to say the vulnerability doesn't apply. Often in these threads there's a perfectly good explanation of why it isn't a real issue, and then people come back with "Okay but can you please update it anyway because I'm forced to audit and my security team/CI are yelling at me".
[0] https://www.npmjs.com/package/npm-audit-resolver
[deleted]
>> Five false alarms wouldn’t be too bad.
>> Unfortunately, there are hundreds.
This is primarily a result of the absurd number of dependencies NPM encourages (requires?) people to use. The duplicates are also there in part because of the large number of dependencies and should not be shown more than once by the tool.
Stop building projects with an absurdly large dependency tree, this is just one problem that results from it.
>> Unfortunately, there are hundreds.
This is primarily a result of the absurd number of dependencies NPM encourages (requires?) people to use. The duplicates are also there in part because of the large number of dependencies and should not be shown more than once by the tool.
Stop building projects with an absurdly large dependency tree, this is just one problem that results from it.
I’d imagine that in most projects, the bulk of the dependency is due to dev tooling. I don’t think it’s fair to optimize for small dependency trees when setting up your buildchain – otherwise you’re precluding any usage of create-react-app or Next or whatever development platform. This problem is further compounded by the fact that those tools encourage including dev dependencies as regular dependencies, since the output is compiled anyway.
The answer here is probably some kind of static analysis to know which packages end up shipping in the actual bundle to users. I think Dan referenced some work in that regard.
The answer here is probably some kind of static analysis to know which packages end up shipping in the actual bundle to users. I think Dan referenced some work in that regard.
>> The answer here is probably some kind of static analysis...
So even more tools?
So even more tools?
npm is a bit nuts on its own. I started learning react this year and the course I'm taking had me install that create react app module or similar. It dragged in 1700 dependencies, and the folder for a hello world app was almost 90mb iirc. How can you possibly pretend you have any control over your app or it’s security in that situation?
The reason it’s so big is because it’s your build chain. I’ve always found this criticism of JS annoying – you’ve also got to pull in a few hundred MB of tooling for C++, Java, or Rust — it’s just that in the JS case, the “compiler” is usually per project.
In my experience, Java and Rust have the exact same problem. The nature of the dependencies is different, though; in Rust or Java, you'll have dependencies that handle a protocol, set up something like TLS, manage state, that kind of thing. In Javascript, you'll have a single dependency that provides a single function that does a recursive file search, or a dependency like is-number that checks if a variable is a number.
Javascript seems to prefer millions of tiny dependencies of thousands of larger libraries, which is a choice that can be defended. The difference is not necessarily one in lines of code or binary size, but one in amount of vendors trusted. Many libraries also handle trivial code that (in my opinion) should be part of the programming language or basic developer knowledge already. The is-something packages that fill Javascript dependency trees to the brim can only be considered as failings of the language in my opinion.
As a developer, I trust parties like webpack, gulp, and Facebook, but I haven't heard about jonschlinkert (nothing against him, just a random name I picked) and I don't know who maintains is-number, is-path-cw, is-path-in-cwd, is-path-inside or path-is-inside and how reliable they are. All of these dependencies seem like excellent methods in a library, but they all could've been part of a single dependency no more than 60 lines of code in length. Many NPM packages feel less like libraries and more like automated StackOverflow answers. Adding a vendor to your supply chain for just 40 lines of open source code is just inefficient; why risk trusting yet another vendor to not inject malware in the future like this?
The Java world has some popular names like Apache, Google and Jetbrains that maintain large libraries so it's easy to build a chain of trust. Rust is moving the Javascript way, with hundreds of megabytes of dependencies from thousands of individual repositories, but at least most of its packages add something nontrivial.
C++ doesn't have a package manager, at least not in the same way other languages do. C++ libraries usually come from very specific toolkits or single sources (like Linux package managers). There's tons of packages for C++ development, but all of them are kept up to date by a single organisation on my machine.
Javascript seems to prefer millions of tiny dependencies of thousands of larger libraries, which is a choice that can be defended. The difference is not necessarily one in lines of code or binary size, but one in amount of vendors trusted. Many libraries also handle trivial code that (in my opinion) should be part of the programming language or basic developer knowledge already. The is-something packages that fill Javascript dependency trees to the brim can only be considered as failings of the language in my opinion.
As a developer, I trust parties like webpack, gulp, and Facebook, but I haven't heard about jonschlinkert (nothing against him, just a random name I picked) and I don't know who maintains is-number, is-path-cw, is-path-in-cwd, is-path-inside or path-is-inside and how reliable they are. All of these dependencies seem like excellent methods in a library, but they all could've been part of a single dependency no more than 60 lines of code in length. Many NPM packages feel less like libraries and more like automated StackOverflow answers. Adding a vendor to your supply chain for just 40 lines of open source code is just inefficient; why risk trusting yet another vendor to not inject malware in the future like this?
The Java world has some popular names like Apache, Google and Jetbrains that maintain large libraries so it's easy to build a chain of trust. Rust is moving the Javascript way, with hundreds of megabytes of dependencies from thousands of individual repositories, but at least most of its packages add something nontrivial.
C++ doesn't have a package manager, at least not in the same way other languages do. C++ libraries usually come from very specific toolkits or single sources (like Linux package managers). There's tons of packages for C++ development, but all of them are kept up to date by a single organisation on my machine.
Other libraries are usually self contained though, unlike JS libraries which contains dependencies and then those dependencies has dependencies etc. Probably has to do with JS lacking a standard library so it is a pain to do anything without including dependencies.
That’s hardly true. Some standard libs are smaller, but in most other languages I can think of there isn’t the complete dependency hell. I want to install one package, the 50 dependencies or their ancestors create an audit he’ll (is the package secure, so I trust the developer to not inject malware, abandon the package, or add more dependencies), etc.
My point was that if you're comparing JS dependencies to other languages, you need to include their compilers too, since e.g. TypeScript projects depend on `tsc`. If you include the size of C/C++/Rust/Java/etc. compilers, I'm sure you'll find 50mb+ of dependencies. You're right that it's self-contained though (to be fair, so is `tsc` – most projects could shed a lot of dependencies by abandoning Babel in favor of `tsc` or esbuild).
Sure but gcc, clang and python have had a substantial amount of review. There’s no assurance that similar occurs in the non ecosystem.
Yeah, that’s the biggest issue, IMO.
Even a simple app with only React as a production requirement will have dozens of issues a month.
There are some packages that don’t have as many dependencies such as Typescript or Prettier, but that’s not enough, since the most popular bundlers have hundreds of of dependencies.
No matter how careful you are, you get flooded by security issues.
Even a simple app with only React as a production requirement will have dozens of issues a month.
There are some packages that don’t have as many dependencies such as Typescript or Prettier, but that’s not enough, since the most popular bundlers have hundreds of of dependencies.
No matter how careful you are, you get flooded by security issues.
Just curious, have you tried vite with react? It uses esbuild, which might result in a heavier dependency folder, but I also wonder if getting rid of webpack removes a lot of the CRA dependencies.
So I don't work as a security professional but what I remember from IT-sec class in uni is that in order to craft an exploit you need to be vulnerable and the vulnerability needs to be exploitable.
If I put a database with default credentials on the internet, there is both a vulnerability and it is exploitable. Bad. If I run a database with default credentials on my dev machine, it is vulnerable, but not exploitable. Perfectly fine.
For real security work you also need to think about impact. Hacker dropping production database = we all lose our jobs. Co-worker connecting to my computer and dropping database as a joke = no real harm done.
So three things to think about: - Vulnerability - Exploitability - Impact
What I really don't like about npm audit is how it presents itself as "security tool" and how vulnerabilities are presented. "6 critical, 10 high vulnerabilities" with a red color screams "fix me now!!!". This is not fair to users because npm has no idea of either the exploitability or the impact of the vulnerability.
Why present users with a prompt "please fix me now!!" and not even mention that exploitability and impact need to be measured first? Seems like they forgot that prompt...
If I put a database with default credentials on the internet, there is both a vulnerability and it is exploitable. Bad. If I run a database with default credentials on my dev machine, it is vulnerable, but not exploitable. Perfectly fine.
For real security work you also need to think about impact. Hacker dropping production database = we all lose our jobs. Co-worker connecting to my computer and dropping database as a joke = no real harm done.
So three things to think about: - Vulnerability - Exploitability - Impact
What I really don't like about npm audit is how it presents itself as "security tool" and how vulnerabilities are presented. "6 critical, 10 high vulnerabilities" with a red color screams "fix me now!!!". This is not fair to users because npm has no idea of either the exploitability or the impact of the vulnerability.
Why present users with a prompt "please fix me now!!" and not even mention that exploitability and impact need to be measured first? Seems like they forgot that prompt...
Vulnerabilities reported to CVE carry a risk score (CVSS) that conveys this information in standard way.
It's really disappointing to hear an important member of the javascript community not maintaining their library and then blaming npm when people rightfully complain about it.
This is like getting mad at the guidebook for showing which plants are weeds when your neighbors complain that your unmaintained garden is full of weeds.
If Dan says "npm audit is a stain on the entire npm ecosystem", maybe it's safe to say that Create React App is a stain on the entire react ecosystem. The best time to maintain it was every single month of its existence because that's how you maintain software.
Facebook has abandoned Create React App. Dan stated that he intentionally does not maintain the project. Rather than complain about npm audit, they should give Create React App over to the community who actually use it instead of keeping it shambling along as a zombie with their name on it.
And if Facebook doesn't want to give it up, the best thing we can do as a community is to move on to any of the other great tools available that are actually maintained.
This is like getting mad at the guidebook for showing which plants are weeds when your neighbors complain that your unmaintained garden is full of weeds.
If Dan says "npm audit is a stain on the entire npm ecosystem", maybe it's safe to say that Create React App is a stain on the entire react ecosystem. The best time to maintain it was every single month of its existence because that's how you maintain software.
Facebook has abandoned Create React App. Dan stated that he intentionally does not maintain the project. Rather than complain about npm audit, they should give Create React App over to the community who actually use it instead of keeping it shambling along as a zombie with their name on it.
And if Facebook doesn't want to give it up, the best thing we can do as a community is to move on to any of the other great tools available that are actually maintained.
> This is like getting mad at the guidebook for showing which plants are weeds when your neighbors complain that your unmaintained garden is full of weeds.
Wouldn't it be more like getting mad at a guidebook which falsely claims that every plant in your garden is a weed, forcing you to second-guess all of its assertions and therefore wasting a lot of your time?
Wouldn't it be more like getting mad at a guidebook which falsely claims that every plant in your garden is a weed, forcing you to second-guess all of its assertions and therefore wasting a lot of your time?
It's like claiming that the weeds are not dangerous in your garden not that the guidebook got them wrong.
Excellent rundown of many problems I've encountered in recent versions of npm and node.
I've never ever got `audit fix` or `audit fix --force` to solve any of the mentioned vulnerabilities. Ever. I even relied on downloading every dependency one by one to find that there where other offending packages. I just gave up.
It's really useless and deceptive.
I've never ever got `audit fix` or `audit fix --force` to solve any of the mentioned vulnerabilities. Ever. I even relied on downloading every dependency one by one to find that there where other offending packages. I just gave up.
It's really useless and deceptive.
The title is "Broken by Design". Then the author proceeds to explain how the design is actually reasonable and meant to work ok, and how the problem lies in the relevance and context of the vulnerabilities reported.
That doesn't seem to be the meaning of "broken by design".
That doesn't seem to be the meaning of "broken by design".
It is just pure noise:
Add `audit=false` to your ~/.npmrc to disable it
Let's talk about solutions.
I'm late to the conversation here, responded on Twitter and went to sleep.
There's a push to address the npm audit situation. It's an initiative under the OpenJS Foundation. I kinda started the whole conversation by implementing a tool that makes it acceptable instead of ditching npm audit.
It's called npm-audit-resolver and I've written about it here https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Also check out the collab space and the tool itself https://github.com/openjs-foundation/pkg-vuln-collab-space https://www.npmjs.com/package/npm-audit-resolver
There's a push to address the npm audit situation. It's an initiative under the OpenJS Foundation. I kinda started the whole conversation by implementing a tool that makes it acceptable instead of ditching npm audit.
It's called npm-audit-resolver and I've written about it here https://dev.to/naugtur/do-you-need-help-with-your-npm-audit-...
Also check out the collab space and the tool itself https://github.com/openjs-foundation/pkg-vuln-collab-space https://www.npmjs.com/package/npm-audit-resolver
Agreed, you shouldn't see low risk security warnings by default, they're more appropriate for larger projects, with something to lose.
The [1] yarn package manager is much nicer to work with in many more ways.
[1] https://classic.yarnpkg.com/en/
The [1] yarn package manager is much nicer to work with in many more ways.
[1] https://classic.yarnpkg.com/en/
TFA identifies "high risk" false positives too, so this response doesn't seem to have anything to do with them problem as stated by TFA.
So currently the algorithm is... check (dev)Dependencies and descendent/transient dependencies to see if they exist in a security advisory database if they do, highlight and surface them to the user.
What are alternatives? A way to ignore or mark a dependency as safe? Could this be abused if an author can just mark a dependency as safe?
Or perhaps, actually analyze syntax with a tool like ESLint (parse -> AST -> validate) to check that dangerous parts of libraries are not in use? This solution comes with it's own complications. Who is authoring these validations?
Perhaps there are other strategies I'm not aware of.
What are alternatives? A way to ignore or mark a dependency as safe? Could this be abused if an author can just mark a dependency as safe?
Or perhaps, actually analyze syntax with a tool like ESLint (parse -> AST -> validate) to check that dangerous parts of libraries are not in use? This solution comes with it's own complications. Who is authoring these validations?
Perhaps there are other strategies I'm not aware of.
The the main problem is the fact that this audit happens with no context, and the audit results offer no information about the context an issue applies to either. Every issue should have a clear explanation about why and where it's an issue, and be tagged. Then we'd just need a way to hint npm what context a package will be used in, similarly to what we already do for devDependencies.
Also going through an audit result in a CLI isn't really the best experience. I wish I could just click a link and open up the report in a browser to drill down into issues.
Also going through an audit result in a CLI isn't really the best experience. I wish I could just click a link and open up the report in a browser to drill down into issues.
No it’s not. The main problem is the dependency tree hell. If an ancestor version bumps, you should probably version bump too, irrespective of exploitability.
Don’t like it? Try using more maintainable dependency trees.
Don’t like it? Try using more maintainable dependency trees.
The interaction is not the worse thing about npm audit. The security model of the tool has a big hole depending on how you use it: https://mulch.dev/blog/CVE-2020-5252-python-safety-vuln/
In essence, if you are scanning an environment that is already compromised, `npm audit` results can't be relied upon if you are running it in the same environment. It should be self-evident but I'm sure plenty of people use the tool this way.
In essence, if you are scanning an environment that is already compromised, `npm audit` results can't be relied upon if you are running it in the same environment. It should be self-evident but I'm sure plenty of people use the tool this way.
I recently implemented my own npm vulnerability audit tool for the CIO department of a major org - it just adds 'vulnerability' in red next to any npm-based project in their spreadsheet.
Difficult problem to solve. It seems like the reasonable solution here is to have auditing integrated with the build system—but there are so many wacky build systems for JavaScript.
I think this is a really great article and highlights an important weakness that modern security tools in this context have: they don't distinguish between vulnerabilities that can be triggered by malicious developer code, and vulnerabilities that can be triggered by malicious users/websites.
For a proper assessment, such differences need to be encoded in the security advisory, and the audit tool needs to analyze if the code is called at run time or build time, and then act accordingly.
For a proper assessment, such differences need to be encoded in the security advisory, and the audit tool needs to analyze if the code is called at run time or build time, and then act accordingly.
Anyone else read the title to mean the author had audited Npm and found the whole thing was broken by design?
Because I've felt that way for years.
Because I've felt that way for years.
Seems to me that if you don't have a pure functional language with tree-shaking, and only including existing (non-generated) code, you won't know if any (3rd party) code in particular will be called in compiled output, and you'll just have to guess and err on the side of paranoia Is there a fix?
Instead of marking dependencies as safe by the developer or by the end user, I wonder if the immediate parent can mark it as safe (because it has the appropriate context) and then npm audit can avoid reporting that "vulnerability" when it sees it.
I like his point about `npm audit --production` being a good way to cut down the noise. But Github doesn't seem to take dev dependencies into account when sending security alerts. I get emails about non-issues from them all the time.
I think they are just saying that context matters in security vulnerabilities, and npm audit doesn't have that context.
Well yes, correct, well done. By this metric, every security tool ever written is probably pointless.
Well yes, correct, well done. By this metric, every security tool ever written is probably pointless.
Note that `npm audit` runs _during every install_ so people who use it don't necessarily consciously understand what's happening. Many of them are beginners and have never used a security tool before (or even want to use it).
As mentioned in the article we ran into the same overload and just ended up running `npm audit --production` as part of our CI pipeline since that's what would be going out.
This is why something like living at head is important. If npm audit reports something you should just be able to upgrade to the latest version. Being stuck with old versions is not good. Sure a vulnerability might not effect you now, but what if someone on your team uses that dependency again in a way that ran be exploited, or what if a new vulnerability comes out that actually effects you. You will be stuck on an old version and have to struggle to update.
> Inlining dependencies kind of goes against the whole point of npm
I mean, this is why people love language with deep, solid standard libraries. You don't have a situation where a problem in a sub-sub-sub-sub-dependency provokes five different groups of people to all issue an update, one after another. You just upgrade your underlying installation to the latest patch version and continue.
Language ecosystems where a few lines of code constitutes a library fundamentally result in you being dependent on a huge number of outside people to cooperate on updates. That's what's broken. Not a tool which tells you that you have out-of-date libraries and, by the by, hooks into CVE databases.
OP should stop and consider whether it might be beneficial to inline some of those dependencies before dismissing it out-of-hand. If you never use the dependency in such a way as to present a real security risk... and you don't need feature updates from upstream, i.e. the software is fine as-is when you first incorporated the dependency... then why wouldn't you inline the dependency?
If anything, inlining the dependency will allow static code analyzers to point out all the parts of the dependency which you're not using (i.e. dead code) and eliminate it all. That way, even if the dependency were to be discovered to have a security fault, if the faulty code was in a section that you eliminated as dead code... then you don't have a security problem in the first place!
I mean, this is why people love language with deep, solid standard libraries. You don't have a situation where a problem in a sub-sub-sub-sub-dependency provokes five different groups of people to all issue an update, one after another. You just upgrade your underlying installation to the latest patch version and continue.
Language ecosystems where a few lines of code constitutes a library fundamentally result in you being dependent on a huge number of outside people to cooperate on updates. That's what's broken. Not a tool which tells you that you have out-of-date libraries and, by the by, hooks into CVE databases.
OP should stop and consider whether it might be beneficial to inline some of those dependencies before dismissing it out-of-hand. If you never use the dependency in such a way as to present a real security risk... and you don't need feature updates from upstream, i.e. the software is fine as-is when you first incorporated the dependency... then why wouldn't you inline the dependency?
If anything, inlining the dependency will allow static code analyzers to point out all the parts of the dependency which you're not using (i.e. dead code) and eliminate it all. That way, even if the dependency were to be discovered to have a security fault, if the faulty code was in a section that you eliminated as dead code... then you don't have a security problem in the first place!
It's true that the signal to noise ratio is high for much of these, but whatever solution we settle on, it should take into account that forcing even beginners to learn how to use npm audit means that security will be taken into consideration from the start, which is both valuable and a net benefit.
It isn't broken by design. I sympathize and understand it's annoying to have to comb through false positives and mark them as such, but until a level of AI we're nowhere near being near exists, you can't automate this process, so the only alternative is to ignore all vulnerabilities.
Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are sensitive by default because the cost of lots of false positives is annoyed developers who have to slow down their delivery cadence. The cost of false negatives is anything from compromising user data to losing your company to bringing down the power grid of a major city depending on what the application is.
Which of those is a higher cost? npm can't possibly know the answer to that, so it has to default to assuming security actually matters to you. If it doesn't, your local policies can be more lax, but don't expect the tool to change for you.
Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are sensitive by default because the cost of lots of false positives is annoyed developers who have to slow down their delivery cadence. The cost of false negatives is anything from compromising user data to losing your company to bringing down the power grid of a major city depending on what the application is.
Which of those is a higher cost? npm can't possibly know the answer to that, so it has to default to assuming security actually matters to you. If it doesn't, your local policies can be more lax, but don't expect the tool to change for you.
>it's annoying to have to comb through false positives and mark them as such
There is no way to "mark them as such". That's half of the issue. The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. They just ran `npm install`, and npm adopted default behavior of showing these warnings. For a lot of people this is their first programming environment.
>you can't automate this process, so the only alternative is to ignore all vulnerabilities.
I don't think the answer is necessarily automation. But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
Or at least I'd like npm to offer a reliable way to update packages deep in the tree and override the resolved versions. Currently, there isn't such a way.
There is no way to "mark them as such". That's half of the issue. The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. They just ran `npm install`, and npm adopted default behavior of showing these warnings. For a lot of people this is their first programming environment.
>you can't automate this process, so the only alternative is to ignore all vulnerabilities.
I don't think the answer is necessarily automation. But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
Or at least I'd like npm to offer a reliable way to update packages deep in the tree and override the resolved versions. Currently, there isn't such a way.
> The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. ... For a lot of people this is their first programming environment.
This attitude makes me kind of uncomfortable. Like, I have taught software development to a decent number of folks, but I've always done so in a relatively isolated environment. If one is buying into web programming, I have a hard time feeling like it matters that it's their first programming environment--it is a hostile place (the web) and some understanding of that hostility is pretty high on the list, I think, of Things To Get Used To. There's definitely a tension there with "don't overwhelm a novice", but I don't necessarily think optimizing for the novice case is wise, especially when we want those novices to have their heads on a swivel, too.
> But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
I definitely agree with this, though, and this is a good way to help make something like `npm audit` more intelligible and useful.
This attitude makes me kind of uncomfortable. Like, I have taught software development to a decent number of folks, but I've always done so in a relatively isolated environment. If one is buying into web programming, I have a hard time feeling like it matters that it's their first programming environment--it is a hostile place (the web) and some understanding of that hostility is pretty high on the list, I think, of Things To Get Used To. There's definitely a tension there with "don't overwhelm a novice", but I don't necessarily think optimizing for the novice case is wise, especially when we want those novices to have their heads on a swivel, too.
> But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.
I definitely agree with this, though, and this is a good way to help make something like `npm audit` more intelligible and useful.
> This attitude makes me kind of uncomfortable.
I think the point though is that security warnings need to be actionable and high-signal. Experienced folks are absolutely tuning out the security warnings on npm install, because 95% of the warnings are like the examples in the post -- I know they don't affect me/my use case and there's nothing I can do about them anyway. The effect is only compounded for novices who run "npx create-react-app hello-world" and immediately see something incomprehensible about a vulnerability in react-scripts > webpack > watchpack > watchpack-chokidar2 > chokidar > glob-parent. It either discourages them from programming entirely or it teaches them to ignore security warnings.
I don't disagree with your overall point -- e.g. we should absolutely teach novices "here's what XSS is and how to avoid it" early and often. But if a dependency manager is going to surface a vulnerability alert every time I install dependencies, the alerts should be 1) high severity (to the point where I should actually stop using the package if I am unable to patch/upgrade) or 2) at least immediately actionable. The current npm audit implementation does the opposite -- 95% of the alerts are totally irrelevant to my actual security posture, and the suggested command to upgrade a vulnerable dependency is unreliable and can actually downgrade to an older, even-less-secure version (!).
I think the point though is that security warnings need to be actionable and high-signal. Experienced folks are absolutely tuning out the security warnings on npm install, because 95% of the warnings are like the examples in the post -- I know they don't affect me/my use case and there's nothing I can do about them anyway. The effect is only compounded for novices who run "npx create-react-app hello-world" and immediately see something incomprehensible about a vulnerability in react-scripts > webpack > watchpack > watchpack-chokidar2 > chokidar > glob-parent. It either discourages them from programming entirely or it teaches them to ignore security warnings.
I don't disagree with your overall point -- e.g. we should absolutely teach novices "here's what XSS is and how to avoid it" early and often. But if a dependency manager is going to surface a vulnerability alert every time I install dependencies, the alerts should be 1) high severity (to the point where I should actually stop using the package if I am unable to patch/upgrade) or 2) at least immediately actionable. The current npm audit implementation does the opposite -- 95% of the alerts are totally irrelevant to my actual security posture, and the suggested command to upgrade a vulnerable dependency is unreliable and can actually downgrade to an older, even-less-secure version (!).
Yeah, this I totally agree with. Actionable alerts are important. The idea that novices should be opted out of ecosystem security concerns generally, less so (not least because they'll create their own security problems in the process).
Frontend is however typical first place to develop in. And backend should never trust frontend anyway, making frontend actually lower risk place. You will hack yourself basically.
> Frontend is however typical first place to develop in
I think that, even today, this is a bold claim to make. Do you have something to substantiate it?
I think that, even today, this is a bold claim to make. Do you have something to substantiate it?
That is not the cost of false positives. The cost of false positives is ignoring it completely. I have had jobs where we just block the security scanners because they won’t listen to any feedback about why what they are scanning was intentionally setup for the purpose of testing vulnerabilities on an internal only network. Additionally at other jobs security tickets just start to get ignored because they send too many tickets that do not matter. I feel the security field likes to ignore most feedback and play holier than though. At all companies i worked at i have only had one good security team that worked with people instead of just throwing things over the wall.
I agree about the false positive problem. Boy who cried wolf and all. I've also worked with security vendors who offer to run "free" vulnerability scans for you, and to absolutely nobody's surprise, they find vulnerabilities that just happen to be the ones that they can fix, if you buy what they are selling.
Still, your example is problematic. Beware the "internal-only network". Such a thing has mostly lost meaning today, and it was never much more than a picket fence anyway. "All devices must be capable of maintaining their security policy on an un-trusted network." https://collaboration.opengroup.org/jericho/commandments_v1....
Still, your example is problematic. Beware the "internal-only network". Such a thing has mostly lost meaning today, and it was never much more than a picket fence anyway. "All devices must be capable of maintaining their security policy on an un-trusted network." https://collaboration.opengroup.org/jericho/commandments_v1....
The issue was "intentionally setup for testing purposes".
What testing purposes? How long was it up? Without knowing more, that doesn't eliminate it as a problem. There are plenty of cases where something set up "just for testing" ended up being the entry point for attackers.
The alternative is hiding most of the reports by default. It's a firehose, and almost all of it is guaranteed to be useless. For example, denial of service in a devDependency shouldn't be shown unless the user specifically requests it. Denial of service shouldn't be marked "high" importance, even though occasionally it might be quite important to some user.
> It makes beginners miserable
Can we please tell beginners not to start programming with node.js?
Teaching beginners to start with “go-to” technologies became industry standard already as it helps corporations to become more and more monopolist and dictate new industry standards.
Can we please tell beginners not to start programming with node.js?
Teaching beginners to start with “go-to” technologies became industry standard already as it helps corporations to become more and more monopolist and dictate new industry standards.
It also has some false results.
Like package x has a vulnerability in version 1.x
And you have a private package @company/x with version 1.x.
Than npm audit will blame your private package, even if you dont have used the original package x.
How much of the problem here is that npm audit is a little annoying and creates problems unrelated to security?
Or do commenters here actually believe that npm audit should treat a DoS of a development machine as a non-vulnerability?
(Please tell me it’s the former)
Or do commenters here actually believe that npm audit should treat a DoS of a development machine as a non-vulnerability?
(Please tell me it’s the former)
> Or do commenters here actually believe that npm audit should treat a DoS of a development machine as a non-vulnerability?
I believe that npm audit should treat a DoS of a development machine by a trusted developer as a non-vulnerability. "Code I (or a fellow committer) wrote uses a lot of CPU" isn't a vulnerability. If I care to prevent this, I should run said code within a cgroup with limited resources, not panic about theoretical expense in one part of the codebase while necessarily allowing arbitrary execution elsewhere. "npm audit" is crying wolf, just as the author said.
I like the proposal [1] near the end: "If I own an npm package I need to be able to tag a certain transitive vulnerability category as not affecting my usage of that transitive package." This is particularly important for npm given things like create-react-app but would also be a good idea for "cargo audit" and such.
[1] https://twitter.com/dan_abramov/status/1412380714012594178
I believe that npm audit should treat a DoS of a development machine by a trusted developer as a non-vulnerability. "Code I (or a fellow committer) wrote uses a lot of CPU" isn't a vulnerability. If I care to prevent this, I should run said code within a cgroup with limited resources, not panic about theoretical expense in one part of the codebase while necessarily allowing arbitrary execution elsewhere. "npm audit" is crying wolf, just as the author said.
I like the proposal [1] near the end: "If I own an npm package I need to be able to tag a certain transitive vulnerability category as not affecting my usage of that transitive package." This is particularly important for npm given things like create-react-app but would also be a good idea for "cargo audit" and such.
[1] https://twitter.com/dan_abramov/status/1412380714012594178
This should be: npm: broken by design.
Or even, since it's in fact the language itself that sets the tone for the entire ecosystem: javascript: broken by design.
Or even, since it's in fact the language itself that sets the tone for the entire ecosystem: javascript: broken by design.
Funny. I had a discussion about the same topic in context of docker image security scanners last week. 100% agree with the author here!
I do wonder if a typed language shouldn't have distinct types for trusted vs untrusted strings and binaries.
Correct me if I'm wrong.
But, one of the goals in software engineering right now is reproducible builds. This means building from source. And of course we'll want to automate that. We've already made inroads with CI
So, correct me if I'm wrong, these are still vulnerabilities.
Tragedy of the commons stuff.
This article might be honest. But I hope in the future we don't need devil's advocate arguments.
But, one of the goals in software engineering right now is reproducible builds. This means building from source. And of course we'll want to automate that. We've already made inroads with CI
So, correct me if I'm wrong, these are still vulnerabilities.
Tragedy of the commons stuff.
This article might be honest. But I hope in the future we don't need devil's advocate arguments.
Still, the original post might inspire realignment of bug fixing incentives.
This is how it works by design.
Agreed. I have been trained by npm to ignore its audit messages.
[deleted]
the machine doesn't need to be compromised, just check it into source control
Bad/ill formed regexes took down the internet recently so they are in fact an attack surface https://blog.cloudflare.com/details-of-the-cloudflare-outage...
Vulnerabilities are just code paths that can behave in unexpected ways and be abused.... I'm not sure the author's point, that development configuration could not hide malicious code? Why not?
All NPM does is scan to the dependency graph for vulnerability reports, it doesn't make any assessment of your consuming application's use-case. If you don't find this useful that is fine, don't use it.
I think it's totally worthwhile to figure out which tools rely on insecure dependencies.
Also looks like you can specify npm to ignore dev dependencies:
> Any packages in the tree that do not have a version field in their package.json file will be ignored. If any --omit options are specified (either via the --omit config, or one of the shorthands such as --production, --only=dev, and so on), then packages will be omitted from the submitted payload as appropriate.
All NPM does is scan to the dependency graph for vulnerability reports, it doesn't make any assessment of your consuming application's use-case. If you don't find this useful that is fine, don't use it.
I think it's totally worthwhile to figure out which tools rely on insecure dependencies.
Also looks like you can specify npm to ignore dev dependencies:
> Any packages in the tree that do not have a version field in their package.json file will be ignored. If any --omit options are specified (either via the --omit config, or one of the shorthands such as --production, --only=dev, and so on), then packages will be omitted from the submitted payload as appropriate.
>I'm not sure the author's point, that development configuration could not hide malicious code? Why not?
Quite the opposite! Quoting the article:
As any security professional will tell you, development dependencies actually are an attack vector, and perhaps one of the most dangerous ones because it’s so hard to detect and the code runs with high trust assumptions. This is why the situation is so bad in particular: any real issue gets buried below dozens of non-issues that npm audit is training people and maintainers to ignore. It’s only a matter of time until this happens.
My point is that in the sea of non-issues, real issues are easy to miss and ignore.
>If you don't find this useful that is fine, don't use it.
You can't "not use it" because it's literally the default behavior built into `npm install` now. Of course there are ways to opt out, but this doesn't alleviate the confusion.
Quite the opposite! Quoting the article:
As any security professional will tell you, development dependencies actually are an attack vector, and perhaps one of the most dangerous ones because it’s so hard to detect and the code runs with high trust assumptions. This is why the situation is so bad in particular: any real issue gets buried below dozens of non-issues that npm audit is training people and maintainers to ignore. It’s only a matter of time until this happens.
My point is that in the sea of non-issues, real issues are easy to miss and ignore.
>If you don't find this useful that is fine, don't use it.
You can't "not use it" because it's literally the default behavior built into `npm install` now. Of course there are ways to opt out, but this doesn't alleviate the confusion.
All it does is look to see if either your direct dependencies or descendant dependencies exist in the advisory database....
It seems like a simple algorithm that works pretty well. Perhaps ignoring certain dependencies makes sense, via an ignore list.
I just find the title "NPM is broken by design" to be a little hyperbolic, when it seems like the complaint is that it's tedious removing all the low-quality dependencies from your project. node security/npm-audit has at least increased the conversation around security for many around the npm ecosystem, where there wasn't much-if-any discussion prior. I think they deserve credit for this.
EDIT: I'm not sure why I'm being downvoted.
It seems like a simple algorithm that works pretty well. Perhaps ignoring certain dependencies makes sense, via an ignore list.
I just find the title "NPM is broken by design" to be a little hyperbolic, when it seems like the complaint is that it's tedious removing all the low-quality dependencies from your project. node security/npm-audit has at least increased the conversation around security for many around the npm ecosystem, where there wasn't much-if-any discussion prior. I think they deserve credit for this.
EDIT: I'm not sure why I'm being downvoted.
Think of the article as customer feedback. The customer may not appreciate everything the product does for them, nor may they be using it in the fashion where it would be most effective.
So what? Your job (if this was your customer) would be to figure out how to make them happier. Maybe you are getting down-voted because (while you're correct on some fine points), you're broadly dismissing the concerns of the article. Case in point: those "low-quality dependencies" aren't something you can easily switch out for quality parts... they're deep dependencies of many of npm's flagship tools and frameworks.
So what? Your job (if this was your customer) would be to figure out how to make them happier. Maybe you are getting down-voted because (while you're correct on some fine points), you're broadly dismissing the concerns of the article. Case in point: those "low-quality dependencies" aren't something you can easily switch out for quality parts... they're deep dependencies of many of npm's flagship tools and frameworks.
Perhaps the problem is that user's are entirely too entrenched with using blackbox frameworks they don't fully understand like create-react-app, and not that we have a tool that discloses when these frameworks contain vulnerable dependencies.
If we are in a situation where swapping out dependencies becomes so difficult that we just throw our hands up, is that really an issue with tooling?
EDIT:
BTW I'm not opposed to a PR that would allow for a .gitignore style list to ignore warning on specified deps. That could be useful. The issue I have would be respecting other peoples "auditignore" list or whatever... Because just because someone like @danabramov thinks including some package is not a threat, I may or may not agree with him.
If we are in a situation where swapping out dependencies becomes so difficult that we just throw our hands up, is that really an issue with tooling?
EDIT:
BTW I'm not opposed to a PR that would allow for a .gitignore style list to ignore warning on specified deps. That could be useful. The issue I have would be respecting other peoples "auditignore" list or whatever... Because just because someone like @danabramov thinks including some package is not a threat, I may or may not agree with him.
I agree with the author. Just like them, I only write code without any bugs that can be affected by the "vulnerabilities". I also never commit to upstream so others won't be able to edit my code. All my projects only run on my machine (which is of course also absolutely exploit-free and it's not connected to the internet).
You're being snarky, which is fine, but the author addresses that. If you're compromised, the attacker is not going to dig through your development folder to inject a regex that makes your build slow. They'll exploit privilege escalation bugs to install a bitcoin miner, ransomware, a DDOS bot node, or use some other vulnerability to grab and/or exploit your secrets. They'll do it the most direct way possible, not via some half-broken regex parser.
A lack of imagination by the author, unfortunately...
A DoS on your build machine and dev machine can be indeed be critical issues. Imagine this scenario:
Your source code is somehow compromised and attackers slip in rogue code to your production site. It siphons off passwords or other PII. The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive, so you can’t just spin a fixed build and re-deploy. You’ll sync your main branch to figure out what is going on, perhaps ready to make a build from your dev machine, but running yarn build hangs. It might take you 1 minute to solve or 5 hours - hard to guess. But every minute you’re delayed is another minute the attacker is siphoning off your production data.
npm audit isn’t perfect, but I don’t agree with the author that devDependencies can’t have critical vulnerabilities. Build machines and dev machines are critical infrastructure. Recall the method of attack of SolarWinds [1].
Related: we all trust that the “many eyes” of open source contributors will keep our dependencies relatively clean, but this function is not infinite. There is some threshold of lines of code and rate of change that will outstrip the community’s natural ability to find and fix problems. I wish the npm community was more sensitive to the risks that are inherent in current practices. Efforts to limit dependencies and perhaps somehow tag which versions have completed a security audit (and by whom) would be great to see.
[1] https://krebsonsecurity.com/2021/01/solarwinds-what-hit-us-c...
A DoS on your build machine and dev machine can be indeed be critical issues. Imagine this scenario:
Your source code is somehow compromised and attackers slip in rogue code to your production site. It siphons off passwords or other PII. The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive, so you can’t just spin a fixed build and re-deploy. You’ll sync your main branch to figure out what is going on, perhaps ready to make a build from your dev machine, but running yarn build hangs. It might take you 1 minute to solve or 5 hours - hard to guess. But every minute you’re delayed is another minute the attacker is siphoning off your production data.
npm audit isn’t perfect, but I don’t agree with the author that devDependencies can’t have critical vulnerabilities. Build machines and dev machines are critical infrastructure. Recall the method of attack of SolarWinds [1].
Related: we all trust that the “many eyes” of open source contributors will keep our dependencies relatively clean, but this function is not infinite. There is some threshold of lines of code and rate of change that will outstrip the community’s natural ability to find and fix problems. I wish the npm community was more sensitive to the risks that are inherent in current practices. Efforts to limit dependencies and perhaps somehow tag which versions have completed a security audit (and by whom) would be great to see.
[1] https://krebsonsecurity.com/2021/01/solarwinds-what-hit-us-c...
> Your source code is somehow compromised and attackers slip in rogue code to your production site.
Really at this point it's too late to do anything else, instead of trying to dos your dev machine he can instead do simpler things like delete your ssh key from the machine. But let's play along:
> The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive
There is nothing any attacker can do with the static files on the server that will trigger and RegEx DoS in your local development. Aside from the fact that you wouldn't download whatever is on the server back to your machine, even if you did it would never trigger such a DoS since (in the examples in the link) these are modules related to running a dev version of a frontend project based on the raw source files.
Your scenario is only true when an attacker pwned both your production server and your laptop. A regex DoS is really the last thing you worry about at that stage.
Really at this point it's too late to do anything else, instead of trying to dos your dev machine he can instead do simpler things like delete your ssh key from the machine. But let's play along:
> The attackers also take advantage of several of these RegEx DoS vulnerabilities to prevent you from quickly fixing the problem. When you discover the issue, you’ll first see that your build machine is unresponsive
There is nothing any attacker can do with the static files on the server that will trigger and RegEx DoS in your local development. Aside from the fact that you wouldn't download whatever is on the server back to your machine, even if you did it would never trigger such a DoS since (in the examples in the link) these are modules related to running a dev version of a frontend project based on the raw source files.
Your scenario is only true when an attacker pwned both your production server and your laptop. A regex DoS is really the last thing you worry about at that stage.
I see your point that even worse things can happen when dev machines are compromised. The point I tried to make is that even a DoS of your machine can be a big problem.
> There is nothing any attacker can do with the static files on the server that will trigger a RegEx DoS...
IIUC, an attacker could change my package.json to include inputs to browserlist that trigger a RegEx DoS. To do that, the attacker only needs to make a fraudulent commit. Given how easy most teams make it to commit code, this isn’t too high of a bar.
> There is nothing any attacker can do with the static files on the server that will trigger a RegEx DoS...
IIUC, an attacker could change my package.json to include inputs to browserlist that trigger a RegEx DoS. To do that, the attacker only needs to make a fraudulent commit. Given how easy most teams make it to commit code, this isn’t too high of a bar.
If you, as an attacker, are going to make a fraudulent commit and change package.json, you could just pull in a bogus dependency or add a new script/command that gets run on the build/dev machine. I agree the bar to submitting a fraudulent commit is unfortunately too low for many teams. But, this also extends to package publishing, too.
We've seen packages published through compromised dev keys or the maintainer granted a new bad actor rights to publish. Here we could envision a situation where someone reports a legitimate, but low impact, security issue and also publishes a bogus package that everyone is now upgrading to. On the whole, these messages aren't encouraging people to upgrade to a new, verified release (i.e., a fixed version). They're encouraging people to upgrade to the latest release. Even if you suspend the bad actor thought experiment, the latest release of a package in all likelihood hasn't been audited beyond ensuring the previously reported security issue has been addressed. Upgrading indiscriminately is a risky activity as well. I don't think we should be encouraging people to do that (in any language ecosystem).
It's not lack of imagination of attack vectors at play here. Treating extremely low risk factors as if they're high priority reduces trust in the system as a whole. Getting developers to care more about security is a laudable effort, but I think the `npm audit` approach (as reported in the post) is going to encourage bad practices.
We've seen packages published through compromised dev keys or the maintainer granted a new bad actor rights to publish. Here we could envision a situation where someone reports a legitimate, but low impact, security issue and also publishes a bogus package that everyone is now upgrading to. On the whole, these messages aren't encouraging people to upgrade to a new, verified release (i.e., a fixed version). They're encouraging people to upgrade to the latest release. Even if you suspend the bad actor thought experiment, the latest release of a package in all likelihood hasn't been audited beyond ensuring the previously reported security issue has been addressed. Upgrading indiscriminately is a risky activity as well. I don't think we should be encouraging people to do that (in any language ecosystem).
It's not lack of imagination of attack vectors at play here. Treating extremely low risk factors as if they're high priority reduces trust in the system as a whole. Getting developers to care more about security is a laudable effort, but I think the `npm audit` approach (as reported in the post) is going to encourage bad practices.
I agree with all of your points except for the very last one.
Are we shooting the messenger (npm audit) here? Seems like the problem is lack of trust and lack of information about security of dependencies. Npm audit is just pointing that out. The size of the problem makes it very uncomfortable.
Are we shooting the messenger (npm audit) here? Seems like the problem is lack of trust and lack of information about security of dependencies. Npm audit is just pointing that out. The size of the problem makes it very uncomfortable.
I think where we disagree is I don't think npm audit is pointing out useful information. Or, at least, it's including a lot of useless information that makes finding the useful information hard to see. Now that it's the default when installing packages, most JS developers are going to have to deal with it. Given a lot of the data is not applicable, unnecessarily repetitive, and has severity mismatch with reality, I think most people are just going to ignore the output. If not that, they're going to blindly update packages and when that doesn't work due to dependency chains and such, they're going to start saddling open source maintainers with busy work. The net result is that when the next real mass security issue hits, it's going to get overlooked or ignored with all the other noise.
What could have been a useful tool for CI is not going to be enabled because who wants their build failing for a week while trying to orchestrate a mass dependency upgrade? I just took a look a Create React App (CRA)-based I have and "yarn audit" indicates 2,288 packages were scanned with 337 vulnerabilities (that's pretty good, I've seen it list over 500 vulnerabilities in the past). This is an app using CRA, with Relay as a code generating GraphQL client, and Antd as a component framework. There's also supporting tooling such as TypeScript, ESLint, and Prettier. Beyond that, very few dependencies are being used. Almost none of them are actually shipping in the product, whether that be due to them being build-related tools or because tree-shaking just strips most of the library out. It's a SPA hosted in CloudFront, so even DoS attacks have a very different profile than products where the server is rendering views. In this particular case, there's no shared user-generated content and XSS mitigations are in place, so the best a customer could do is DoS themselves.
I try to be very diligent about auditing dependencies before upgrading, but I can't reasonably extend that to transitive dependencies. 2,288 packages is an absurd number and this app isn't even doing all that much. I believe it's quite likely that it'd be impossible to get the npm/yarn audit report down to 0, given how quickly the JS ecosystem moves -- upgrade one dependency, deal with whatever API changes are needed, and now another has a vulnerability of some sort listed. It's certainly a burden on a small development team.
Just to be clear, I'm not advocating sticking your head in the sand and pretend everything's fine. I'm saying when you have vulnerabilities listed as high severity and you've verified they're not problematic in a project, then you've addressed the issue. But, now you have a tool that you need to satisfy and if you don't then you're going to miss the next security issue because CI shut off the "npm audit" check.
What could have been a useful tool for CI is not going to be enabled because who wants their build failing for a week while trying to orchestrate a mass dependency upgrade? I just took a look a Create React App (CRA)-based I have and "yarn audit" indicates 2,288 packages were scanned with 337 vulnerabilities (that's pretty good, I've seen it list over 500 vulnerabilities in the past). This is an app using CRA, with Relay as a code generating GraphQL client, and Antd as a component framework. There's also supporting tooling such as TypeScript, ESLint, and Prettier. Beyond that, very few dependencies are being used. Almost none of them are actually shipping in the product, whether that be due to them being build-related tools or because tree-shaking just strips most of the library out. It's a SPA hosted in CloudFront, so even DoS attacks have a very different profile than products where the server is rendering views. In this particular case, there's no shared user-generated content and XSS mitigations are in place, so the best a customer could do is DoS themselves.
I try to be very diligent about auditing dependencies before upgrading, but I can't reasonably extend that to transitive dependencies. 2,288 packages is an absurd number and this app isn't even doing all that much. I believe it's quite likely that it'd be impossible to get the npm/yarn audit report down to 0, given how quickly the JS ecosystem moves -- upgrade one dependency, deal with whatever API changes are needed, and now another has a vulnerability of some sort listed. It's certainly a burden on a small development team.
Just to be clear, I'm not advocating sticking your head in the sand and pretend everything's fine. I'm saying when you have vulnerabilities listed as high severity and you've verified they're not problematic in a project, then you've addressed the issue. But, now you have a tool that you need to satisfy and if you don't then you're going to miss the next security issue because CI shut off the "npm audit" check.
1. You wouldn't wait for a full build in that scenario, but deploy a known-good last image or emergency shut down.
2. If you are doing a full build and fail because of the regex DOS, then that build would also contain the attacker injected siphoning code, which would make your entire exercise futile in the first place
3. Not obviously messing with the network or crashing build machines would be a better way of siphoning data for longer.
4. This is very contrived.
2. If you are doing a full build and fail because of the regex DOS, then that build would also contain the attacker injected siphoning code, which would make your entire exercise futile in the first place
3. Not obviously messing with the network or crashing build machines would be a better way of siphoning data for longer.
4. This is very contrived.
I think if the attacker got as far as deploy their code in your CI/CD pipeline and prod system, there's no "quickly" fixing it. There's a full shut down, restoring from trusted backup, full data and code audit and a lot of pain in the future validating and restoring the code and the data. Quick rebuild is not something that would be your priority there - how do you know this quick build won't be compromised anyway? If somebody got into your internal systems on the level they could modify the code, it's not a 1 minute problem and not a 5 hours problem, it's much bigger...
I think the typical scenario is that you understand how big of problem it is only in retrospect.
In the moment, your first thought is that there is some type of quick fix that will restore functionality (if your site is down) or evict the intruder if something funny is detected. As a sibling commenter said, most teams would try to deploy a previous known-good build asset.
But I stand by my point that a DoS of a development system can indeed be critical! I’m surprised to find that I appear to be in the minority here...
In the moment, your first thought is that there is some type of quick fix that will restore functionality (if your site is down) or evict the intruder if something funny is detected. As a sibling commenter said, most teams would try to deploy a previous known-good build asset.
But I stand by my point that a DoS of a development system can indeed be critical! I’m surprised to find that I appear to be in the minority here...
I think you're confusing scenarios here. If you have known-good build, then there's no way it is compromised by regexp DOS attack - because that's the build that happened before the attack. So if you build that - setting aside the wisdom on doing that on a compromised system - the regexp DoS is not relevant. If you are building the modified code then one should definitely question why would you want to do something like that - build and deploy known compromised code.
That seems to be the root of your confusion - it's not that development system DoS is not bad, it's that if you are at the point it's possible your security is already broken in much bigger way. It's like complaining that running "rm -rf /" under root would wipe all your files and that's a DoS - without taking into account that if somebody could run commands under root on your system it's not your system anymore. It's not that wiping all the files isn't bad - it's that the reason for why the situation is bad is much earlier that that.
That seems to be the root of your confusion - it's not that development system DoS is not bad, it's that if you are at the point it's possible your security is already broken in much bigger way. It's like complaining that running "rm -rf /" under root would wipe all your files and that's a DoS - without taking into account that if somebody could run commands under root on your system it's not your system anymore. It's not that wiping all the files isn't bad - it's that the reason for why the situation is bad is much earlier that that.
So many people are reporting these in tons of different projects: https://github.com/search?q=regex+denial+of+service&type=iss...
Anyhow it is just annoying and they broke NPM Audit based on these reports.
It is good to fix all possible bugs, but many of these are not anywhere close to the level of bad that the reports are making them to be.
But maybe this is needed to just get rid of these issues in genera? So a wave of regex vulnerability reports and then we build this type of checking into prettier or similar and we do not have these in the future?
EDIT: It appears there as a project that found 100s of CVE reported Regex vulnerabilities in npm projects -- this is maybe one of the sources of mass reports. See the bottom of this resume: https://yetingli.github.io