NPM and NodeJS should do more to make ES Modules easy to use(borischerny.com)
borischerny.com
NPM and NodeJS should do more to make ES Modules easy to use
https://borischerny.com/javascript,/typescript/2024/06/19/ES-Modules-Are-A-Mess.html
229 comments
> Allow es modules to be require()-ed. It's be synchronous and slow and ideologically impure, but it'll make it so that millions of projects (mostly private!) will be able to start adopting es modules without a massive refactor.
This is already available in Node.js 22 [1] with ` --experimental-require-module` flag.
[1]: https://nodejs.org/en/blog/announcements/v22-release-announc...
This is already available in Node.js 22 [1] with ` --experimental-require-module` flag.
[1]: https://nodejs.org/en/blog/announcements/v22-release-announc...
Before Node had any/experimental module support (April 2019, https://nodejs.medium.com/announcing-a-new-experimental-modu...), I'd been using std-things/esm loader for Node.js, which seemed to have no problem allowing CJS and ESM to interop, require and import each other. In my experience it worked flawlessly. https://github.com/standard-things/esm
It's wild to think about: esm has taken nearly a decade to be feasible for folks to use, without heavy encumbrance; it was the key feature of es-2015/es6!
It took many more years for import-maps to become a thing, such that we could start using modules in a modular fashion; we've had to use bundlers/codemods to re-resplve dependencies, unless you've been going to go Deno's original path of just hitting absolute urls wherever they be. There's still no standing proposal for how we get import-maps on service-workers & shared workers; modules still are not a ubiquitous feature of the core web & JavaScript ecosystems. 9 years latter. https://github.com/WICG/import-maps/issues/2
It's been hard being a webdevs, when the future remains so partially implemented.
The article's exploration of the bomb ecosystem is telling. And is much of the reason I hope JavaScript Registry (jsr) project might succeed. Npm will never escape it's past. CJS is going to be in there for a long time. Being on a registry where everything is esm & typescript, that sounds divine. https://deno.com/blog/jsr-is-not-another-package-manager
It's wild to think about: esm has taken nearly a decade to be feasible for folks to use, without heavy encumbrance; it was the key feature of es-2015/es6!
It took many more years for import-maps to become a thing, such that we could start using modules in a modular fashion; we've had to use bundlers/codemods to re-resplve dependencies, unless you've been going to go Deno's original path of just hitting absolute urls wherever they be. There's still no standing proposal for how we get import-maps on service-workers & shared workers; modules still are not a ubiquitous feature of the core web & JavaScript ecosystems. 9 years latter. https://github.com/WICG/import-maps/issues/2
It's been hard being a webdevs, when the future remains so partially implemented.
The article's exploration of the bomb ecosystem is telling. And is much of the reason I hope JavaScript Registry (jsr) project might succeed. Npm will never escape it's past. CJS is going to be in there for a long time. Being on a registry where everything is esm & typescript, that sounds divine. https://deno.com/blog/jsr-is-not-another-package-manager
Did anyone actually want ES modules, and are anyone using ES modules in production today? I don't think so except for transpiling languages that compile and bundle the JS. The only argument against lazy loading is that single threaded UI's will freeze for a few ms when the code loads, but if you compare that to the time it takes to fully load a modern website that point is moot.
There are a few hiccups, but I've found ESM to be a huge win overall. The killer feature for me is the ability to write code that can be imported in a browser without any sort of build step. You don't even need node/npm. Just download the file/directory from somewhere and import. It's perfectly feasible to build a complex browser app with many dependencies using nothing but git submodules.
The main thing you're missing out on is the optimizations that come with bundlers. But they should be treated as just that: optimizations. You shouldn't be required to use a bundler just to develop/distribute JS programs.
The main thing you're missing out on is the optimizations that come with bundlers. But they should be treated as just that: optimizations. You shouldn't be required to use a bundler just to develop/distribute JS programs.
(Most) everyone was already using bundlers at this point. Tree shaking was much easier to do after the dependency graph became computeable. Which it wasn't with CJS; require could happen anywhere with any parameters.
And require happening anywhere while being sync was kind of a non-starter. That freeze is amplified by most projects having hundreds of not thousands of requires across it's dependency graph.
There's definitely some truth to your gripes. Back in 2015 there was a ton of latent hope http/2 push and ESM would let us get away from bundling & perhaps even needing build steps at all. That hasn't panned out.
Indeed, push was removed entirely! https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYL... Except it's still the core backend for Push API. Alas, fetch, in spite of many years of asking for it, never got the ability to observe a Push request, so devs literally never had a chance to explore the possible content update/content discovery use cases that make so so much sense in Web Push Protocol. Crying waste & shame; Push had promise outside of asset delivery but it never got a chance (except the deeply browser-intermediated Push API).
There are a variety of tools & options for bundlers to emit bundled esm. Which would nicely make your bundle usable by multiple different consumers.
There's definitely some truth to your gripes. Back in 2015 there was a ton of latent hope http/2 push and ESM would let us get away from bundling & perhaps even needing build steps at all. That hasn't panned out.
Indeed, push was removed entirely! https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYL... Except it's still the core backend for Push API. Alas, fetch, in spite of many years of asking for it, never got the ability to observe a Push request, so devs literally never had a chance to explore the possible content update/content discovery use cases that make so so much sense in Web Push Protocol. Crying waste & shame; Push had promise outside of asset delivery but it never got a chance (except the deeply browser-intermediated Push API).
There are a variety of tools & options for bundlers to emit bundled esm. Which would nicely make your bundle usable by multiple different consumers.
Bundler developer here: some bundlers (like Parcel) use esm imports within their generated runtime output to facilitate loading dynamic bundles.
It's a bit simpler (and I'd assume faster) than the traditional approach of dynamically creating a script tag at runtime.
Additional features like access to import.meta lets you get the path to the current script which also helps bundlers with resolving the paths to dynamic imports without requiring the developer to specify the base path explicitly.
There are problems though, for example you cannot retry an import() if the internet drops. The failed import is cached so the browser will fail the import again on retry, even if the connection is restored https://github.com/fatso83/retry-dynamic-import
It's a bit simpler (and I'd assume faster) than the traditional approach of dynamically creating a script tag at runtime.
Additional features like access to import.meta lets you get the path to the current script which also helps bundlers with resolving the paths to dynamic imports without requiring the developer to specify the base path explicitly.
There are problems though, for example you cannot retry an import() if the internet drops. The failed import is cached so the browser will fail the import again on retry, even if the connection is restored https://github.com/fatso83/retry-dynamic-import
EXACTLY!!!
(Though I'd be happier with importSync() to identify the use of ESM packages.)
(Though I'd be happier with importSync() to identify the use of ESM packages.)
Implementing it in require() allows a library to migrate from CJS to ESM, without breaking any of its downstream users.
That's fair. I worry a little about revising something this fundamental invisibly but, still, fair point.
That's awesome! I can't wait to try this.
Maybe I'm in the minority, but using NPM packages that have changed over to modules continue to be one of my biggest annoyances in NodeJS:
Maybe I'm in the minority, but using NPM packages that have changed over to modules continue to be one of my biggest annoyances in NodeJS:
npm install node-fetch
const fetch = require('node-fetch');
Error [ERR_REQUIRE_ESM]: require() of ES Module
npm install node-fetch@2You might just be using the node-fetch example as a stand-in to make your point about esm/cjs, but the fetch api is built in to Node (since v16.15) and is no longer even considered experimental as of Node 21+.
https://nodejs.org/docs/latest-v22.x/api/globals.html#fetch
https://nodejs.org/docs/latest-v22.x/api/globals.html#fetch
By god do I agree with this. I feel like all my most diabolical ESM issues are beating the tsconfig.json, webpack.js, and package.json into shape to get ESM and legacy working.
> There's no return on investment to convert
Nothing except doing things the standard way so that standards-compliant language implementations (and other tools) can work with them instead of targetting IE^H^H NodeJS's non-standard quirks.
> the payoff is "it's the shiny new thing"
NodeJS's reality distortion field has claimed another.
Ecmascript modules are going on 10 years old at this point. (For comparison, that's more than the amount of time that NodeJS existed when modules weren't a part of the language.)
Nothing except doing things the standard way so that standards-compliant language implementations (and other tools) can work with them instead of targetting IE^H^H NodeJS's non-standard quirks.
> the payoff is "it's the shiny new thing"
NodeJS's reality distortion field has claimed another.
Ecmascript modules are going on 10 years old at this point. (For comparison, that's more than the amount of time that NodeJS existed when modules weren't a part of the language.)
> Ecmascript modules are going on 10 years old at this point.
False. Ecmascript defined the syntax for imports but not the mechanism for module loading. I've been using ES6 imports with CJS modules for almost as long as the syntax has existed (without problems!); the incompatibility you're describing is ones of years old.
> Nothing except doing things the standard way so that standards-compliant language implementations
Node's way of handling modules isn't the same as the way the browser works (and it can't be: see node_modules). There's no singular all-encompassing "JavaScript ESM" standard. There's not even consistency between server-side JS runtimes.
But more importantly, saying "the way your code worked for fifteen years is now shit and won't work going forward, you need to rewrite it because there's a new thing now" is an incredibly fucked way to run an ecosystem. See: Python 3.
False. Ecmascript defined the syntax for imports but not the mechanism for module loading. I've been using ES6 imports with CJS modules for almost as long as the syntax has existed (without problems!); the incompatibility you're describing is ones of years old.
> Nothing except doing things the standard way so that standards-compliant language implementations
Node's way of handling modules isn't the same as the way the browser works (and it can't be: see node_modules). There's no singular all-encompassing "JavaScript ESM" standard. There's not even consistency between server-side JS runtimes.
But more importantly, saying "the way your code worked for fifteen years is now shit and won't work going forward, you need to rewrite it because there's a new thing now" is an incredibly fucked way to run an ecosystem. See: Python 3.
> Node's way of handling modules isn't the same as the way the browser works
That's a Node problem. 100% foreseeable (and avoidable).*
There's a reason why I compared NodeJS to Internet Explorer in my first comment—because it's extremely apt. Too bad most NPM programmers have blinders such that their entire worldview consists of NPM+NodeJS (not unlike the way programmers who were used to exclusively targeting Microsoft tech 20 years ago lived in their own parochial world) and an expectation that everyone else should bend over backwards to accommodate their way of life.
If you as a programmer want not to be burned by IE-/NodeJS-like shenanigans, then don't run arms outstretched into those flames.
* And, no, it's not an anachronism to say that. There's a reason why the notion of "forward compatibility" exists—at least among the wise. For an example of how to approach things sensibly (read: avoid getting burned exactly like one should otherwise expect to), have a look at the position/approach to forward compatibility that the TypeScript team (initially a Microsoft undertaking, funnily enough) has taken. There's no reason anyone should expect to be able to throw caution to the wind, YOLO their way forward with their hands covering their ears while shouting "la-la-la" and for everything to turn out okay. That is profoundly unwise.
> an incredibly fucked way to run an ecosystem
Right. Which makes it doubly crazy the way that folks underneath the NodeJS carnival tent have taken to approaching things for the last 15 years—and the last 10 years in particular. (Again, to emphasize: 10 years!)
But of course I'm being facetious, because when you said "an ecosystem" you didn't narrowly mean "the NodeJS ecosystem". You meant "the JS ecosystem"—leaning heavily on the very same merger doctrine of the-NodeJS-ecosystem-is-the-JS-ecosystem-which-is-the-NodeJS-ecosystem that I mentioned NPMers believing in before.
> See: Python 3.
Sure, and Python is proprietary—with a BDFL, a community rallying for the most part behind a single implementation (where you get what you pay for if you go astray), and no strong commitment to standards/compatibility for most of its existence; the Python project can do whatever the Python project wants to do. Likewise Perl. Likewise, you know... Dart. That's how those things go.
JS is fundamentally different, and that's what you and countless other NPM programmers seem not to be getting. See also my earlier comment:
The difference is the language/standard in question neither originated with NodeJS nor is NodeJS now nor has it ever been led by the people behind the language/standard
<https://news.ycombinator.com/item?id=40744081>
The node_modules ecosystem is, in fact, not the whole world, nor is it synonymous with "JS". (Heck, it's not even synonymous with CommonJS[1].) NPMers can ignore this fact at their peril, or they can wise up and accept that they can't simply will things to go the way they apparently think they should be able to make them go despite mountains of evidence and wisdom to the contrary.
1. <https://news.ycombinator.com/item?id=31528800>
That's a Node problem. 100% foreseeable (and avoidable).*
There's a reason why I compared NodeJS to Internet Explorer in my first comment—because it's extremely apt. Too bad most NPM programmers have blinders such that their entire worldview consists of NPM+NodeJS (not unlike the way programmers who were used to exclusively targeting Microsoft tech 20 years ago lived in their own parochial world) and an expectation that everyone else should bend over backwards to accommodate their way of life.
If you as a programmer want not to be burned by IE-/NodeJS-like shenanigans, then don't run arms outstretched into those flames.
* And, no, it's not an anachronism to say that. There's a reason why the notion of "forward compatibility" exists—at least among the wise. For an example of how to approach things sensibly (read: avoid getting burned exactly like one should otherwise expect to), have a look at the position/approach to forward compatibility that the TypeScript team (initially a Microsoft undertaking, funnily enough) has taken. There's no reason anyone should expect to be able to throw caution to the wind, YOLO their way forward with their hands covering their ears while shouting "la-la-la" and for everything to turn out okay. That is profoundly unwise.
> an incredibly fucked way to run an ecosystem
Right. Which makes it doubly crazy the way that folks underneath the NodeJS carnival tent have taken to approaching things for the last 15 years—and the last 10 years in particular. (Again, to emphasize: 10 years!)
But of course I'm being facetious, because when you said "an ecosystem" you didn't narrowly mean "the NodeJS ecosystem". You meant "the JS ecosystem"—leaning heavily on the very same merger doctrine of the-NodeJS-ecosystem-is-the-JS-ecosystem-which-is-the-NodeJS-ecosystem that I mentioned NPMers believing in before.
> See: Python 3.
Sure, and Python is proprietary—with a BDFL, a community rallying for the most part behind a single implementation (where you get what you pay for if you go astray), and no strong commitment to standards/compatibility for most of its existence; the Python project can do whatever the Python project wants to do. Likewise Perl. Likewise, you know... Dart. That's how those things go.
JS is fundamentally different, and that's what you and countless other NPM programmers seem not to be getting. See also my earlier comment:
The difference is the language/standard in question neither originated with NodeJS nor is NodeJS now nor has it ever been led by the people behind the language/standard
<https://news.ycombinator.com/item?id=40744081>
The node_modules ecosystem is, in fact, not the whole world, nor is it synonymous with "JS". (Heck, it's not even synonymous with CommonJS[1].) NPMers can ignore this fact at their peril, or they can wise up and accept that they can't simply will things to go the way they apparently think they should be able to make them go despite mountains of evidence and wisdom to the contrary.
1. <https://news.ycombinator.com/item?id=31528800>
The whole thing feels like IPv6 adoption to me: Make v2 of a thing that is undeniably better, but downplay/ignore the migration strategy and then just complain that people are lazy if they haven't migrated yet.
Although on a payoff note, it's not even the shiny new thing, it's almost a decade old. We really need a Python 2.7 style "no: we're done with this. Don't uplift your code as much as you deem necessary, but we're moving on. You've had a decade by now" event for Node.
> it's almost a decade old
The syntax is a decade old, and people have been using CJS with ES6 import syntax this whole time without major issues. Node ESM support is an entirely different self-inflicted problem that's been around for just a couple years. Don't gaslight folks into thinking this was something that the ecosystem was planning for a decade. It simply wasn't.
The syntax is a decade old, and people have been using CJS with ES6 import syntax this whole time without major issues. Node ESM support is an entirely different self-inflicted problem that's been around for just a couple years. Don't gaslight folks into thinking this was something that the ecosystem was planning for a decade. It simply wasn't.
If by "just a couple years" you mean five years. There is no gas lighting here: I am directly blaming Node for hurting the uptake of ESM by not planning to switch to ESM-first (with CJS second) as part of the ESM support work, and I'm blaming Node even more for then also not starting that planning work when ESM in Node was considered mature enough to not live behind an experimental feature flag anymore.
Node was, from the get-go, strongly tied to V8's development roadmap. They (like everyone else in the JS language space at the time) were watching the ESM debate, and had every opportunity to plan for a "few years out" switchover, starting the moment TC39 accepted ESM into the official language spec, with V8 following suit.
They didn't, but thankfully it's never too late: Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely".
Node was, from the get-go, strongly tied to V8's development roadmap. They (like everyone else in the JS language space at the time) were watching the ESM debate, and had every opportunity to plan for a "few years out" switchover, starting the moment TC39 accepted ESM into the official language spec, with V8 following suit.
They didn't, but thankfully it's never too late: Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely".
> Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely".
So this has two consequences:
1. NPM is split in two. All the packages that don't upgrade are useless. If you rely on one, you either need to do the work to port it (and all of its dependencies) before you can even consider migrating your own code. Or pray that someone else will do it.
2. Companies simply won't upgrade. The last version of Node to support CJS will persist for a decade or more. It might get forked and make it harder for Node to actually drive the ecosystem forward. If you think io.js won't happen again as a result of a decision like this, you're mistaken.
So this has two consequences:
1. NPM is split in two. All the packages that don't upgrade are useless. If you rely on one, you either need to do the work to port it (and all of its dependencies) before you can even consider migrating your own code. Or pray that someone else will do it.
2. Companies simply won't upgrade. The last version of Node to support CJS will persist for a decade or more. It might get forked and make it harder for Node to actually drive the ecosystem forward. If you think io.js won't happen again as a result of a decision like this, you're mistaken.
1) no they aren't, just like Python 2.7 libraries still work just fine with python 2.7 and that's fine and entirely on you (HI GRAPHICS INDUSTRY!).
If a 5 year old library doesn't work with current Node, that's perfectly fine. Kind of already the case right now anyway, so nothing new under the sun there. Packages that are still actively used, though, get 4 years to uplift. that is plenty of time.
2) how is that different from companies that still use Node 14 or even 10 in production today because their dependency chains make it impossible to uplift even without taking CJS vs. ESM into consideration?
Everyone who's already stuck in the past is going to stay stuck in the past, anyone who can afford to uplift their codebase over two to four years will be better off, including the entire JS dev landscape. Let's not pretend that the bad habits of others should hold everyone else back. That's how we got here in the first place.
If a 5 year old library doesn't work with current Node, that's perfectly fine. Kind of already the case right now anyway, so nothing new under the sun there. Packages that are still actively used, though, get 4 years to uplift. that is plenty of time.
2) how is that different from companies that still use Node 14 or even 10 in production today because their dependency chains make it impossible to uplift even without taking CJS vs. ESM into consideration?
Everyone who's already stuck in the past is going to stay stuck in the past, anyone who can afford to uplift their codebase over two to four years will be better off, including the entire JS dev landscape. Let's not pretend that the bad habits of others should hold everyone else back. That's how we got here in the first place.
> it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library
A middle ground answer would be using import() in CJS to asynchronously require the ESM module. It would require some hacking around your loading flow to ensure the module loads before you try to do anything with it but would still be preferable to rebuilding the entire thing.
A middle ground answer would be using import() in CJS to asynchronously require the ESM module. It would require some hacking around your loading flow to ensure the module loads before you try to do anything with it but would still be preferable to rebuilding the entire thing.
I have a custom loader that can operate in both sync and async mode with the same syntax; it started off as a single loader to load AMD and CommonJS but I’ve been meaning to add ESM support as well (it was more opaque than I liked when I last tried, required too much browser magic).
https://github.com/mqudsi/loader
https://github.com/mqudsi/loader
I don't disagree! But is there really an argument (other than an ideological one) for why it needs to be an awaited import? Why does it matter that it's async?
ESM (as standardized, not necessarily as implemented in Node) treats import specifiers as URLs. That alone would inherently be async because network requests are async across basically all JS runtimes. ESM was also later extended to support top level await, making module execution itself async.
Node has recently taken a more pragmatic approach, by treating local specifiers as potentially synchronous, allowing sync require() of ESM… which then fails if the required module uses top level await.
Node has recently taken a more pragmatic approach, by treating local specifiers as potentially synchronous, allowing sync require() of ESM… which then fails if the required module uses top level await.
Fetching code at runtime over the network makes sense in a browser, it doesn’t make sense outside of a browser and would actually be a huge security risk without all of the security features that browsers have. So it just reinforces the perception that ESM was only designed for browsers.
Both TC39 and node utterly cocked this up. The design of ESM is bad, async imports are a stupid idea, and node’s implementation has been beset by ideology.
Both TC39 and node utterly cocked this up. The design of ESM is bad, async imports are a stupid idea, and node’s implementation has been beset by ideology.
"Sometimes works, but can also blow up at runtime after any upgrade" seems to me like a "pragmatic cure" that is worse than the existing "disease" of needing to treat all ESM imports as async and/or migrating all uses of `require()` to `await import()`. If incrementally moving a CJS code base to use `await import()` in more places is too hard then you probably have other problems you've overlooked for too long (bad promise usage, callback waterfalls, etc).
(But I'm a hardline "all you need is type=module" sort and think a CJS=>TS emitting ESM "rip off the bandaid" approach to CJS legacy libraries works and is fundamentally easier than most of the "pragmatic" solutions to CJS/ESM interop. It is past time to jettison CJS out the airlock.)
(But I'm a hardline "all you need is type=module" sort and think a CJS=>TS emitting ESM "rip off the bandaid" approach to CJS legacy libraries works and is fundamentally easier than most of the "pragmatic" solutions to CJS/ESM interop. It is past time to jettison CJS out the airlock.)
I'd say it's exactly the opposite. Neither require() or import require an async code structure. The biggest reason I avoid ESM like a plague is that many of my projects are not, and do not need to be, async. Adding import() is a huge hassle because it is async.
Adding an importSync() function would be a perfect solution.
Adding an importSync() function would be a perfect solution.
That was my first thought, and what I attempted first in my project. But it turned out that rewriting everything, while being incredibly tedious, was significantly easier to reason about and ensure continuity than developing my own bespoke dependency loader in a large project where not all portions are well understood.
Thank you for saying this. I got 'flagged' when I said it more irritably. The solution is so simple and the payoff so great that it seems to me that the standards bodies are a bad process.
Me? I'd prefer importSync() to having require() cover ESM files. I think it would be more clear but, either way, it would do exactly what you say, open the CJS world to migration.
Me? I'd prefer importSync() to having require() cover ESM files. I think it would be more clear but, either way, it would do exactly what you say, open the CJS world to migration.
> I'd prefer importSync() to having require() cover ESM files.
My only criticism is that in order to do incremental conversion, both need to work. At that point, you'd might as well just eliminate the cognitive overhead and have one function.
My only criticism is that in order to do incremental conversion, both need to work. At that point, you'd might as well just eliminate the cognitive overhead and have one function.
[deleted]
> It's be synchronous and slow and ideologically impure
Sure. It also "just works" at the CLI. Sometimes I just want to test code quickly at the CLI. I'm not into "architecting" this part of my code as much as ES thinks I want to.
Sure. It also "just works" at the CLI. Sometimes I just want to test code quickly at the CLI. I'm not into "architecting" this part of my code as much as ES thinks I want to.
What about the createRequire function?
Why should I need to go out of my way to do this? Why do I need to care what kind of module I'm importing and reach into the stdlib to run code? It should just work.
To be frank I don't care about these questions, perhaps you're right and it should, but whatever. It's a solution I used in the three cases it was necessary that blocked moving my codebase to ESM.
You’re missing the point. Of course createRequire is a solution, it’s always been. The problem are tools and sub-dependencies that you can’t touch.
I don't know which is which. I don't care. I don't understand the benefit of a top-level await if I can simply await in a different file. I use Typescript which adds a layer in-between anyway. At work, we use Angular, which (I think) uses both Typescript and maybe esbuild. Or Webpack. Does it compile to ESM, or CJS? Who knows, and it will change in 2 years again anyway.
All of that is something that I consider to be platform-level. It's insane that millions of feature-writing devs are expected to know all these arcana.
Then again, it might be fixed™ soon Ⓡ
https://joyeecheung.github.io/blog/2024/03/18/require-esm-in...
All of that is something that I consider to be platform-level. It's insane that millions of feature-writing devs are expected to know all these arcana.
Then again, it might be fixed™ soon Ⓡ
https://joyeecheung.github.io/blog/2024/03/18/require-esm-in...
> All of that is something that I consider to be platform-level
I agree with this, but the whole point of the blog-post is that the "platform" currently handles this rather poorly.
I have yet to see a frontend-project that was bigger than some three-person-garage-hobby that didn't occasionally run into CJS versus ESM issues. Maybe not something that pops up on the radar of all the devs in the project but at least at the level of folks who take care of the setup and whatnot, it often pops up in rather painful fashion. Case in point; a few angular-versions back, umd-bundles were dropped, which at least for the project I worked on caused me quite a lot of headache as some of our tool-chain (most notably our testing-setup) relied on angular shipping commonJS-compatible modules.
It's currently also a major pain for anyone publishing an npm-package, even if it's primarily intended to be run on node. The kind of incantations one has to do are just insane (especially if you dared to import from node:crypto or want to support more than just the latest lts); I've just stopped bothering after tearing my hair out for a weekend to no avail, even though I really wanted to support ESM as well.
I agree with this, but the whole point of the blog-post is that the "platform" currently handles this rather poorly.
I have yet to see a frontend-project that was bigger than some three-person-garage-hobby that didn't occasionally run into CJS versus ESM issues. Maybe not something that pops up on the radar of all the devs in the project but at least at the level of folks who take care of the setup and whatnot, it often pops up in rather painful fashion. Case in point; a few angular-versions back, umd-bundles were dropped, which at least for the project I worked on caused me quite a lot of headache as some of our tool-chain (most notably our testing-setup) relied on angular shipping commonJS-compatible modules.
It's currently also a major pain for anyone publishing an npm-package, even if it's primarily intended to be run on node. The kind of incantations one has to do are just insane (especially if you dared to import from node:crypto or want to support more than just the latest lts); I've just stopped bothering after tearing my hair out for a weekend to no avail, even though I really wanted to support ESM as well.
The whole point of this is that ESM was very poorly handled. It's irrelevant to TypeScript, esbuild, Webpack, etc. That tooling handles the complexity for you and sometimes that backfires. I've run into a lot of headaches with the TypeScript compiler.
The fact that you don't know whether your codebase uses TypeScript, esbuild or Webpack is disappointing. It means that those worries have been handled for you and you don't care to learn them, which is never a good stance if you work with this on a daily basis. But I somewhat agree that it should all be vastly simpler.
I also kind of agree with the downvoted/flagged comment re: Golang. The way the JavaScript ecosystem works is highly dependent on the way Node is handled. And Node has, for many years, made ESM needlessly complicated.
The fact that you don't know whether your codebase uses TypeScript, esbuild or Webpack is disappointing. It means that those worries have been handled for you and you don't care to learn them, which is never a good stance if you work with this on a daily basis. But I somewhat agree that it should all be vastly simpler.
I also kind of agree with the downvoted/flagged comment re: Golang. The way the JavaScript ecosystem works is highly dependent on the way Node is handled. And Node has, for many years, made ESM needlessly complicated.
> The fact that you don't know whether your codebase uses TypeScript, esbuild or Webpack is disappointing. It means that those worries have been handled for you and you don't care to learn them
En contraire. The problem is that some years ago I did learn the basics, but the ground has shifted underneath my feet multiple times. There's not really an incentive to learn the finer points of ESM-vs-CJS if Typescript hides the input into the system, Angular has its own ideas about modules, uses Webpack or babel internally and then spits out CJS anyway.
The whole JS ecosystem is nice if you edit 10 files by hand, but I just don't know what I should do with the knowledge that I should "require cjs" when I need Typescript for type safety and it'll only let me "import ts" anyway.
En contraire. The problem is that some years ago I did learn the basics, but the ground has shifted underneath my feet multiple times. There's not really an incentive to learn the finer points of ESM-vs-CJS if Typescript hides the input into the system, Angular has its own ideas about modules, uses Webpack or babel internally and then spits out CJS anyway.
The whole JS ecosystem is nice if you edit 10 files by hand, but I just don't know what I should do with the knowledge that I should "require cjs" when I need Typescript for type safety and it'll only let me "import ts" anyway.
[deleted]
Now we see the benefit of having adults in charge, e.g. as is the case with golang for example.
Go has the benefit of not having to reach a distributed consensus amongst a variety of individual browser vendors. Try compiling a large Go project with tinygo to get a glimpse of what it's like to have to deal with multiple independent runtimes [1]. If the browser vendors had been able to ship ES4 or ES5 with module support between 1999 and 2009, then Node probably would have implemented it from the very beginning, and there would be no dichotomy between CJS and ESM.
[1] https://github.com/evanw/esbuild/issues/1111
[1] https://github.com/evanw/esbuild/issues/1111
I followed advice of Sindre Sorhus [1] and moved all my packages and apps to ESM year ago and couldn't be happier. Only Jest and ts-jest were problematic, so I replaced them with Vitest. I also never encountered problems because of the so-called dual-package hazard [2]; but IMO this isn't that much different than when you have two copies of React in node_modules - it's simply an npm/dependencies problem, not ESM problem.
[1] https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3...
[2] https://nodejs.org/api/packages.html#packages_dual_package_h...
[1] https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3...
[2] https://nodejs.org/api/packages.html#packages_dual_package_h...
People never wanna work or learn. It's not even that big a pill to swallow.
Refactoring your entire codebase just to use that one ES module that is incompatible with CJS is a big pill to swallow if your codebase is ... big (or you have many).
True, but if you just do it, even if it's a large undertaking, the benefits of ESM-only code are bigger than being able to use more packages. And someday you will have to move to ESM anyway.
The last sentence is a good way of consoling yourself. I hope no one takes his advice seriously and implements it. Good luck with your own nonsense.
On the flip side we have other amazing devs who are also active with cutting edge libraries and simply do the small amount of extra work to make their modules available in both ESM and CommonJS to this day:
https://github.com/WebReflection/uhtml
https://github.com/WebReflection/uhtml
ES Modules are better in every way.
But I do believe they got the syntax wrong - should have been "from fs import { readFile }" so that auto-complete works. Python got that right, but that's the only thing Python got right ;)
But I do believe they got the syntax wrong - should have been "from fs import { readFile }" so that auto-complete works. Python got that right, but that's the only thing Python got right ;)
I tend to agree, but I’ve found it mostly doesn’t matter as I stopped typing import statements almost altogether. I type a reference to the thing I want to import, in the place I want to use it, choose a suggestion from the editor, and let the editor write the import statement as a side effect. It only doesn’t work when dealing with aliases, and even then only when something isn’t cached yet.
You can type the real name, import it with autocomplete, then rename the symbol (f2), which creates the alias.
I should’ve been more clear that I meant import specifier aliases. Eg in TypeScript, this is often determined by `paths` in tsconfig.json. It may also be controlled by `imports` in package.json, or any number of other tools and configs.
[deleted]
That doesn't work so well for default or wildcard imports.
(I'd argue that default + wildcard imports are bad, but ESM went out of its way to have them :shrug:)
(I'd argue that default + wildcard imports are bad, but ESM went out of its way to have them :shrug:)
It works okay for defaults IME, and yeah I generally avoid both but especially namespace (wildcard) imports.
I've been newly doing a lot more front-end development and I've been sort of impressed by the autocomplete. You start typing the name you want to import and it autocompletes the name and module that it comes from. That actually seems more useful to me.
Yep. Especially with relative imports, it’s easier to type `import { myHelperFunc` and let LSP complete `tion } from "../../../util"`.
> But I do believe they got the syntax wrong - should have been "from fs import { readFile }" so that auto-complete works.
Personally, I prefer the "import x from y" format because it makes it easier to visually scan where an import is coming from; fair point about auto-complete though.
Personally, I prefer the "import x from y" format because it makes it easier to visually scan where an import is coming from; fair point about auto-complete though.
You can work around this by typing your import statement inside-out:
import {} from 'fs';
Then move your cursor back inside the {} and you'll have nice autocomplete. Works with object destructuring, too.
import {} from 'fs';
Then move your cursor back inside the {} and you'll have nice autocomplete. Works with object destructuring, too.
I think that particular issue could be solved with tooling - if you can autocomplete from “package name” to a given exported identifier, you must already have a mapping from “package name” to “exported identifier” somewhere in your machine’s memory; duplicate this mapping and swap the keys for values, now “exported identifier” points to “package name” as well.
But I do agree that it should be “from ‘foo’ import { … }”, because then the code would look nicer.
But I do agree that it should be “from ‘foo’ import { … }”, because then the code would look nicer.
I dislike the assumption that natural languages are a good base for the syntax of programming languages,
In my personal contrarian opinion I would have wanted something along the lines of
In my personal contrarian opinion I would have wanted something along the lines of
import "url" with {namedImport}, defaultImport;
import "url" assert {type:"js"} with {namedImport}, defaultImport;
> I think that particular issue could be solved with tooling - if you can autocomplete from “package name” to a given exported identifier, you must already have a mapping from “package name” to “exported identifier” somewhere in your machine’s memory; duplicate this mapping and swap the keys for values, now “exported identifier” points to “package name” as well.
Sorry I don't follow. What the "from ..." syntax does (when typing it) is to narrow down 10k possibilities (the set of all exported functions in all in-scope modules) down to a few dozen. How would this be possible otherwise?
Illustration:
Case 1: import <just about anything is possible here - can't auto suggest>
Case 2: from fs import <only fs functions are in scope, start autosuggesting on keypress>
Sorry I don't follow. What the "from ..." syntax does (when typing it) is to narrow down 10k possibilities (the set of all exported functions in all in-scope modules) down to a few dozen. How would this be possible otherwise?
Illustration:
Case 1: import <just about anything is possible here - can't auto suggest>
Case 2: from fs import <only fs functions are in scope, start autosuggesting on keypress>
A core problem here is that modules are generally a unique key, but exported identifiers are not. My editor does what you're talking about, but can be unhelpful because if I type in "create" I get 83,000 results. That would be much more helpful if it were pre-filtered by module so I get 6 results instead.
I’m so used to fuzzy search at this point that it didn’t even occur to me, but yeah, you’re right.
I guess tooling can still provide a solution - a snippet on “import”, where the first tab stop takes you to the module path string, and the next tab stop jumps back to inside the braces?
I guess tooling can still provide a solution - a snippet on “import”, where the first tab stop takes you to the module path string, and the next tab stop jumps back to inside the braces?
I feel like it’s going to be an inevitable language update to allow that syntax. I’m surprised TypeScript hasn’t succumbed to the pressure to support that syntax, but they’re really strict about where they allow deviations from the JS superset.
> I’m surprised TypeScript hasn’t succumbed to the pressure to support that syntax
They won't do this without consensus in TC39. They shouldn't either; that'd be worse than this niggle.
They won't do this without consensus in TC39. They shouldn't either; that'd be worse than this niggle.
Whether they should or not and whether that would be bad or not depends on how close of a relationship you believe should/does exist between TypeScript and JavaScript l, specifically in the TS to JS direction.
If I said language foo that isn’t TS but transpiled to JS all the same was considering that syntax, people might have a different opinion.
If I said language foo that isn’t TS but transpiled to JS all the same was considering that syntax, people might have a different opinion.
Since Typescript 1.0 it has mostly taken the path of wanting to follow TC-39's standards rather than lead them. The most notable exception was an experimental flag for decorators at too early of a stage that's currently playing out in a compatibility war between the too many projects depending on the experimental flag in Production builds versus the actually standardized behavior.
Between that and some remaining warts from <1.0 mistakes/incorrect assumptions, there's plenty of evidence that it is a good thing that other than its type system TS focuses on following standards rather than trying to lead them.
There's a place for the language "foos" that want to lead and champion new standards. As an ecosystem we all seem to benefit from Typescript not being that language (anymore, mostly not since 1.0 with the few obvious mistakes aside). It's a part of what makes Typescript trustworthy as Production tooling. It's also what helps make Typescript mostly "cheap" and "unextraordinary" in Production build pipelines.
Between that and some remaining warts from <1.0 mistakes/incorrect assumptions, there's plenty of evidence that it is a good thing that other than its type system TS focuses on following standards rather than trying to lead them.
There's a place for the language "foos" that want to lead and champion new standards. As an ecosystem we all seem to benefit from Typescript not being that language (anymore, mostly not since 1.0 with the few obvious mistakes aside). It's a part of what makes Typescript trustworthy as Production tooling. It's also what helps make Typescript mostly "cheap" and "unextraordinary" in Production build pipelines.
This was a great decision, and what distinguishes Typescripts from other guest languages, by constraining themselves with being a linter and compiler, it is the same world.
Not a two world reality where the guest language has its own mini-platform and tooling, on top of the actual platform.
Not a two world reality where the guest language has its own mini-platform and tooling, on top of the actual platform.
It is not that big of a deal, you can write
import {} from "some/package"
Typescript devs acknowledged that one reason of its success is being a linter for JavaScript and little else, hence why they no longer do language level experiements outside the type system.
They've made zero functional difference to me. What do you like about them?
Really? I have a project that loads modules based on calculated values. How will you do that in a pure ESM world?
Using dynamic import. Here's some documentation to get you started.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Dynamic import() is asynchronous. I have things that are not asynchronous.
To rewrite to be asynchronous gets me exactly nothing.
In work that is asynchronous, I use import(). It's fine. Also, it gives me nothing but, since that particular thing is already async, why not?
This is a terrible, badly thought out thing that was done by ECMA that harms a ton of developers and that would have been entirely saved if somebody had asked me.
I would have said, add importSync() and there will be literally no problems ever again.
To rewrite to be asynchronous gets me exactly nothing.
In work that is asynchronous, I use import(). It's fine. Also, it gives me nothing but, since that particular thing is already async, why not?
This is a terrible, badly thought out thing that was done by ECMA that harms a ton of developers and that would have been entirely saved if somebody had asked me.
I would have said, add importSync() and there will be literally no problems ever again.
No code or event handlers could run while importSync is resolving. I think including it would have been a mistake. The only other thing that's remotely similar are alert() and inputBox().
IMO it would be a pandora's box of unending problems. A function doing a synchronous dynamic import creates a bad user experience. It's also a pretty rare circumstance in my experience. Dynamic imports are relatively uncommon to begin with. Having one that must be inside a non-async function is unheard of for me.
If you had your way, I think the net result would have been worse.
IMO it would be a pandora's box of unending problems. A function doing a synchronous dynamic import creates a bad user experience. It's also a pretty rare circumstance in my experience. Dynamic imports are relatively uncommon to begin with. Having one that must be inside a non-async function is unheard of for me.
If you had your way, I think the net result would have been worse.
I don't think it is a big problem given functions like fs.readFileSync already exists. It doesn't have to be a JS standard but be a server-side JS runtime specific function, which can be used when hacks are needed.
Node could just do that. `fs` isn't an ES spec, and nor does `importSync` need to be. The blame can't be placed on ESM for this.
I agree. The blame belongs to the NodeJS steering committee. It's their failure and it's a big one.
The only excuse is that ESM happened during the big Deno Node wars when the community was split, everyone was mad and, apparently making dumb decisions.
I read that a new version of NodeJS has revised require() to bring in ESM modules. That's good enough. (I would prefer importSync() for clarity.)
The only excuse is that ESM happened during the big Deno Node wars when the community was split, everyone was mad and, apparently making dumb decisions.
I read that a new version of NodeJS has revised require() to bring in ESM modules. That's good enough. (I would prefer importSync() for clarity.)
I understand that importSync() is blocking. That is fine when it is fine. I am a programmer. I can make sync vs async decisions.
The browsers would never implement it.
This is not a talk about browsers, hence the readFileSync examples
The reason we got here in the first place was trying to standardize the language and its module loading. If you come up with a solution that doesn't work in browsers, we're back to the same thing. Instead of the `require()` polyfill, we'd need the `importSync` polyfill.
Nor do they have require(). That’s not the point.
And the fact that require() is non-standard is why we're talking about ESM now. Node can cook up another non-standard package loading thing whenever they want. It just won't be ESM. And if you don't care what browsers do, then we're not talking about ESM.
You wouldn’t, because it’s a misunderstanding of what “module” means and how they work. It’s like saying “maybe I need this file” - there is no maybe, you either need it (to run conditional logic upon) or you don’t.
You get an approximation of conditionally loading a module by simply converting everything to ESM and enabling tree-shaking. When module loading has no side effects this is simple to do. If you’re unsure at build time whether you will call a certain function from a certain module and that’s the reason you’re conditionally loading it, again enable ESM and tree shaking and import the function and call it conditionally except do so at runtime. You’ll only get exactly what you need to avoid the bloat of the whole module and your dependency graph will also be more correct as a bonus!
You get an approximation of conditionally loading a module by simply converting everything to ESM and enabling tree-shaking. When module loading has no side effects this is simple to do. If you’re unsure at build time whether you will call a certain function from a certain module and that’s the reason you’re conditionally loading it, again enable ESM and tree shaking and import the function and call it conditionally except do so at runtime. You’ll only get exactly what you need to avoid the bloat of the whole module and your dependency graph will also be more correct as a bonus!
You are simply wrong.
I've been a full time NodeJS programmer for fourteen years. Module isn't some pure theoretical concept. It is a file that exports something in Node.
There's a ton of 'maybe I need this file' in the world. For example, I have things that are polymorphic by instance. I grab a config entry to decide which of several modules are going to be used. The others are completely unnecessary.
You might say, just add a build step that chooses the right ones. To which I would say, Are you nuts? Why would I screw up a perfectly working paradigm?
To get "an approximation of conditionally loading".
I've been a full time NodeJS programmer for fourteen years. Module isn't some pure theoretical concept. It is a file that exports something in Node.
There's a ton of 'maybe I need this file' in the world. For example, I have things that are polymorphic by instance. I grab a config entry to decide which of several modules are going to be used. The others are completely unnecessary.
You might say, just add a build step that chooses the right ones. To which I would say, Are you nuts? Why would I screw up a perfectly working paradigm?
To get "an approximation of conditionally loading".
As a matter of fact _module_ is a pure theoretical concept. The fact that we’re discussing at least two different implementations of modules alone proves that it’s at least more abstract than _file_. It’s certainly not strictly related to Node, plenty of languages (and their associated runtimes, if applicable) have modules. Further, it wouldn’t matter what a specific implementation of modules within Node is, nor would 14 years of experience with Node.
You’re missing the point entirely, perhaps intentionally? If you’re using modules you’re already using a “build step” - code that runs before your actual code. Your “perfectly working paradigm” is only such if you intentionally disregard the core reason for modules: modularity.
You’re missing the point entirely, perhaps intentionally? If you’re using modules you’re already using a “build step” - code that runs before your actual code. Your “perfectly working paradigm” is only such if you intentionally disregard the core reason for modules: modularity.
Your point is that modules has a conceptual underpinning. My point is that the actual world in not the same as that concept.
I use modules for all the reasons. Your assertion that require() or importSync() represents a willful misunderstanding is silly and arrogant.
I use modules for all the reasons. Your assertion that require() or importSync() represents a willful misunderstanding is silly and arrogant.
My assertion is that, having been made aware of the pitfalls of effectful imports, continuing to argue for using them requires a willful misunderstanding.
People outside the context of this discussion could simply be uninformed, they don’t need to be willfully misunderstanding.
People outside the context of this discussion could simply be uninformed, they don’t need to be willfully misunderstanding.
No. It is a result of a practical, engineering perspective. I love theory. It's necessary for moving the world forward.
I also love getting things to work. Three quarters of NPM packages are CJS. Several of them are ones I wrote.
The reason they are not ESM is that the theoretically pure folks had control and I voted with my fingers because everything I do has to fit in my a substantial amount of existing CJS code.
Arguing for practical solutions that accommodate an imperfect world is absolutely correct.
I also love getting things to work. Three quarters of NPM packages are CJS. Several of them are ones I wrote.
The reason they are not ESM is that the theoretically pure folks had control and I voted with my fingers because everything I do has to fit in my a substantial amount of existing CJS code.
Arguing for practical solutions that accommodate an imperfect world is absolutely correct.
> code that runs before your actual code
But this is not a build step. A build step would transform the code beforehand
For good measure, it is an operation that executes by the runtime for every execution, not as a redeployment.
Turns out that the ESM purists struggle to find reasons to condemn those of us who don’t have the need or resources to the chase bleeding edge of purity.
Turns out that the ESM purists struggle to find reasons to condemn those of us who don’t have the need or resources to the chase bleeding edge of purity.
Can someone explain to me what the advantages of ESM actually are to me as a backend dev who uses import / export syntax in TS already?
Parent article mentions static analysis and synchronous loading on startup but that has never been an issue for me despite building some large and complex Node apps over the years.
I’ve looked into this in the past but all I could find are strong opinions without solid technical reasoning.
Parent article mentions static analysis and synchronous loading on startup but that has never been an issue for me despite building some large and complex Node apps over the years.
I’ve looked into this in the past but all I could find are strong opinions without solid technical reasoning.
A big technical reason especially specific to using Typescript: not transpiling to CJS speeds up your Typescript builds.
It also opens up more options when Typescript is just "type stripped" rather than transpiled to an incompatible module syntax. You probably still want a full Typescript compile at CI time to get robust typechecking, and you'll have the Typescript LSP doing its thing in your IDE still, but you can use type-stripping tools like esbuild for very fast type stripping in some portions of your inner dev loop.
The static analysis that ESM supports is handy because it opens up technical benefits like Tree Shaking. Your apps might run on SSDs, you probably don't have a lot of reasons to bundle them, you likely aren't worried about on disk size, you might not be worried about bundle publish size to npm, and so you might not think Tree Shaking applies to you, but V8 under the hood of Node is still going to do Tree Shaking of memory and garbage collection for you with ESM in ways that it simply cannot with CJS. I've seen some real memory performance gains in Node apps just switching from CJS to ESM already, and V8's ESM optimizations only seem to get better as more ESM is deployed in the wild.
There are more, smaller technical benefits, but "type stripping not transpiling" and "in memory tree shaking" are strong ones that are easy to overlook.
It also opens up more options when Typescript is just "type stripped" rather than transpiled to an incompatible module syntax. You probably still want a full Typescript compile at CI time to get robust typechecking, and you'll have the Typescript LSP doing its thing in your IDE still, but you can use type-stripping tools like esbuild for very fast type stripping in some portions of your inner dev loop.
The static analysis that ESM supports is handy because it opens up technical benefits like Tree Shaking. Your apps might run on SSDs, you probably don't have a lot of reasons to bundle them, you likely aren't worried about on disk size, you might not be worried about bundle publish size to npm, and so you might not think Tree Shaking applies to you, but V8 under the hood of Node is still going to do Tree Shaking of memory and garbage collection for you with ESM in ways that it simply cannot with CJS. I've seen some real memory performance gains in Node apps just switching from CJS to ESM already, and V8's ESM optimizations only seem to get better as more ESM is deployed in the wild.
There are more, smaller technical benefits, but "type stripping not transpiling" and "in memory tree shaking" are strong ones that are easy to overlook.
This is good information - thank you. I would need to do some benchmarks to measure the impact - might make for a nice blog post in future! :)
IMO the game changer in your scenario is when type annotations are added as a standard JS feature:
https://github.com/tc39/proposal-type-annotations
At that point (and with ES modules) you'd be able to run your TS code directly without any transpilation necessary. I'd love to remove all the build process junk from my projects and have them run quicker.
I've written a few projects as plain ESM JS with JSDoc type annotations and it truly is a joy to bypass build steps. JSDoc type annotations do get annoying as a project grows to a certain size, though.
https://github.com/tc39/proposal-type-annotations
At that point (and with ES modules) you'd be able to run your TS code directly without any transpilation necessary. I'd love to remove all the build process junk from my projects and have them run quicker.
I've written a few projects as plain ESM JS with JSDoc type annotations and it truly is a joy to bypass build steps. JSDoc type annotations do get annoying as a project grows to a certain size, though.
I would argue the benefit is just making everything much simpler. You never need to think about module systems and interoperability -- there is only one module system to deal with.
That proposal seems to have stalled, and I doubt it's going anywhere. Here is the thing: people want hard, strict compiler checks that can be enforced at all time, not Python-style type hints. If you want JavaScript but type checked, Typescript is the de-facto standard. Either do "checkJS" or convert the project to Typescript. In fact, the latter is much better in terms of expressiveness.
That proposal seems to have stalled, and I doubt it's going anywhere. Here is the thing: people want hard, strict compiler checks that can be enforced at all time, not Python-style type hints. If you want JavaScript but type checked, Typescript is the de-facto standard. Either do "checkJS" or convert the project to Typescript. In fact, the latter is much better in terms of expressiveness.
It only makes it simpler if you have a new project and can be ESM pure. If you are working with existing code or need non-ESM modules, everything is much more complicated.
When I see “stage 1” and “if they stick within a certain reasonably large subset of the language” the cynical old dev in me starts waving a YAGNI sign. :)
Honestly as you read through the proposal the immediate impression is "they're just describing TypeScript syntax", which is... fine, I think? But I do feel like it would be a considerably quicker process for Node to just add the ability to parse a TS file than it will be to go through this formal approval process.
The process is standardizing the ECMAScript language. It would make types (as comments) standard for all JS runtimes, not just Node. I don’t think the community at large wants more non-standard Node stuff, as that’s a frequent point of pain and complaint (see OP article as an example).
Fwiw the github page for that proposal isn't linking to what seems to be the most recent discussion *. I thought type-annotations-as-comments was a no-brainer, but it seems messy. The Sept 2023 discussion shows that it is complicated by the current variety of annotation schemes (TypeScript is not the only one), and the parsing subtleties, which gave rise to unresolved questions about what is the basic motivation. bummer.
* https://github.com/tc39/notes/blob/main/meetings/2023-09/sep...
* https://github.com/tc39/notes/blob/main/meetings/2023-09/sep...
> The aim of this proposal is to enable developers to run programs written in TypeScript, Flow, and other static typing supersets of JavaScript without any need for transpilation,
> if they stick within a certain reasonably large subset of the language. <-------- !!!!!!!!!!
js + type annotations !== typescript
The most recent discussion sums this up with, I think this comment:
> EAO: Okay. So the sense I get overall of this whole proposal, that it’s more of a – it’s a solution looking for a problem that it’s trying to solve a year ago when this got accepted for Stage 1, the problem statement that was in fact considered then was only effective we made up or made concrete during the meeting itself, and then – so what it looks like now is that since then, that problem statement has evolved to this current form of unifying or unforking JavaScript and somehow then presenting type annotations as a way of achieving this result. However, I’ve not been able to find any conversation anywhere or description of how in practice this unification is supposed to happen as a consequence of accepting type annotations.
No one had a good answer to that, because there isn't one.
Anyhow, tldr; this has basically nothing to do with any tangible reason to think ESM is good or useful in anything other than a massively broad, speculative 'maybe in the future' kind of way.
...it is most certainly not any kind of advantage or reason to use ESM for anyone, right now.
> if they stick within a certain reasonably large subset of the language. <-------- !!!!!!!!!!
js + type annotations !== typescript
The most recent discussion sums this up with, I think this comment:
> EAO: Okay. So the sense I get overall of this whole proposal, that it’s more of a – it’s a solution looking for a problem that it’s trying to solve a year ago when this got accepted for Stage 1, the problem statement that was in fact considered then was only effective we made up or made concrete during the meeting itself, and then – so what it looks like now is that since then, that problem statement has evolved to this current form of unifying or unforking JavaScript and somehow then presenting type annotations as a way of achieving this result. However, I’ve not been able to find any conversation anywhere or description of how in practice this unification is supposed to happen as a consequence of accepting type annotations.
No one had a good answer to that, because there isn't one.
Anyhow, tldr; this has basically nothing to do with any tangible reason to think ESM is good or useful in anything other than a massively broad, speculative 'maybe in the future' kind of way.
...it is most certainly not any kind of advantage or reason to use ESM for anyone, right now.
> js + type annotations !== typescript
It's very close though. The proposal is well written and covers the cases where existing Typescript code would not parse correctly/work as intended under the proposal. Most of those cases are pre-1.0 Typescript features still in the language for backwards compatibility but generally frowned upon and that you probably already have strict warnings and linter errors preventing you from using today. The big exception is a lot of TS codebases heavily use `enum` and I've still not seen enough suggestions to tighten the linter warning against it. (But there are at least some of us suggesting avoiding TS `enum` today.)
It's very close though. The proposal is well written and covers the cases where existing Typescript code would not parse correctly/work as intended under the proposal. Most of those cases are pre-1.0 Typescript features still in the language for backwards compatibility but generally frowned upon and that you probably already have strict warnings and linter errors preventing you from using today. The big exception is a lot of TS codebases heavily use `enum` and I've still not seen enough suggestions to tighten the linter warning against it. (But there are at least some of us suggesting avoiding TS `enum` today.)
Being able to copy paste typescript snippets in the browser console is enough of a benefit in my opinion
> Can someone explain to me what the advantages of ESM actually are to me as a backend dev who uses import / export syntax in TS already?
1. The downstream users of your backend library can use ESM JS.
2. You can author (or output) isomorphic code that works in Node.js and other environments.
3. Static analysis tools can more reliably understand your dependency graph, e.g. find unnecessary code/modules/packages.
4. You can use a custom load API to do customized operations. E.g. import a YAML file as a JavaScript object.
If none of these apply to you individually, then you don't have a lot to be gained.
1. The downstream users of your backend library can use ESM JS.
2. You can author (or output) isomorphic code that works in Node.js and other environments.
3. Static analysis tools can more reliably understand your dependency graph, e.g. find unnecessary code/modules/packages.
4. You can use a custom load API to do customized operations. E.g. import a YAML file as a JavaScript object.
If none of these apply to you individually, then you don't have a lot to be gained.
> 1. The downstream users of your backend library can use ESM JS.
ESM code can use CJS packages.
> 2. You can author (or output) isomorphic code that works in Node.js and other environments.
TS compilation makes this possible already - e.g. I have authored API SDK packages that can be used in FE and BE code.
> 3. Static analysis tools can more reliably understand your dependency graph, e.g. find unnecessary code/modules/packages.
An earlier sibling comment mentions this - this is potentially a good reason but I need to do some benchmarking to see the actual impact (would make for a good blog post one day) :)
> 4. You can use a custom load API to do customized operations. E.g. import a YAML file as a JavaScript object.
TIL but it seems like a niche feature that I would never need in BE world.
ESM code can use CJS packages.
> 2. You can author (or output) isomorphic code that works in Node.js and other environments.
TS compilation makes this possible already - e.g. I have authored API SDK packages that can be used in FE and BE code.
> 3. Static analysis tools can more reliably understand your dependency graph, e.g. find unnecessary code/modules/packages.
An earlier sibling comment mentions this - this is potentially a good reason but I need to do some benchmarking to see the actual impact (would make for a good blog post one day) :)
> 4. You can use a custom load API to do customized operations. E.g. import a YAML file as a JavaScript object.
TIL but it seems like a niche feature that I would never need in BE world.
> ESM can use CJS packages
True. Everything is a default import, but it does work.
> I have authored API SDK packages that can be used in FE and BE code.
FE code can't use CJS. You need to have an extra step to convert to ESM (or AMD, webpack chucks, etc).
And the general consensus is that frontend JS has too much complexity.
---
CJS works pretty well for Node.js.
If it didn't, it wouldn't have been adopted.
The primary advantage is authors can write and distribute isomorphic code. (Again, without relying on heavy or kludgy module system conversations.)
True. Everything is a default import, but it does work.
> I have authored API SDK packages that can be used in FE and BE code.
FE code can't use CJS. You need to have an extra step to convert to ESM (or AMD, webpack chucks, etc).
And the general consensus is that frontend JS has too much complexity.
---
CJS works pretty well for Node.js.
If it didn't, it wouldn't have been adopted.
The primary advantage is authors can write and distribute isomorphic code. (Again, without relying on heavy or kludgy module system conversations.)
There isn't that much difference on the backend if you use Node and Typescript. ESM is the standard for JavaScript and you'll probably run into situations where you can't use a new JavaScript feature with CommonJS as early as you can with ESM. There are disadvantages and advantages to how the modules load on Node, but ESM is generally improving faster than CommonJS.
If you're looking into other runtimes ESM used to be the way to go because of Deno, but these days you're likely either running Node or Bun and both work well with CommonJS and ESM.
If you do both frontend and backend work, or if your team has a lot of cross-over between the two sides of the ecosystem, then it'll likely be easier for you to use ESM on both ends as CommonJS isn't supported by browsers.
I think the primary reason CommonJS is still around is mostly because Typescript replaces it's import/export module system with something that's basically similar in syntax to the way ESM does it untill it gets transpiled into Javascript. If people actually had to work with CommonJS modules in 2024 then I think they'd likely go insane.
Not everyone will agree with me on this, but I don't think there is a reason to rewrite old projects into ESM unless you have a very good reason to do so. That being said, there isn't really a good reason to start new projects with CommonJS either... Unless you have a really good reason to do so.
If you're looking into other runtimes ESM used to be the way to go because of Deno, but these days you're likely either running Node or Bun and both work well with CommonJS and ESM.
If you do both frontend and backend work, or if your team has a lot of cross-over between the two sides of the ecosystem, then it'll likely be easier for you to use ESM on both ends as CommonJS isn't supported by browsers.
I think the primary reason CommonJS is still around is mostly because Typescript replaces it's import/export module system with something that's basically similar in syntax to the way ESM does it untill it gets transpiled into Javascript. If people actually had to work with CommonJS modules in 2024 then I think they'd likely go insane.
Not everyone will agree with me on this, but I don't think there is a reason to rewrite old projects into ESM unless you have a very good reason to do so. That being said, there isn't really a good reason to start new projects with CommonJS either... Unless you have a really good reason to do so.
A statically analyzable dependency tree of modules. No more "foo is undefined" when using cyclic require()
I think the module imports apis are a python2/3 moment for node.js ecosystem. There is no clearly superior way and as a consequence not too many people care, however it hurts for real.
The proposal to disable node.js style imports will just split ecosystem and make a large part of industry stick to ancient version / make a fork. Is that really worth the gain? Just check how long it took some bigger projects to migrate from python2 to python3
The proposal to disable node.js style imports will just split ecosystem and make a large part of industry stick to ancient version / make a fork. Is that really worth the gain? Just check how long it took some bigger projects to migrate from python2 to python3
[deleted]
[deleted]
The difference is the language/standard in question neither originated with NodeJS nor is NodeJS now nor has it ever been led by the people behind the language/standard (unlike Python)...
When you hear "NodeJS", you really need to bethinking of it in the same category as IE (wrt browser behavior) or Visual C/C++. It's but one, often (knowingly/deliberately) quirky, non-standard implementation by a group that doesn't necessarily have your best interests at heart or the interests of those outside their own platform umbrella.
When you hear "NodeJS", you really need to bethinking of it in the same category as IE (wrt browser behavior) or Visual C/C++. It's but one, often (knowingly/deliberately) quirky, non-standard implementation by a group that doesn't necessarily have your best interests at heart or the interests of those outside their own platform umbrella.
My only complaint about the current state of ESM is that I can't figure out how to resolve the following:
1. I'm developing a library that depends on d3js
2. I want the library to be usable without any build tools. Just clone my repo and host the files from a static server. Or just import directly from jsdelivr
3. I also want people to be able to use NPM to install my library if they so choose
The problem is if I vendor d3js, then developers who consume my library via NPM might end up with 2 copies of d3js in their app, if their app also uses d3 directly. But if I don't vendor it, then my ESM-only users have to use an import map to resolve the bare specifier in the browser, which is kind of ugly and confusing.
More details: https://stackoverflow.com/q/78645299/943814
1. I'm developing a library that depends on d3js
2. I want the library to be usable without any build tools. Just clone my repo and host the files from a static server. Or just import directly from jsdelivr
3. I also want people to be able to use NPM to install my library if they so choose
The problem is if I vendor d3js, then developers who consume my library via NPM might end up with 2 copies of d3js in their app, if their app also uses d3 directly. But if I don't vendor it, then my ESM-only users have to use an import map to resolve the bare specifier in the browser, which is kind of ugly and confusing.
More details: https://stackoverflow.com/q/78645299/943814
That's when I started vite first time. I think it's the build tool closest to no build tool.
Why does it matter that people use ES modules instead of requires? They’re compatible enough. Javascript is weird because it has this directive for browsers to keep compatibility, but then has proponents for language changes that people try to force on everyone through framework/library use and design. All to the benefit of someone reading code the way they want to read code.
It hasn’t changed because it’s not a real problem. This is like forcing main as the default branch in git.
It hasn’t changed because it’s not a real problem. This is like forcing main as the default branch in git.
Well, to begin with, they are NOT compatible, in the general sense. What you perceive to be compatible works because of tons of (often dirty and ugly) hacks people put together, with many issues. (Hint: top-level await). Do your research.
Hence the “enough” modifier. NodeJS isnt a browser framework, do your research.
I could not agree more. In fact, the introduction of ESM has been entirely pernicious, causing more problems, not fewer.
Why? Because there is no importSync(). If that existed, it would be easy to interoperate.
Why? Because there is no importSync(). If that existed, it would be easy to interoperate.
I think it would have been more prudent if Node had chosen to stay closer to Babel compatibility in terms of how it handled ESM + Require. I know there were reasons to break things... but it could have been smoother.
Why do you want an `importSync()`?
You could achieve the same thing with a bunch of `import()` calls, but I'm not sure why you'd want to when you can just use the `import` keyword instead.
You could achieve the same thing with a bunch of `import()` calls, but I'm not sure why you'd want to when you can just use the `import` keyword instead.
I have a ton of projects and a huge amount of utilities and such, that all use CJS.
You cannot just use 'import' if you also use CJS. To use 'import', it must be an ESM module and the you cannot use require() to access CJS utilities.
In CJS code, one can use import() to bring in ESM modules but, import() is asynchronous. The problem is that I have a lot of code that is not asynchronous. To add an asynchronous function requires substantial rewriting that I am rarely willing to do.
If, in addition to the asynchronous import(), there was a synchronous importSync(), I could use ESM modules any time I want. That would allow me to...
1) Use and support new utilities that are ESM.
2) Write new utilities that are ESM. Presently, I always target CJS because that's what I have to use.
I have read that there is a new version of NodeJS that supports using require() for ESM modules. That will be a huge, huge, huge improvement although I would prefer importSync() so that it was clear when I was using an ESM module. Even so, I will take it.
You cannot just use 'import' if you also use CJS. To use 'import', it must be an ESM module and the you cannot use require() to access CJS utilities.
In CJS code, one can use import() to bring in ESM modules but, import() is asynchronous. The problem is that I have a lot of code that is not asynchronous. To add an asynchronous function requires substantial rewriting that I am rarely willing to do.
If, in addition to the asynchronous import(), there was a synchronous importSync(), I could use ESM modules any time I want. That would allow me to...
1) Use and support new utilities that are ESM.
2) Write new utilities that are ESM. Presently, I always target CJS because that's what I have to use.
I have read that there is a new version of NodeJS that supports using require() for ESM modules. That will be a huge, huge, huge improvement although I would prefer importSync() so that it was clear when I was using an ESM module. Even so, I will take it.
[deleted]
Or you can use Bun and have it handle all the nonsense. Or esbuild, but then you get a big blob, which isn’t really usable for a lot of things.
The extensions were always silly to me. Who changes only a few files to esmodules? You either change your whole project or not at all.
The extensions were always silly to me. Who changes only a few files to esmodules? You either change your whole project or not at all.
> Who changes only a few files to esmodules? You either change your whole project or not at all.
Transition. It’s a much lighter lift to transform a project piece by piece than do the whole thing.
Transition. It’s a much lighter lift to transform a project piece by piece than do the whole thing.
Bun handles all of the nonsense but the catch is that it blows up both in dev and in production. Wake me up when Bun doesn't segfault/fail to work at all.
Does it actually handle it though? I use bun to run my remix app at work and I have run into ESM/CJS issues.
I look forward to the day when Bun can run Next.js. Apparently the main showstopper is in the Next.js router.
I'm not steeped in the history of this issue but I periodically run a maintained and up to date NextJS app via Bun in a dev env just to monitor performance and compatibility. It uses the app router and the edge runtime middleware, hosted from a Docker container. I haven't seen massive benefits using Bun since Next doesn't use the Bun-specific libraries that make their performance numbers great, but I haven't seen any game breaking issues either, it runs and all the app's E2Es pass.
Just use JSR[0] and only deal with npm when a project forces you to do things backwards. Since JSR packages are available on npm, there's nothing lost.
[0]https://jsr.io/
[0]https://jsr.io/
Neat to get to know JSR, but doesn't this have very little to do with the contents of the article you responding to? While I know NPM is in the title, the actually article is about module imports which JSR doesn't change.
The article is saying "this ecosystem should change because it doesn't do specific things that make sense for it to do".
JSR says "here's a way to leave the original ecosystem alone, but have a better-fitting ecosystem that does the things you want". (not sure about the 'jsr doesn't change' bit? It literally requires ES modules, in lieu of common js modules).
Seems pretty relevant, to me, but YMMV. I understand it's not a direct answer, but it seems as relevant as "here's typescript" to someone saying "we should add types to javascript".
JSR says "here's a way to leave the original ecosystem alone, but have a better-fitting ecosystem that does the things you want". (not sure about the 'jsr doesn't change' bit? It literally requires ES modules, in lieu of common js modules).
Seems pretty relevant, to me, but YMMV. I understand it's not a direct answer, but it seems as relevant as "here's typescript" to someone saying "we should add types to javascript".
jsr seems great! I hope they work out their kinks but I'm really glad someone is trying to push things forward.
I have mixed feelings about the JSR requirement for explicit typing. Other than that, it is pretty great!
Hm? This is news to me. As far as I know, you can just publish pure javascript. I know that it's going to give you a worse 'score' if you don't have types, and it may require explicit typing of TS (but, strictly, so does TS), but I wasn't aware that there was not any way to publish to JSR without types. I'll have to look in to that, thanks!
This is the best feature imo. Never again will my time be drained by someone else’s failure to type their spaghetti.
At the very least, they finally should switch over to ESM-by-default and announce that 2 or 3 major versions in advance. "From Node 25 all code is assumed ESM unless you have `"type": "commonjs"` in your package.json" is not a particularly difficult message to send out and would stop people from writing new projects using the now legacy CJS (super great that it existed back when JS had nothing even close to a dependency model, but it should have been retired once ESM went from stage 4 to "this is literally and officially how JavaScript works")
The fact that `npm init` still not default to `type: module` is baffling
>NodeJS can officially drop support for require and module.exports in a future version, creating a bit more pressure to migrate.
This will never, ever happen. Too much of the foundational ecosystem relies on it.
This will never, ever happen. Too much of the foundational ecosystem relies on it.
> This will never, ever happen.
I hope for the sake of the nodjs ecosystem, you are wrong.
Fragmentation issues are one of the many reasons that nodejs struggle with adoption.
I hope for the sake of the nodjs ecosystem, you are wrong.
Fragmentation issues are one of the many reasons that nodejs struggle with adoption.
> nodejs struggle with adoption
Woah there. If nodejs is struggling for adoption I don’t know what isn’t.
Woah there. If nodejs is struggling for adoption I don’t know what isn’t.
Let e put it differently: nodejs would have more adoption if it had less perceived instability.
If it does happen, it'll only cause more fragmentation. Because lots of projects and libraries simply won't upgrade. Look at Python: it took them decades to get folks to upgrade to py3k because they broke the old code. Do it to Node and it'll be decades of fragmentation.
Breaking code that doesn’t need to be broken is the biggest redflag for a platform or foundational library.
> Too much of the foundational ecosystem relies on it.
Not saying it needs to be done, but this can possibly be handled via a tranpiler layer. It might make CommonJS a bit slower to startup (but fully functional), which is an added incentive to move.
Again, I don't have any opinion on whether this is a good thing. Like you said, too many modules depend on it.
Not saying it needs to be done, but this can possibly be handled via a tranpiler layer. It might make CommonJS a bit slower to startup (but fully functional), which is an added incentive to move.
Again, I don't have any opinion on whether this is a good thing. Like you said, too many modules depend on it.
You don’t really even need a transpiler. Require is basically just a function which loads a file, wraps it in some boilerplate, and evals it. You could just rip that runtime code out of node and provide it as a compatibility library.
> but this can possibly be handled via a tranpiler layer. It might make CommonJS a bit slower to startup (but fully functional), which is an added incentive to move.
"The new version runs your old code slower and it's enough to notice" would be a death blow to the ecosystem. No major company is going to upgrade to that. It's already a project to drag a major codebase kicking and screaming to a new LTS release of your runtime (even if it makes code faster), do you really think folks will invest the time to work to make their code run slower unless they invest even more effort and modify ~every module in their codebase?
ESM isn't implicitly good. The benefits of ESM are almost entirely theoretical for pretty much everyone who is content with CJS. I manage codebases right now that total half a million lines of typescript and js and I think I simply wouldn't notice any improvements from converting any of the code. The point being: this isn't a problem for me or people in similar situations, it's a problem for the folks working on Node and the standards. Forcing folks to upgrade or making their code worse to exert any kind of pressure does nothing to help anyone.
"The new version runs your old code slower and it's enough to notice" would be a death blow to the ecosystem. No major company is going to upgrade to that. It's already a project to drag a major codebase kicking and screaming to a new LTS release of your runtime (even if it makes code faster), do you really think folks will invest the time to work to make their code run slower unless they invest even more effort and modify ~every module in their codebase?
ESM isn't implicitly good. The benefits of ESM are almost entirely theoretical for pretty much everyone who is content with CJS. I manage codebases right now that total half a million lines of typescript and js and I think I simply wouldn't notice any improvements from converting any of the code. The point being: this isn't a problem for me or people in similar situations, it's a problem for the folks working on Node and the standards. Forcing folks to upgrade or making their code worse to exert any kind of pressure does nothing to help anyone.
An insanely bad idea.
This is timely to me because I am in the middle of modernizing a project that was ejected from CRA years ago and now won’t build in Node 18.
I’ve worked on a few big React projects but haven’t really looked much into how the build works, I found out upgrading one thing forced me to upgrade other things and I wound up making a lot of changes by hand to the build scripts and figured I’d probably screw something up. Dependencies changing from CommonJS to ESM was probably the most common problem that can frequently solved by version bumps (at risks of adopting changes you don’t want)
At some point I decided to try the alternate path of switching to Vite for a build system since I’ve had good luck working with it for some VR side projects.
It’s funny how you can code on front-end Javascript and not need to learn about CommonJS until something like this hits.
I’ve worked on a few big React projects but haven’t really looked much into how the build works, I found out upgrading one thing forced me to upgrade other things and I wound up making a lot of changes by hand to the build scripts and figured I’d probably screw something up. Dependencies changing from CommonJS to ESM was probably the most common problem that can frequently solved by version bumps (at risks of adopting changes you don’t want)
At some point I decided to try the alternate path of switching to Vite for a build system since I’ve had good luck working with it for some VR side projects.
It’s funny how you can code on front-end Javascript and not need to learn about CommonJS until something like this hits.
Surely, this will be the thing tacked onto JavaScript that will make it easy to scale and reduce toolchain complexity......
Bun put a lot of work into making both “import” and “require” always work regardless of whether it’s given a commonjs or an ESM target. I’d say that’s half of the right idea: make only import work with anything.
Another angle that might be effective: take the most popular aspect of commonjs - `require(‘extensionless-string’)` - and tie it to the least popular aspect, .cjs extensions.
Another angle that might be effective: take the most popular aspect of commonjs - `require(‘extensionless-string’)` - and tie it to the least popular aspect, .cjs extensions.
But why do you want to break half of a working solution?
In fairness - given a choice and some time to think, I would probably not choose to actively break commonjs. However, if I was tasked with increasing ESM adoption, I would go about it like this.
It's closer to how Babel worked out of the box for half a decade before Node added their implementation. It's Node that broke a half-working solution.
As someone who only dabbles in JS, one problem is that ESM makes polyfills impossible. And polyfills are mandatory in the JS ecosystem.
It's quite meaningful for dependencies to be fetched asynchronously, but sometimes you really need something to be executed in the order it's written.
It's quite meaningful for dependencies to be fetched asynchronously, but sometimes you really need something to be executed in the order it's written.
I would argue that polyfills are not needed anymore, you are safe using ES2022 or even ES2023.
Mostly agree with the above, outside some ES3-ish scripting environments (Adobe and others), polyfills for a lot of functionality isn't needed strictly for JS support. And even then, can still be part of bundling or shimmed outside your application script(s).
And if you're bundling, then you can add them into the process. I've been avoiding most things that could require a polyfill since 2018 or so anyway as green browsers are pretty feature complete. As much as I'd like the F# style pipelines, and there are a couple other niceties, I don't miss much.
And if you're bundling, then you can add them into the process. I've been avoiding most things that could require a polyfill since 2018 or so anyway as green browsers are pretty feature complete. As much as I'd like the F# style pipelines, and there are a couple other niceties, I don't miss much.
Polyfills are often necessary because new features are still added to the standard library, and runtimes all have varying levels of support for new features.
False, I just added one the other day for the scheduler API.
And then here is one of the biggest backend JS frameworks (NestJS) clinging to CJS/holding off on migrating to ES modules. https://github.com/nestjs/nest/issues/13319
And, if you read that issue, you will see that they are not because the only real benefit is spiritual, 'more consistent with the future'.
Give them an importSync() and the migration can happen incrementally.
Give them an importSync() and the migration can happen incrementally.
The whole node npm ecosystemand tooling is so bad and broken. No other language is so frustrating. JavaScript was bad in the early years took a good way but is now stuck again. I tried webcomponents with lit. All very easy in the beginning. But when you try to use an external stylesheet you hit the wall of the modern js world: css can not be imported in js without a massive tool stack.
I whish tue main focus would be tool- and buildless. This is really what is lacking
> The whole node npm ecosystemand tooling is so bad and broken. No other language is so frustrating.
Neither Node nor NPM are languages.
> when you try to use an external stylesheet you hit the wall of the modern js world: css can not be imported in js without
Uh...?
Neither Node nor NPM are languages.
> when you try to use an external stylesheet you hit the wall of the modern js world: css can not be imported in js without
Uh...?
[deleted]
I'm tempted to say we need a complete reset of the NodeJS ecosystem.
It will never happen because it would require coordination and money, but instead of having millions upon millions of different NPM packages, many of which are downright harmful, we should have a careful selection of maybe the top 20,000 or so.
And then maybe call the next generation of node something else, maybe ProJS.
It will never happen because it would require coordination and money, but instead of having millions upon millions of different NPM packages, many of which are downright harmful, we should have a careful selection of maybe the top 20,000 or so.
And then maybe call the next generation of node something else, maybe ProJS.
Isn't the problem that ESM is proving almost too much of a reset?
I'm not sure I understand what you're going for.
I'm not sure I understand what you're going for.
if you look at most npm packages, you can see that versions that do not support es modules are downloaded more. and it's been like this for years. an example package: https://www.npmjs.com/package/p-queue?activeTab=versions
Barely a third of "high-impact" packages on npm are ESM. And that's with a generous definition of what an ESM package is. [1]
[1]: https://github.com/wooorm/npm-esm-vs-cjs
[1]: https://github.com/wooorm/npm-esm-vs-cjs
Node should just do like bun and support intermixing both. It was a mistake to force this schism -- untold hours of frustration and busywork for maintainers and developers with no hope of actually "completing" a mythical full transition to the new world.
And for what? In Node specifically, it's not as if esm actually solves any real problem! In the greater ecosystem, sure it has some benefits, but Node doesn't even have to choose. Just support both at once, like bun and build systems have for a while, and let's move on from this nonsense.
And for what? In Node specifically, it's not as if esm actually solves any real problem! In the greater ecosystem, sure it has some benefits, but Node doesn't even have to choose. Just support both at once, like bun and build systems have for a while, and let's move on from this nonsense.
“Just” is doing a lot of heavy lifting here. It’s so frustrating watching people think it’s just because a bunch of smart and hard working people are either lazy or stubborn.
I'm completely clueless w.r.t js modules, but I'm wondering how did Bun manage to do it? Does it work well?
It works great for 90% of use cases, but getting that last 10% to work is really really hard so Bun (and node 22 which supports the same thing with an experimental) just throws an error in those cases. The most notable thing is `require`-ing an async ESM module from CJS, because require is a synchronous call you cant just async-ify it trivially.
[deleted]
totally unrelated, but for the first time did i see a discuss here to threads. i had to double check if it was really the threads by instagram.
not being able to mock es modules in tests is a real pain
Jesus, what the hell?!
Why do you dislike `require()` so much?
Just imagine a person who doesn't write modules in Node.JS, but he would like his small scripts to be placed in one JS-file - without any additional directories and `package.json`. His script, for example, updates data for a desktop widget, is easily bypassed by standard nodejs modules and he has hundreds of such scripts in his folder. Why does he need all this `import` overhead?
Why do you dislike `require()` so much?
Just imagine a person who doesn't write modules in Node.JS, but he would like his small scripts to be placed in one JS-file - without any additional directories and `package.json`. His script, for example, updates data for a desktop widget, is easily bypassed by standard nodejs modules and he has hundreds of such scripts in his folder. Why does he need all this `import` overhead?
It’s such a waste of time to have this change so often. Can we just stop changing things? Require and module.exports was fine.
old thing bad
new thing good
new thing good
I don’t care if it’s good or bad, I just want it to work, and have it still work 10 years from now
[deleted]
[deleted]
Completely disagree on getting rid of .mjs, .mts, .cjs, etc. This has actually solved a lot of the problems we had of mixed module loading IMO
It's probably going to be a bandaid for a while longer, but the problem is "mixed module loading" and we should fix that problem rather than rely on bandaids long term. The last .cjs file I've needed for a while now in type="module" libraries has been for eslint config files and eslint finally supports ESM configs. Everything is just .js, as it should be.
I agree with the article that type="module" should be the default and once it is the well established default the bandaid of .mjs, .cjs, et al extensions should probably provoke a deprecation warning rather than being another long-term tech debt part of Node.
I agree with the article that type="module" should be the default and once it is the well established default the bandaid of .mjs, .cjs, et al extensions should probably provoke a deprecation warning rather than being another long-term tech debt part of Node.
And yet approximately 100% of js/ts devs are using ES module syntax and don’t write blog posts about it. Magic.
According to the article, three quarters of the npm projects do not use ESM. Their programmers must be very magical.
Yes, magic of transpilers (typescript, esbuild, swc, babel etc).
tqwhite(1)
Or JS ecosystem should drop ES modules since they have only brought pain and unnecessary complexity without real benefits for years.
IMO, it's the best thing to happen to Node. It (kinda) unifies browser and server syntax, and across competing server-side JS frameworks (Node, Deno, Bun). CommonJS was clearly a NodeJS thing.
We embraced ESM a couple of years back. It was a bit harder earlier on; for example our test frameworks didn't support mocking ESM code well enough. But now there's hardly anything to complain about. There's no going back really - we wouldn't want to.
We embraced ESM a couple of years back. It was a bit harder earlier on; for example our test frameworks didn't support mocking ESM code well enough. But now there's hardly anything to complain about. There's no going back really - we wouldn't want to.
>IMO, it's the best thing to happen to Node. It (kinda) unifies browser and server syntax, and across competing server-side JS frameworks (Node, Deno, Bun). CommonJS was clearly a NodeJS thing.
what the fuck was the point of unifying this particular thing at such a great PITA cost, when obviously none of node APIs work in the browser?
should things like Buffer be removed too? the browser only has Uint8Array, after all.
what the fuck was the point of unifying this particular thing at such a great PITA cost, when obviously none of node APIs work in the browser?
should things like Buffer be removed too? the browser only has Uint8Array, after all.
ES modules work natively in browsers, which are the slow-moving ships of the ecosystem. Changing course for them is completely unrealistic.
There’s a billion browser clients out there. That ensures that whatever API they ship, an increasing amount of code will be written directly against it. Everybody else just needs to adapt or be seen as incompatible with the baseline of JavaScript.
There’s a billion browser clients out there. That ensures that whatever API they ship, an increasing amount of code will be written directly against it. Everybody else just needs to adapt or be seen as incompatible with the baseline of JavaScript.
Show me a real product on the web that does not compile or bundle and uses native modules for code delivery. It's a dead technology that only fragments the ecosystem and makes life harder.
The web is not just for "real products". It's also a platform with tens of millions of hobbyists and dabblers writing HTML and JavaScript.
I promise you they will use any feature that's documented on MDN as being available in all browsers. And ES modules do make life easier for someone who's just writing HTML and JS and doesn't know anything about build tools.
IMO, it's the wrong attitude that we should focus on the professional ecosystem and forget about this use case. Probably most people writing front-end code got their start by creating a HTML file in Notepad (or equivalent) and loading it via file:// into the browser.
I promise you they will use any feature that's documented on MDN as being available in all browsers. And ES modules do make life easier for someone who's just writing HTML and JS and doesn't know anything about build tools.
IMO, it's the wrong attitude that we should focus on the professional ecosystem and forget about this use case. Probably most people writing front-end code got their start by creating a HTML file in Notepad (or equivalent) and loading it via file:// into the browser.
Well, that's what I'm talking about. These web standards are not for real use, but for idealists who do not develop real products. And because of their idealistic desires, we ended up with a fragmented ecosystem and made the life of developers more difficult. Good job.
By 'real product' you mean yet another generic 'web app' that does the same as other hundred generic 'web apps' but its marketing team says differently?
By 'real product' I mean real existing product in real world, not in the fantasies of native_js witnesses.
It's a great development experience today: no waiting for bundlers, easy inner dev loop, let the Browser do all the loading just like Web 1.0 and JS development was maybe supposed to feel.
It will become more common in products. There are going to be startups that see that great development experience and long for some returns towards "just FTP it to the web server as-is" CI/CD processes. There are going to be more product owners everywhere (startups, Enterprise) that think that HTTP/2+ adoption (or even HTTP/1.1 properly configured) is acceptable in the real world and that they are done with most of the needs for bundling.
It will become more common in products. There are going to be startups that see that great development experience and long for some returns towards "just FTP it to the web server as-is" CI/CD processes. There are going to be more product owners everywhere (startups, Enterprise) that think that HTTP/2+ adoption (or even HTTP/1.1 properly configured) is acceptable in the real world and that they are done with most of the needs for bundling.
Ah, no bundlers, no minification, no types, a dream! So, real product example pls? And how you templating your html? Not JSX, obviously, so, string literals?
Personally, I'm doing no bundlers (of my own code) and no minification in my development loop, but still using types with tsc --watch. Typescript is faster the closer it gets to just "type stripping" (where the target and module formats are both something like `es${currentYear - 2}`, especially in the default emit pattern where the JS files just get output side-by-side the TS files they input). TSX to JS transformation, with Typescript (as the only tool in that chain [0]), isn't much extra work to slow it down. (TSX is a pretty easy transformation from its syntax to simple function calls. The hard part of TSX is getting types right.)
[0] So no Babel, only Typescript. You probably don't need Babel in 2024.
(ETA: I've even documented this development pattern for a library I built: https://worldmaker.net/butterfloat/#/getting-started)
[0] So no Babel, only Typescript. You probably don't need Babel in 2024.
(ETA: I've even documented this development pattern for a library I built: https://worldmaker.net/butterfloat/#/getting-started)
I was also reminded that I blogged about all of this back in March, too: https://blog.worldmaker.net/2024/03/13/node-packaging/
Many bundlers output module format - it makes features like code splitting (chunking into separate files) convenient.
ES modules mean you don't need to bundle your code; you just include your index.js in HTML, and all 30,000 JS files of your project come to the user's browser without trouble or delay (let's wish them luck, lol). Since you're bundling, it doesn't matter which module type you use; CJS has worked with code splitting perfectly for over 10 years. However, it's a pain every time you try to import a CJS library in your ESM or vice versa. The truth is, you can't just drop all legacy CJS packages in most real-world projects.
> “all 30,000 JS files of your project come to the user's browser without trouble or delay (let's wish them luck, lol)”
If only there was something in the HTTP protocol to make it more efficient to load multiple requests from the same server. Alas that must be a pipe dream, and every little image and script is loaded separately.
Oh wait, it’s not 1996 anymore…?
If only there was something in the HTTP protocol to make it more efficient to load multiple requests from the same server. Alas that must be a pipe dream, and every little image and script is loaded separately.
Oh wait, it’s not 1996 anymore…?
I more or less agree, though I could get used to just about anything.
( see also: ES modules are terrible https://news.ycombinator.com/item?id=29137663 )
To suggest deprecating CommonJS so as to reduce fragmentation seems so bizarre to me. Audacious.
( see also: ES modules are terrible https://news.ycombinator.com/item?id=29137663 )
To suggest deprecating CommonJS so as to reduce fragmentation seems so bizarre to me. Audacious.
Couldn’t disagree more. It’s wonderful having the same module syntax on server and client.
Huh? You need to elaborate a bit before making a statement like that, friend.
I had been using adminjs at work. Their new major version was ESM-only, and it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library. I expect that's the situation at hundreds of thousands of other companies.
Like for all the belly aching that happened over async functions (and the whole function color rant), synchronous and asynchronous functions work together just fine through plain old promises. You can easily use async functions alongside functions that use old fashioned callbacks. ESM vs CJS is a file coloring (versus function coloring) problem, but there's no interoperability. There's no escape hatch when you just need one file to use another file but their colors are incompatible.