Speeding up Prettier locally and on your CI with dprint(david.deno.dev)
david.deno.dev
Speeding up Prettier locally and on your CI with dprint
https://david.deno.dev/posts/faster-prettier-with-dprint/
52 comments
To be clear, I created this site quickly on Deno Deploy this weekend (I'm the author) and didn't bother getting a proper domain yet. Hacker news is cutting off the subdomain of david.deno.dev and this is just a personal post about the dprint CLI and dprint-plugin-prettier, which aren't available in Deno's CLI.
Deno does use some other Rust-based dprint plugins for deno fmt though and dprint-plugin-prettier uses an embedded Deno runtime with Prettier snapshotted in it. Also, Deno now has a similar built-in incremental formatting and linting as of 1.21 last week, so those subcommands finish almost immediately after the first run... though they were already very fast https://deno.com/blog/v1.21#incremental-formatting-and-linti...
Deno does use some other Rust-based dprint plugins for deno fmt though and dprint-plugin-prettier uses an embedded Deno runtime with Prettier snapshotted in it. Also, Deno now has a similar built-in incremental formatting and linting as of 1.21 last week, so those subcommands finish almost immediately after the first run... though they were already very fast https://deno.com/blog/v1.21#incremental-formatting-and-linti...
Wow... this is so seamless I didn't even notice VSCode using `deno fmt` under the hood. I'm gonna have to double check to make sure I don't have Prettier running.
> This might go against the grain nowadays, but I'm really productive with JavaScript/TypeScript
Big number of people pointing out the quirks about a language and its ecosystem does not mean that they are not good in terms of productivity, it just means that they are just used a lot.
"There are only two kinds of languages: the ones people complain about and the ones nobody uses" — Bjarne Stroustrup
The complaining actually helps, such as the example you can see in the node.js ecosystem, better and better tools are being built every day, and also "better" alternatives like Deno (in quotes because IMHO it still needs to prove itself).
When Python also got big because of the ML uprising, there was a time that we saw a post complaining about it every day. Same was true for PHP like 10-15 years ago.
So don't feel the need to explain why you chose your stack unless you really want to. Languages don't have feelings, you can leave them and learn a new one if it helps you accomplish things.
Big number of people pointing out the quirks about a language and its ecosystem does not mean that they are not good in terms of productivity, it just means that they are just used a lot.
"There are only two kinds of languages: the ones people complain about and the ones nobody uses" — Bjarne Stroustrup
The complaining actually helps, such as the example you can see in the node.js ecosystem, better and better tools are being built every day, and also "better" alternatives like Deno (in quotes because IMHO it still needs to prove itself).
When Python also got big because of the ML uprising, there was a time that we saw a post complaining about it every day. Same was true for PHP like 10-15 years ago.
So don't feel the need to explain why you chose your stack unless you really want to. Languages don't have feelings, you can leave them and learn a new one if it helps you accomplish things.
I just spent the better part of an hour trawling through issues in the CRA, Webpack, and Axios github issues pages, pertaining to the webpack4 -> webpack5 (10/10/2020) release.
I do not believe there is willpower for institutional changes to the javascript ecosystem that exists on timelines of less then 5 years.
I do not believe there is willpower for institutional changes to the javascript ecosystem that exists on timelines of less then 5 years.
Yeah I recently went through something similar too. Spent a ton of time just digging through GitHub issues.
Basically I've been dealing with the ESM changes. Add in the TypeScript complexity and all of the sudden I've now spent hours debugging some obscure issue rather than being productive... I've debated moving away from Node.js altogether because of it but there's no real realistic alternatives right now (for frontend development/bundling using React/Remix).
I probably can't do any better, but the whole ESM migration has been completely mishandled. Even more infuriating is the response that things are working _as expected_... Hopefully Deno gains some traction and we can put this all behind us.
Basically I've been dealing with the ESM changes. Add in the TypeScript complexity and all of the sudden I've now spent hours debugging some obscure issue rather than being productive... I've debated moving away from Node.js altogether because of it but there's no real realistic alternatives right now (for frontend development/bundling using React/Remix).
I probably can't do any better, but the whole ESM migration has been completely mishandled. Even more infuriating is the response that things are working _as expected_... Hopefully Deno gains some traction and we can put this all behind us.
Honestly I'd ditch CRA and hit up Vite nowadays. You will have to deal with a few poorly made packages (e.g. from Amazon) but it is a breath of fresh air.
I'm using Vite for a small greenfield project. Perhaps it's the selection of packages I use, but setting up was rather painful.
Vite Vue + Typescript template run fine in dev mode but could not be built for production, thanks to a combination of an Vue issue and a tsconfig flag.
Node polyfills work on `yarn dev`, but throw "kn.nextTick is not a function" for production build. Apparently I have to use package `rollup-plugin-polyfill-node` instead of `rollup-plugin-node-polyfills`.
Now with all the setting up out of the way, it does feels good to have a dev server starting almost instantly, hot reload and full page reload stay constantly fast. Sometime though hot reload just fails and I had to manually reload the page, or manually remount the component I'm working on (by navigating back and forth) to see the changes.
It also feels like I'm not even using typescript but plain javascript. Vite would happily serve broken code and some errors will only be shown on `yarn build`. I lost a watch compiler essentially.
Vite Vue + Typescript template run fine in dev mode but could not be built for production, thanks to a combination of an Vue issue and a tsconfig flag.
Node polyfills work on `yarn dev`, but throw "kn.nextTick is not a function" for production build. Apparently I have to use package `rollup-plugin-polyfill-node` instead of `rollup-plugin-node-polyfills`.
Now with all the setting up out of the way, it does feels good to have a dev server starting almost instantly, hot reload and full page reload stay constantly fast. Sometime though hot reload just fails and I had to manually reload the page, or manually remount the component I'm working on (by navigating back and forth) to see the changes.
It also feels like I'm not even using typescript but plain javascript. Vite would happily serve broken code and some errors will only be shown on `yarn build`. I lost a watch compiler essentially.
I wouldn’t really use my build/dev process to say my errors as they’re not parsed by vscode. I generally rely on either vscode inline type checking or a parsed tsc watch task that vscode generates automatically and helpfully displays the output of in my problems pane. Bundling and typechecking are different.
I’ve had none of these issues. `tsc` should work for you out of the box. It also uses Rollup for bundling, so there is no reason you can’t build production packages with it.
Until you get to testing, and c8, and you find that for some magic variation of ESM it’ll just count all lines as covered, regardless of whether there’s anything even on them (coverage for blank lines, yay).
+1, Vite is incredible
In one project we've been moving the tooling to Deno and it's remarkable how much easier it is to work with than the entire Nodejs ecosystem.
[deleted]
The title is misleading. It's 40x faster only on the second run not because of any hotpath code code speedup, but because it has an option to use a cache to remember state of previous files. The full check durations are on the same ballpark. This has nothing to do with deno.
Say prettier also added a cache to save the state of previous files, it would also be only 1s.
Say prettier also added a cache to save the state of previous files, it would also be only 1s.
Yes, the 40x faster is only for the runs after the first in this scenario. That said, if I disable the incremental cache locally (dprint check --incremental=false) and run this on the example code linked to in the article, on my machine the first run is 8.0s. If I run prettier's CLI it's 24.8s. dprint is faster because it formats files in parallel with Prettier and has Prettier snapshotted in a Deno runtime. There's some thread contention that's making it a little slower than it could be with v8 and I've been lazy to investigate it.
Also, dprint uses very fast globbing with two threads working together in order to collect files. I would guess Prettier would still be slower if it did this because it's programmed in JS (though not by much), but it doesn't do it.
Also, dprint uses very fast globbing with two threads working together in order to collect files. I would guess Prettier would still be slower if it did this because it's programmed in JS (though not by much), but it doesn't do it.
Adding caching to Prettier is reasonably easy, and most people will already have it through lint-staged or something similar.
I'm much more impressed by "dprint is 3* faster than Prettier for the same task" than "dprint is 40* faster than Prettier because dprint caches results and Prettier doesn't". That makes me want to look at dprint. The "40* faster" figure makes it look like the benchmark is wrong.
I'm much more impressed by "dprint is 3* faster than Prettier for the same task" than "dprint is 40* faster than Prettier because dprint caches results and Prettier doesn't". That makes me want to look at dprint. The "40* faster" figure makes it look like the benchmark is wrong.
The title of the article was and is "Speeding up Prettier locally and on your CI with dprint". If you use the Rust-based plugins instead of Prettier it will be much faster as noted at the end of the article.
To add another advantage of using dprint not mentioned, is that since it's pluggable you can use Prettier and other formatters all from the same formatting CLI. The main dprint repo does this in its dprint.json (https://github.com/dprint/dprint/blob/3d822a48133358ec4e2d5b...)
To add another advantage of using dprint not mentioned, is that since it's pluggable you can use Prettier and other formatters all from the same formatting CLI. The main dprint repo does this in its dprint.json (https://github.com/dprint/dprint/blob/3d822a48133358ec4e2d5b...)
That 40x faster title is an editorialised one from the submitter, not the articles title itself
[deleted]
Great to see dprint on the front page! While here it's speeding up prettier with caching/wasm, it can replace prettier altogether and be even faster (for example, with `dprint-plugin-typescript` for TS/JS dev). So, if you are doing web dev and not yet ready to switch to Deno, you can at least speed up formatting by using `dprint`.
David, I already owe you guys more beer than I can buy for all the great work on the Deno toolchain, but if we could get this issue[1] with HTML template tags going, it would be easier for me to convince the Lit crowd to chip in. Tanks again for the great work!
[1] https://github.com/dprint/dprint-plugin-typescript/issues/9
David, I already owe you guys more beer than I can buy for all the great work on the Deno toolchain, but if we could get this issue[1] with HTML template tags going, it would be easier for me to convince the Lit crowd to chip in. Tanks again for the great work!
[1] https://github.com/dprint/dprint-plugin-typescript/issues/9
It's definitely something I would like to implement. I haven't found a parser that would be good for formatting yet so maybe I will have to write my own :/
It is worth having Deno even if you are not using it: just as a Js/Ts LSP server (it's inbuit). Smaller and better than tss etc.
Also makes much more sense for local scripting. Give it a try even if in doubt for production.
Also makes much more sense for local scripting. Give it a try even if in doubt for production.
Can you please elaborate on Deno language server being better over tss? Does it have smaller latency, does it start faster? Is it a drop-in replacement or some features are missing?
It is a drop-in replacement. Deno has a Node compatibility mode [0] which — besides the execution — also applies to LSP, so the Node-specific idioms are processed correctly.
It is not cosmically faster than tsserver, but some — including myself — do prefer it. I wouldn't recommend to change anything if you are busy — but maybe take a glance when in the mood for checking out new things.
[0] https://deno.land/manual@main/node/compatibility_mode
It is not cosmically faster than tsserver, but some — including myself — do prefer it. I wouldn't recommend to change anything if you are busy — but maybe take a glance when in the mood for checking out new things.
[0] https://deno.land/manual@main/node/compatibility_mode
There's so much accidental complexity in the CI workflow purely to enable a tiny amount of caching. I hope the next generation of CI tools can improve this situation.
I know. I've been wondering if there's a way this complexity could be buried in the GitHub action for dprint (https://github.com/dprint/check) so people would only have to specify that and everything would just happen, but I'm not sure it's possible at the moment. Caching seems to be limited to needing to use actions/cache (https://github.com/actions/cache)
Deno is getting closer and closer to being able to compete with Node. A remaining performance issue was just partially fixed with this PR:
https://github.com/denoland/deno/pull/14319
Here's the issue: https://github.com/denoland/deno/issues/10157
https://github.com/denoland/deno/pull/14319
Here's the issue: https://github.com/denoland/deno/issues/10157
We run prettier through eslint, and run eslint through jest which provides parallelism. eslint also has a cache... So I'm not sure what dprint gets you over that?
https://github.com/jest-community/jest-runner-eslint
https://github.com/prettier/eslint-plugin-prettier
https://github.com/jest-community/jest-runner-eslint
https://github.com/prettier/eslint-plugin-prettier
Regardless of Deno, I've been leaning more and more towards dprint over Prettier recently given how fast and configurable it is.
I get Prettier's whole shtick about not being configurable so that your team can't fight over a style guide, but as a team that already had an internal style guide that predates Prettier, I just want a tool that can match what we have already decided on, which is where dprint has fit in nicely.
I get Prettier's whole shtick about not being configurable so that your team can't fight over a style guide, but as a team that already had an internal style guide that predates Prettier, I just want a tool that can match what we have already decided on, which is where dprint has fit in nicely.
Prettier itself would feel much faster if it supported caching like other code formatters (Python's Black, Rust's rust-fmt)
Prettier does support that, in the sense that you can achieve it using an official solution. https://prettier.io/docs/en/precommit.html
It's the Unix mentality of chaining things rather than building it in to the package.
It's the Unix mentality of chaining things rather than building it in to the package.
Yeah, precommit isn’t my favorite thing out of personal preference. So it’s a shame prettier doesn’t have built in caching like other tools in the space.
Serious question: Why not use source control to identify the files with changes and limit the scope of your formatter run to just those files. If the unchanged files met the formatter rules when they were committed, then repeated runs should be guaranteed to pass. Why introduce a cache when you could just do less work?
Because on CI there's no guarantee that the parent commit has already passed. It might be unchecked, in progress, or CI failed despite being a parent commit.
Now you could query the CI from the CI but that seems to be a dubious dependency to me, and user would need to set up an access token.
Now you could query the CI from the CI but that seems to be a dubious dependency to me, and user would need to set up an access token.
How does it take 41 seconds to format 500 source code files? _how_ is that an acceptable amount of time?
For a recent personal project, I wanted to format the generated HTML. My first stab at it used Prettier. But, I was very underwhelmed with the processing speed. I then gave js-beautify[1] a try, which I found to be much, much faster.
[1] https://github.com/beautify-web/js-beautify
[1] https://github.com/beautify-web/js-beautify
I'm wondering if every language is a plugin with dprint, or does it have some built in? I don't see a list of supported languages.
The work that David has been doing with DPrint has been outstanding. They architected a whole plugin system is based on WebAssembly and Wasmer. Super impresive
I wonder if anyone will take this work and make a visual studio code extension from it?
There is one here: https://github.com/dprint/dprint-vscode
Why would you run code formatting tools in a CI pipeline? If the code had been committed in the wrong format the problem should be addressed before the code goes into version control. Stuff like this helps with maintaining a tidy history and consistent code base. Sounds like self inflicted wounds to me.
The workflow is: You create a git branch, commit your changes to it and push them to the repo server. The CI then checks whether the code is formatted as expected. If the code is incorrectly formatted, the CI job fails and prints the format error(s). When the code is formatted properly (no CI jobs fail), you are allowed to merge into the master branch.
Reading your comment I assume you want the repo server to reject incorrectly formatted code at push time. Rejecting commits/pushes has the disadvantage that you cannot share your code/branch with others (or yourself on other devices), and cannot back up your code to the repo server.
You could argue formatting doesn’t take long. I’d argue, the repo server shouldn’t keep me from publishing code to a dev branch.
Reading your comment I assume you want the repo server to reject incorrectly formatted code at push time. Rejecting commits/pushes has the disadvantage that you cannot share your code/branch with others (or yourself on other devices), and cannot back up your code to the repo server.
You could argue formatting doesn’t take long. I’d argue, the repo server shouldn’t keep me from publishing code to a dev branch.
I review code. If the code is improperly formatted I know the person isn't using the right formatting tools and I tell them to fix that. In general I trust my team to actually use our code formatting tools and follow guidance.
How do you assure that the formatting as been done without running it in CI?
A lot of steps in CI aren't there to do something, they're merely there for analysis reasons and conditionally marking builds as failed
A lot of steps in CI aren't there to do something, they're merely there for analysis reasons and conditionally marking builds as failed
Running a formatter in CI and failing the build if it finds anything is a good way to make sure people are running the right tools locally, which is in turn a good way to maintain quality. I've seen many devs work with broken environments and just ignore the problems because it doesn't actually stop them, or because they don't realise it's broken.
https://news.ycombinator.com/item?id=31176453
> I've seen many devs work with broken environments and just ignore the problems because it doesn't actually stop them, or because they don't realise it's broken.
That sounds like an educational issue. I work in a small team in a small organization.
> I've seen many devs work with broken environments and just ignore the problems because it doesn't actually stop them, or because they don't realise it's broken.
That sounds like an educational issue. I work in a small team in a small organization.
Edit: I'm especially impressed with its dependency management. Import maps are a breeze and a welcome change to... whatever is going on in Nodeland. Like, I don't have to set up my imports both in tsconfig.json and package.json and walk this delicate tightrope to make sure that things don't just start breaking. With Deno's import maps it kind of just... works. I also love that I don't have to deal with a `node_modules` folder, and I can choose to vendor my dependencies if I'd like. It reminds me of the Go approach, which I really enjoy.
There's other things I like but these are the big ones. The more I can get switched over to Deno the better.