Show HN: Asdf Clone Written in Rust
36 comments
Nice! How long does `node -v` take now?
EDIT: OK, I see the main trick is to use PATH instead of shims.
> rtx does not use shims and instead updates PATH so that it doesn't have any overhead when simply calling binaries
There is a good reason `asdf-vm` uses shims, and is that it does not have to interplay or worry about other tools that set PATH and tools that need a reference to an executable could be simply set to `~/.asdf/shims`.
Ill take it for a spin, but this choice might have a lot of consequences that are not easy to foresee. A good example is `direnv` which as you mention in the README now requires to be set in `.envrc` and then disable global `rtx` hook I guess.
EDIT: OK, I see the main trick is to use PATH instead of shims.
> rtx does not use shims and instead updates PATH so that it doesn't have any overhead when simply calling binaries
There is a good reason `asdf-vm` uses shims, and is that it does not have to interplay or worry about other tools that set PATH and tools that need a reference to an executable could be simply set to `~/.asdf/shims`.
Ill take it for a spin, but this choice might have a lot of consequences that are not easy to foresee. A good example is `direnv` which as you mention in the README now requires to be set in `.envrc` and then disable global `rtx` hook I guess.
direnv does cause problems, but there is a workaround that functions (calling rtx within direnv). The main problem is it's annoying since you need .envrc and .tool-versions file littered through your apps (or just .envrc with "export RTX_NODEJS_VERSION=18")
There are some short term issues but I think I can resolve them in a way that direnv can't since I know exactly what PATH variables I added and didn't add. (Just remove everything with ~/.local/share/rtx prefixes.)
I could also just implement shims like asdf. The performance cost would be negligible (2-3ms of overhead for me running `rtx exec -- node -v`). I hate shims because they break `which node` though.
There are some short term issues but I think I can resolve them in a way that direnv can't since I know exactly what PATH variables I added and didn't add. (Just remove everything with ~/.local/share/rtx prefixes.)
I could also just implement shims like asdf. The performance cost would be negligible (2-3ms of overhead for me running `rtx exec -- node -v`). I hate shims because they break `which node` though.
The which issue is fair, but you could also consider either adding them to the shell as functions, for the wrapper effect but getting a path listing on where as part of it. Or making the rtx binary respond to argv[0] being different names such that the “wrappers” are just symlinks and save the extra exec. None of these is perfect, but they avoid the issue where path becomes too long to manage, and the function option can make them completely independent of PATH.
> adding them to the shell as functions
This, like shims, would require “reshimming” anytime a new binary is added by pip/npm. I really want to avoid that as I see juniors (and sometimes experienced devs) get in a bad state fairly often forgetting (or not understanding) reshimming.
I’m also not sure how you deal with subprocesses calling binaries. If they’re only shell functions they only exist to the shell.
> save the extra exec
rtx is already over 100x faster than asdf so this wouldn’t be necessary really. I could load test and optimize for setups with a ton of plugins and stuff if needed. My reason for avoiding shims is entirely UX related, not performance. Though I do think asdf would not have its problems if it didn’t use shims. This really is just because asdf is in bash and rtx is in rust. (I think rtx is already a lot more LOC though, and not as clean).
But in regards to PATH, I think I can get rtx to not screw up anything else that would be modifying PATH (slightly different algo than what’s currently used right now). I can’t guarantee that nothing will screw up rtx, but I can prevent direnv in particular from doing it.
It may be a game of whack-a-mole testing with different tools that modify PATH to get them to not conflict. That’s fine, it’s such a better solution.
I also think if rtx is loaded after other hooks it shouldn’t get messed with. It’s only if it gets called since potentially those tools could remove rtx paths and there isn’t much I could do about that.
This, like shims, would require “reshimming” anytime a new binary is added by pip/npm. I really want to avoid that as I see juniors (and sometimes experienced devs) get in a bad state fairly often forgetting (or not understanding) reshimming.
I’m also not sure how you deal with subprocesses calling binaries. If they’re only shell functions they only exist to the shell.
> save the extra exec
rtx is already over 100x faster than asdf so this wouldn’t be necessary really. I could load test and optimize for setups with a ton of plugins and stuff if needed. My reason for avoiding shims is entirely UX related, not performance. Though I do think asdf would not have its problems if it didn’t use shims. This really is just because asdf is in bash and rtx is in rust. (I think rtx is already a lot more LOC though, and not as clean).
But in regards to PATH, I think I can get rtx to not screw up anything else that would be modifying PATH (slightly different algo than what’s currently used right now). I can’t guarantee that nothing will screw up rtx, but I can prevent direnv in particular from doing it.
It may be a game of whack-a-mole testing with different tools that modify PATH to get them to not conflict. That’s fine, it’s such a better solution.
I also think if rtx is loaded after other hooks it shouldn’t get messed with. It’s only if it gets called since potentially those tools could remove rtx paths and there isn’t much I could do about that.
The reality is that shims also cause many problems... So, i guess, choose your poison?
I have been using asdf for over a year now, and I haven't seen `node -v` take so long. Actually, I used to use nvm and it was much slower than asdf, which was my main motivation to switch.
I'm wondering if the lack of slowness I've seen could be due to my simplistic setup: globally set nodejs version, single nodejs version installed, with other tools like git installed by the distro package manager. I also have golang and some other stuff like rclone installed using asdf, but not much more.
Did you also have a similar experience when initially installing node? Did you notice what triggered it or when did asdf start to become slow?
I like asdf and would like to avoid making it slow on my laptop.
Oh and can I safely assume that your home directory is on an SSD and not on an HDD?
I'm wondering if the lack of slowness I've seen could be due to my simplistic setup: globally set nodejs version, single nodejs version installed, with other tools like git installed by the distro package manager. I also have golang and some other stuff like rclone installed using asdf, but not much more.
Did you also have a similar experience when initially installing node? Did you notice what triggered it or when did asdf start to become slow?
I like asdf and would like to avoid making it slow on my laptop.
Oh and can I safely assume that your home directory is on an SSD and not on an HDD?
I'm on an M1 MacBook Pro 16". The performance issues are well understood by the asdf team and the Github issue with most comments is on the topic: https://github.com/asdf-vm/asdf/issues/290
This was with a half dozen plugins and use_legacy_file disabled. Hardly an edge-case. Though I'll note in other tests it was around 200ms (which is the number I put in the README to be fair). @danfritz in this thread said his can take over 2 seconds!
Out of curiosity, what is the difference between `time node -v` and `time ~/.asdf/installs/nodejs/*/bin/node -v`? I've never seen it not add at least 100ms.
This was with a half dozen plugins and use_legacy_file disabled. Hardly an edge-case. Though I'll note in other tests it was around 200ms (which is the number I put in the README to be fair). @danfritz in this thread said his can take over 2 seconds!
Out of curiosity, what is the difference between `time node -v` and `time ~/.asdf/installs/nodejs/*/bin/node -v`? I've never seen it not add at least 100ms.
Apologies for the delay. Now I do notice a delay when using the asdf shim instead of running the binary directly. But it's nowhere near 900ms.
I'm on a T480s with i5 CPU and SSD:
I'm on a T480s with i5 CPU and SSD:
$ time node --version
v18.13.0
real 0m0.080s
user 0m0.079s
sys 0m0.027s
$ time ~/.asdf/installs/nodejs/lts/bin/node --version
v18.13.0
real 0m0.009s
user 0m0.006s
sys 0m0.003sFor me, on a ThinkPad 14s with an SSD on Linux:
$ time node -v
v18.13.0
node -v 0.05s user 0.02s system 94% cpu 0.073 total
$ time ~/.asdf/installs/nodejs/18.13.0/bin/node -v
v18.13.0
~/.asdf/installs/nodejs/18.13.0/bin/node -v 0.00s user 0.00s system 94% cpu 0.004 total
[deleted]
This is awesome. Have you seen http://tea.xyz? Made by the creator of brew. I think it has a lot of interesting ideas, especially since it also has some of the same features as asdf/venv/rbenv/etc: managing multiple versions.
Figured I'd mention it, because I haven't tried tea yet, but I'm an avid asdf and brew user and I've been looking at asdf alternatives.
Figured I'd mention it, because I haven't tried tea yet, but I'm an avid asdf and brew user and I've been looking at asdf alternatives.
I haven't, thanks for sending. It's interesting that it hooks into the "not found" hook of the shell so you can run things like `node^18 --version`.
I also really like that it has really clean install output.
I also really like that it has really clean install output.
It looks interesting. I had been avoiding it because the last time I read about it the creator was going on about blockchains and the current maintainer of brew weighed in saying that brew is totally unassociated with that person so take any comparison with a grain of salt. I'll probably avoid tea until it gets a bit more stable though (if ever). These things take time, homebrew took quite awhile to be really great.
Automatically installing whatever missing thing that you type as a command sounds like a terrible idea. Am I missing something?
Obligatory "Written in Rust " after everything written in rust is still funny to me.
Well, I came here wondering why a Common Lisp library (module)[1] had been rewritten in Rust. Whoops.
[1]: https://asdf.common-lisp.dev
[1]: https://asdf.common-lisp.dev
I knew you wouldn’t be the only one so I made sure that “rtx” was not a popular lisp library. :)
Ohh, this interesting. I love most things about asdf, but not the fact that it's great big heap of bash scripts. I wonder how hard it would be to get this working on (non-WSL) windows and how a truly universal version manager.
that would be unrealistic at least for now. The plugins are all written in bash. You'd first need to rewrite enough plugins for it to be useful. Maybe at one point if rtx takes off I could look into having plugins be in an alternate form though.
Great to see someone is trying to provide a better alternative to asdf!
> aliases and fuzzy-matching
What exactly does fuzzy matching mean? The only examples I see in your README are of the nature
In any case, I hope that, if I try to install a package/plugin and it doesn't exist, it won't just try to install a plugin/package whose name is similar? :nervous_look.jpg:
> aliases and fuzzy-matching
What exactly does fuzzy matching mean? The only examples I see in your README are of the nature
rtx install [email protected] Install a specific version number
rtx install [email protected] Install a fuzzy version number
In any case, I hope that, if I try to install a package/plugin and it doesn't exist, it won't just try to install a plugin/package whose name is similar? :nervous_look.jpg:
What you described is fuzzy matching, basically just prefix matching.
asdf has a handful of commands that let you do prefix matching like `asdf install nodejs latest:20.0` but they're not supported in `.tool-versions` files. This basically just adds that functionality.
I didn't feel like requiring `latest:` was necessary. I felt it was clear that `nodejs 20.0` means `nodejs 20.0.*`. This is how npm has always worked.
asdf has a handful of commands that let you do prefix matching like `asdf install nodejs latest:20.0` but they're not supported in `.tool-versions` files. This basically just adds that functionality.
I didn't feel like requiring `latest:` was necessary. I felt it was clear that `nodejs 20.0` means `nodejs 20.0.*`. This is how npm has always worked.
Will definitely check this out! I'm a long time asdf user but I always struggle with the shims and long timeouts.
I occasionally hit 2sec until my node version is resolved.
I occasionally hit 2sec until my node version is resolved.
As mentioned in my other comment already, it's great to see someone is trying to provide a better alternative to asdf!
From what I can tell by looking at the documentation of `rtx install`,
I take it there is still no way in rtx to simply install all plugins and runtimes listed in .tool-versions, without listing them one by one on the command line? Put differently, I would expect `rtx install` to install all runtimes specified in .tool-versions, including any plugins that are not installed yet.
Would this be something that you would consider adding? I'm asking because not being able to install all plugins mentioned in .tool-versions automatically has been my biggest gripe with asdf.
From what I can tell by looking at the documentation of `rtx install`,
rtx install # installs all runtimes specified in .tool-versions for installed plugins
install a plugin
note that rtx automatically can install plugins when you install a runtimeI take it there is still no way in rtx to simply install all plugins and runtimes listed in .tool-versions, without listing them one by one on the command line? Put differently, I would expect `rtx install` to install all runtimes specified in .tool-versions, including any plugins that are not installed yet.
Would this be something that you would consider adding? I'm asking because not being able to install all plugins mentioned in .tool-versions automatically has been my biggest gripe with asdf.
I actually had this as the default behavior of `rtx install` but it felt a bit wild for it to just go out and download a bunch of stuff. I feel like it’s nice to have a bit more control over which runtimes are and are not managed by rtx.
That said, I was thinking about this yesterday and that it’s currently painful to have to type out each plugin one by one. I added this ticket for making `plugins install` default to installing all plugins in the local tool-versions: https://github.com/jdxcode/rtx/issues/50
What do you think? Is that discoverable and useful enough? I could go a step further and add a `--plugins` argument to install or something.
That said, I was thinking about this yesterday and that it’s currently painful to have to type out each plugin one by one. I added this ticket for making `plugins install` default to installing all plugins in the local tool-versions: https://github.com/jdxcode/rtx/issues/50
What do you think? Is that discoverable and useful enough? I could go a step further and add a `--plugins` argument to install or something.
I just added `rtx install --all` which installs all plugins and all runtimes. It's in v1.3.0
I actually view being written in shell as an advantage for asdf. That means I can just submodule it into my dotfiles and bootstrap my tools without extra steps, no matter where I end up.
Glad to see you fixed what I think is a bad UX decision by asdf, using the subcommand `add` for plugins and `install` for runtime versions.
They nailed the back-end stuff. I would not have come up with such a good, simple plugin design. But yeah, there are several UX issues like this. Part of it is because it’s written in bash, but ones like this I really couldn’t explain.
I’m trying to fix them while also supporting their syntax where I can, muscle memory is strong. So I support “plugins install” and “plugins add” If you notice any aliases that you think should exist, please mention them.
I’m trying to fix them while also supporting their syntax where I can, muscle memory is strong. So I support “plugins install” and “plugins add” If you notice any aliases that you think should exist, please mention them.
Nice! I tried something similar at github.com/happenslol/qwer, I'll look at yours to see what you did differently :-)
This looks great, will be working with this in the next few days. I've always had a bad relationship with asdf as it seems to break existing tool management in creative ways that I'm unable to untangle without uninstalling it completely (long story, much thrashing of keyboard and mouse).
> ... asdf as it seems to break existing tool management in creative ways
Do note that asdf re-uses existing tooling, so trying to combine ruby-build and node version manager and asdf with node and ruby plug-ins is likely to crash and burn (because you end up forcing existing tooling to fight with itself).
Not sure if this was what caused your problems - but my asdf-life got a lot more pleasant when just asdf all the things (that I use).
Does mean that things like vs code needs the right environment / local tool versions and language servers to work with your projects and vs code plug-ins.
Do note that asdf re-uses existing tooling, so trying to combine ruby-build and node version manager and asdf with node and ruby plug-ins is likely to crash and burn (because you end up forcing existing tooling to fight with itself).
Not sure if this was what caused your problems - but my asdf-life got a lot more pleasant when just asdf all the things (that I use).
Does mean that things like vs code needs the right environment / local tool versions and language servers to work with your projects and vs code plug-ins.
The alternative to asdf (at least for me) is to just quit programming. It does what it is supposed to do well enough because in years I have not found anything better or necessary to replace it with.
I like the idea, maybe I'll try it soon. I've recently been using asdf-direnv, configured to switch environments once, manually, and so taking shims out of the $PATH. Definitely giving up a bit of the magic for speed, so not sure if I love the tradeoff.
my issue with that solution is it makes directory changes with `use asdf` in the envrc slow. Of course that's not as bad as making all of the runtimes slow, but still intolerable IMO.
However it is so slow. I was just testing `node -v` and it was taking ~900ms. That kind of overhead is completely unusable. My shell prompt uses runtimes inside of it for various things so this effectively makes every command take multiple seconds to complete.
So I rebuilt it in Rust but using the same plugin ecosystem so it should be a drop-in replacement. I also added a couple of features that I wanted from asdf (aliases and fuzzy-matching).
Let me know what you think! Just know that people have only been using this for a few days so if you see any bugs, they're likely not big hairy issues, just overlooked edge-cases and will be fixed soon.
https://github.com/jdxcode/rtx