NixOS 21.05(nixos.org)
nixos.org
NixOS 21.05
https://nixos.org/manual/nixos/stable/release-notes.html#sec-release-21.05
233 comments
> and a few servers
The problem I see with NixOS on a typical personal server is that you have to setup all these things using nix expressions, from which the actual configuration files are generated.
That means if you e.g. want to install postfix, instead of learning about main.cf you have to learn the syntax of the nix configuration wrapper for postfix, and postfix having hundreds if not thousands of options, many referring to external files/databases, you have to hope that whoever did this configuration wrapper, supported all the features of postfix that you want to use.
I am using nixpkgs on macOS, and I run a personal server with mail (for a bunch of people), web (for a bunch of sites), DNS (for a bunch of domains), etc. and while I would love to switch to declarative configuration, I fear I would run into shortcomings with the configuration wrappers for the various software packages I use.
I am curious if anyone have experience to share?
The problem I see with NixOS on a typical personal server is that you have to setup all these things using nix expressions, from which the actual configuration files are generated.
That means if you e.g. want to install postfix, instead of learning about main.cf you have to learn the syntax of the nix configuration wrapper for postfix, and postfix having hundreds if not thousands of options, many referring to external files/databases, you have to hope that whoever did this configuration wrapper, supported all the features of postfix that you want to use.
I am using nixpkgs on macOS, and I run a personal server with mail (for a bunch of people), web (for a bunch of sites), DNS (for a bunch of domains), etc. and while I would love to switch to declarative configuration, I fear I would run into shortcomings with the configuration wrappers for the various software packages I use.
I am curious if anyone have experience to share?
I run several personal NixOS servers. it's definitely doable without custom expressions (I can't remember the last time I had to write one).
lots of services defined in the NixOS options will have an "escape hatch" option named something like `extraConfig`. this usually lets you append verbose text to a complex config file, like in the case of Postfix where the options couldn't conceivably cover everything.
you can also always just write a static file to /etc if you want or need to:
another useful escape hatch is that you can run Docker/Podman containers declaratively. I have some things I want the flexibility to upgrade on a different cadence than NixOS itself (such as my family's "production" Jellyfin server) that I just run as standalone Docker containers with volume mounts for their data.
lots of services defined in the NixOS options will have an "escape hatch" option named something like `extraConfig`. this usually lets you append verbose text to a complex config file, like in the case of Postfix where the options couldn't conceivably cover everything.
you can also always just write a static file to /etc if you want or need to:
environment.etc.foo-config = {
target = "foo.conf";
text = ''
enable_spline_reticulator = true
'';
};
systemd.services.foo = {
after = [ "network-online.target" ];
wantedBy = [ "multi-user.target" ];
serviceConfig = {
ExecStart = ''
${pkgs.foo}/bin/food \
--config /etc/foo.conf
'';
User = "foo";
Restart = "on-failure";
};
};
another useful escape hatch is that you can run Docker/Podman containers declaratively. I have some things I want the flexibility to upgrade on a different cadence than NixOS itself (such as my family's "production" Jellyfin server) that I just run as standalone Docker containers with volume mounts for their data.
Your best friend is https://search.nixos.org/options
From a quick search, it seems you can use `services.postfix.config` and pass key-value pairs as arbitrary options. Not familiar with this service, but usually if a given one has more complex syntax, there is a “pass-through” option that will simply append to the given config file whatever text you provide, as well as as the sister reply says you can create manually additional files.
From a quick search, it seems you can use `services.postfix.config` and pass key-value pairs as arbitrary options. Not familiar with this service, but usually if a given one has more complex syntax, there is a “pass-through” option that will simply append to the given config file whatever text you provide, as well as as the sister reply says you can create manually additional files.
The thing with a lot of server software (like postfix) is that configuration is usually spread across many files.
And then there is the re-use of things between services, for example DKIM’s generated public key should be made available as a TXT record by the DNS server, the SSL certificate kept up-to-date by the web server should be used by both the smtp and imap servers, though it may need to include the full chain, and services may need to be relaunched, if the certificate is updated, etc.
I was hoping to hear from someone who had managed to get a “full” server running (with SMTP, IMAP, DKIM, DNS, DNSSec, HTTPS via ACME, etc.), because while I know that I can output raw configuration files, it seems like an extremely daunting task to weave all this together in something semantically meaningful.
Right now I have /etc under git control and a Makefile that handles all dependencies between the various pieces (i.e. to ensure proper files are regenerated/indexed as needed, and services relaunched when dependencies are updated).
And then there is the re-use of things between services, for example DKIM’s generated public key should be made available as a TXT record by the DNS server, the SSL certificate kept up-to-date by the web server should be used by both the smtp and imap servers, though it may need to include the full chain, and services may need to be relaunched, if the certificate is updated, etc.
I was hoping to hear from someone who had managed to get a “full” server running (with SMTP, IMAP, DKIM, DNS, DNSSec, HTTPS via ACME, etc.), because while I know that I can output raw configuration files, it seems like an extremely daunting task to weave all this together in something semantically meaningful.
Right now I have /etc under git control and a Makefile that handles all dependencies between the various pieces (i.e. to ensure proper files are regenerated/indexed as needed, and services relaunched when dependencies are updated).
I do have all these things (https://github.com/Mic92/dotfiles/blob/master/nixos/eve/modu..., https://github.com/Mic92/dotfiles/blob/master/nixos/eve/modu..., https://github.com/Mic92/dotfiles/blob/master/nixos/eve/modu... ...). Especially things like ACME work a lot better the NixOS-provided curated and unit-tested acme module compared to setting up it yourself. You can also use the Nixos simple mail server setup that already provides sane defaults and puts the individual nixos modules together: https://gitlab.com/simple-nixos-mailserver/nixos-mailserver/
Appreciate the links, thanks!
Even better: `man configuration.nix` and `man home-configuration.nix`. I used the website first, but found out just using the quite good man pages is much much faster.
NixOS makes it easy to emit raw configuration files if you want. I do this for software where I want more control than the Nix wrapper exposes.
> I am curious if anyone have experience to share?
With one exception (nginx, because it's very well supported) I always just use raw config files and ignore the options. For TVL[0] we've written a bunch of NixOS modules[1] ourselves where the upstream one was either not flexible enough, or deviated strongly from how we wanted things to work.
Nix is the kind of tool that lends itself well to solutions that are more complicated than the problem, but once you get comfortable with the tool it's easy to sidestep that. Despite these warts it's 100% worth the investment.
[0]: https://code.tvl.fyi/about/
[1]: https://cs.tvl.fyi/depot/-/tree/ops/modules
With one exception (nginx, because it's very well supported) I always just use raw config files and ignore the options. For TVL[0] we've written a bunch of NixOS modules[1] ourselves where the upstream one was either not flexible enough, or deviated strongly from how we wanted things to work.
Nix is the kind of tool that lends itself well to solutions that are more complicated than the problem, but once you get comfortable with the tool it's easy to sidestep that. Despite these warts it's 100% worth the investment.
[0]: https://code.tvl.fyi/about/
[1]: https://cs.tvl.fyi/depot/-/tree/ops/modules
Maybe you could propose the improvements in nixpkgs so we can all benefit from them?
In some cases the upstream modules are simply targeting a different use-case. A lot of modules are written by someone scratching a specific itch - ours are no different, but scratch different itches.
I think a better solution (long-term) is actually a redesigned way of doing service declarations where we're dealing with composable bits of data that describe services (so that a module isn't all-or-nothing anymore). Too much on the plate to design that though ...
I think a better solution (long-term) is actually a redesigned way of doing service declarations where we're dealing with composable bits of data that describe services (so that a module isn't all-or-nothing anymore). Too much on the plate to design that though ...
Honestly - how painful was it to learn and get working? And what limitations/niggles are left when using as a workstation?
I love the principles behind Nix, and I like to use it to provide development environments (through nix-shell locally and then using the same setup in CI). But some things can be moderately painful to get going.
I love the principles behind Nix, and I like to use it to provide development environments (through nix-shell locally and then using the same setup in CI). But some things can be moderately painful to get going.
You're looking at a solid 10-20 hours of hacking stuff together to get a decent working system, and you still won't understand a good 80% of what you copied from various repos.
I've been using NixOS for a year now, and I have a definite love/hate relationship. Many times I miss the simplicity of Arch, where I understood _everything_ about my system and how it was configured.
NixOS is like learning everything all over again, but in a weird, poorly documented language+standard lib that nothing else uses.
I think when you are super comfortable with it, is when it becomes truly amazing. But that takes a lot of dedicated time and effort.
I've been using NixOS for a year now, and I have a definite love/hate relationship. Many times I miss the simplicity of Arch, where I understood _everything_ about my system and how it was configured.
NixOS is like learning everything all over again, but in a weird, poorly documented language+standard lib that nothing else uses.
I think when you are super comfortable with it, is when it becomes truly amazing. But that takes a lot of dedicated time and effort.
Yes, it's extremely powerful once you truly understand Nix.
There are efforts to improve documentation, but it still is lacking (I think the biggest problem is that Nix is so big, not just the OS but it can be utilized as a build system).
Just with NixOS is not exactly clear how can you for example build your custom image.
I think https://nix.dev/ is approaching the documentation from the right direction.
There are also many pieces that people built that you need to find.
For example some things that I found accidentally:
* https://github.com/nix-community/poetry2nix
* https://github.com/matthewbauer/nix-bundle
* https://github.com/cleverca22/not-os
Unfortunately those side projects often have even worse documentation.
There are efforts to improve documentation, but it still is lacking (I think the biggest problem is that Nix is so big, not just the OS but it can be utilized as a build system).
Just with NixOS is not exactly clear how can you for example build your custom image.
I think https://nix.dev/ is approaching the documentation from the right direction.
There are also many pieces that people built that you need to find.
For example some things that I found accidentally:
* https://github.com/nix-community/poetry2nix
* https://github.com/matthewbauer/nix-bundle
* https://github.com/cleverca22/not-os
Unfortunately those side projects often have even worse documentation.
If you want to build custom images just use https://github.com/nix-community/nixos-generators
It's documentation is on point.
Would it be possible to add a minimalistic image that contains app, kernel and essential packages? Like what you would get when building a docker container or using the not-os repo that I mentioned earlier.
I think having mini images would be awesome.
I think having mini images would be awesome.
You might like: https://nixery.dev/
e.g. if you want a Nix container with a shell environment, git, and htop:
e.g. if you want a Nix container with a shell environment, git, and htop:
docker run -ti nixery.dev/shell/git/htop bashThis is where I'm sitting right now. I'm trying out Void Linux on my laptop and it's just so simple. I love that I understand my system. I look at Nix as a language and I just have no idea what is going on.
What I'd really like, as a learning resource, is a "Nix for apt/yum users" or a "Nix for Ansible" users guide. There is the cheat sheet (https://nixos.wiki/wiki/Cheatsheet) but it's just not enough.
For example, I wanted to just... create an arbitrary directory in my home directory (~/workspace). Does Home Manager do this? If so, how? Do I just have a shell script that is called within a package? That seems excessive. Is this not a use case for Nix? It looks like it can do arbitrary tasks across the system but the documentation is so bogged down with theory that it takes forever to find practical applications.
What I'd really like, as a learning resource, is a "Nix for apt/yum users" or a "Nix for Ansible" users guide. There is the cheat sheet (https://nixos.wiki/wiki/Cheatsheet) but it's just not enough.
For example, I wanted to just... create an arbitrary directory in my home directory (~/workspace). Does Home Manager do this? If so, how? Do I just have a shell script that is called within a package? That seems excessive. Is this not a use case for Nix? It looks like it can do arbitrary tasks across the system but the documentation is so bogged down with theory that it takes forever to find practical applications.
My advice would be (per the other person here) to ignore home-manager when you're getting started, and then, when you're ready, treat it as a dotfiles manager on steroids.
I've never used it to manage visible directories in my ~/, but for ~/.gitconfig? Absolutely :).
I've never used it to manage visible directories in my ~/, but for ~/.gitconfig? Absolutely :).
> For example, I wanted to just... create an arbitrary directory in my home directory (~/workspace). Does Home Manager do this? If so, how? Do I just have a shell script that is called within a package? That seems excessive. Is this not a use case for Nix? It looks like it can do arbitrary tasks across the system but the documentation is so bogged down with theory that it takes forever to find practical applications.
You are definitively trying to do too much at the same time. Before using Home-Manager (that is a community project, it is not endorsed by NixOS except by allowing to be hosted on the nix-community group on GitHub), I stayed at two years playing with only NixOS. It is only after I got comfortable with Nix and NixOS that I decided to migrate my configuration to Home-Manager. Not saying that you need that much time to use HM, just that HM assumes that you're know a reasonable good amount about Nix and NixOS.
So I would say, start with less: try NixOS first, it is kinda easy to use compared to other "advanced" Linux distros like Arch and Void. The easy things are really easy, the hard things can be really hard at first but I don't think you need to know much about Nix to be proeficient on NixOS nowadays. Also yeah, you will need to search some things, but since the scope is reduced it is easier to find answers.
You are definitively trying to do too much at the same time. Before using Home-Manager (that is a community project, it is not endorsed by NixOS except by allowing to be hosted on the nix-community group on GitHub), I stayed at two years playing with only NixOS. It is only after I got comfortable with Nix and NixOS that I decided to migrate my configuration to Home-Manager. Not saying that you need that much time to use HM, just that HM assumes that you're know a reasonable good amount about Nix and NixOS.
So I would say, start with less: try NixOS first, it is kinda easy to use compared to other "advanced" Linux distros like Arch and Void. The easy things are really easy, the hard things can be really hard at first but I don't think you need to know much about Nix to be proeficient on NixOS nowadays. Also yeah, you will need to search some things, but since the scope is reduced it is easier to find answers.
> how painful was it to learn and get working?
nixpkgs (which you can use on macOS or any Linux distro), quite little, you can get up to speed in 15min as a Homebrew replacement:
NixOS, it's barely different, only generalised to the whole OS, but it's a bit tougher, because there's this abstraction via configuration that generates e.g systems files or other configurations that you'd operate with directly otherwise on a traditional distro. But the minute you screw things up, having the generation selector to pick and boot on straight in the boot loader you can see the absolutely unparalleled value of the proposition.
nixpkgs (which you can use on macOS or any Linux distro), quite little, you can get up to speed in 15min as a Homebrew replacement:
nix-env -qas ruby # 'q'uery 'a'vailable 'search'
nix-env -I ruby. # install by package name (not recommended)
nix-env -iA nixpkgs.ruby_3_0 # install by "attribute", recommended
nix-env -q # 'q'uery (i.e list installed)
nix-env -e ruby # uninstall (like deleting a git ref)
nix-env -q
nix-collect-garbage # cleanup, like `git gc`
# etc...
$ cat shell.nix
{
pkgs ? import <nixpkgs> {}, # you can have multiple of these with different names to have e.g a mix of stable and unstable
}:
let
openssl = pkgs.openssl; # just an example of how to set a var
in pkgs.mkShell {
buildInputs = [
openssl
pkgs.ruby_3_0 # or reference attires directly
];
}
$ nix-shell
(nix) $ ruby --version
ruby 3.0.1p64 (2021-04-05) [x86_64-darwin17]
(nix) $ openssl version
OpenSSL 1.1.1k 25 Mar 2021
(nix) $ exit
$ openssl version # back to the system one
LibreSSL 2.8.3
NixOS, it's barely different, only generalised to the whole OS, but it's a bit tougher, because there's this abstraction via configuration that generates e.g systems files or other configurations that you'd operate with directly otherwise on a traditional distro. But the minute you screw things up, having the generation selector to pick and boot on straight in the boot loader you can see the absolutely unparalleled value of the proposition.
Please don't recommend that people use nix-env. It's one of Nix's biggest footguns and a huge support burden for the maintainers.
We've been actively trying to remove mentions of it from the documentation.
If you want something "installed" use home-manager. If you just want something for quick dev use nix-shell.
We've been actively trying to remove mentions of it from the documentation.
If you want something "installed" use home-manager. If you just want something for quick dev use nix-shell.
I'm so glad this is the official line now. People giving examples using nix-env -I is, IMO, one of the major reasons it's so hard to get up and running.
It's like... here are all the amazing reasons to do declarative config... and here's how you do everything using nix-env -I. Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!
It's like... here are all the amazing reasons to do declarative config... and here's how you do everything using nix-env -I. Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!
> Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!
Guix has a command that converts an imperative profile to a declarative one. Check out the `--export-manifest` option for `guix package`: https://guix.gnu.org/manual/en/guix.html#Invoking-guix-packa...
Adding this to nix-env's replacement would be a sound approach imo. Maybe we could have one that also emits a little message for each package which is used in a NixOS module, directing users to a `programs.whatever` option instead of just dumping it in their `environment.systemPackages` list
Guix has a command that converts an imperative profile to a declarative one. Check out the `--export-manifest` option for `guix package`: https://guix.gnu.org/manual/en/guix.html#Invoking-guix-packa...
Adding this to nix-env's replacement would be a sound approach imo. Maybe we could have one that also emits a little message for each package which is used in a NixOS module, directing users to a `programs.whatever` option instead of just dumping it in their `environment.systemPackages` list
[deleted]
If nix-env is a big ol' footgun, are there any plans to make home-manager a part of the OS itself, instead of just a community project?
My hope is that when Sander van der Burg finishes drafting his RFC for mainlining his process management framework [1], which abstracts over ways to manage local services from systemd to supervisord to Docker, we can actually unify the module collection in Nixpkgs with its clones in nix-darwin and home-manager, and offer a more complete Nix experience as a configuration/service manager rather than just a package manager on non-NixOS.
Fwiw, you can already have nixos-rebuild read the home-manager configs for all of your users and deploy them as part of the normal NixOS config update process using the included NixOS module.
——
1: https://github.com/svanderburg/nix-processmgmt
Fwiw, you can already have nixos-rebuild read the home-manager configs for all of your users and deploy them as part of the normal NixOS config update process using the included NixOS module.
——
1: https://github.com/svanderburg/nix-processmgmt
Random nix usage question... occasionally I run into something like this, where the "install" directions are to clone a git repo and then install in an imperative way. What's the correct way to do this declaritively, e.g. in a home-manager or NixOS config? And why are directions like this in the imperative form? I'd think that people working this closely to core Nix would be pushing declarative setup so there must be something I'm missing.
Another example is nixops 2.0.
Another example is nixops 2.0.
I think this is one of the things that flakes aims to solve. I'm playing around with a new flake-based setup and I'm using flake-utils-plus[1], and I use the overlays builder to make it easy to refer to packages that come from flake inputs in my configuration.nix and similar places, like. I just added the latest Nixops from master to make sure I was giving you a real answer, and I use an overlay like this in my flake.nix:
What you wanna do for projects that don't offer a flake.nix or a ready-made overlay is to use a fetcher like in the Emacs example, but write your own overlay function that invokes Nixpkgs' `callPackage` function on whatever Nix expression inside the repo represents the package you want, or imports them from a release or default.nix as appropriate. Home-manager uses this to define its own little overlay[3], the one that gets used in its flake.nix, and its package is just its default.nix[4].
Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope. For example, this works for nix-processmgmt:
I think maybe the reason instructions aren't given for some projects is that they see those parts of the guide as mostly for beginners or casual experimenters, and they expect advanced or ‘serious’ users to be able to figure it out without much trouble. To some extent I think this is because different people choose to pin packages in the pre-flakes world through a variety of different mechanisms, and the authors of these packages and tools don't know in advance how you want to do it.
In the case of nix-processmgmt, I think Sander doesn't actually expect to have any users! In such cases, an imperative install that users are expected to play with for a little while and then just throw away is supposed to be enough.
You're right, though, that it's odd and disappointing that instructions for the preferred way of doing things are sometimes simply not given. A polite pull request or issue report would probably be well-received. My recommended strategy, if you get stuck, would be to ask for help getting those packages into scope declaratively on Discourse or Matrix, and then to offer the authors of these out-of-tree packages pull requests to modify their READMEs accordingly. :)
PS: You don't necessarily _have_ to use overlays for these. You can also drop expressions that use builtins.fetchTarball and then use `callPackage` or import from those sources directly into lists of packages like `environment.systemPackages`.
—
1: https://github.com/gytis-ivaskevicius/flake-utils-plus
2: https://github.com/nix-community/emacs-overlay#quickstart
3: https://github.com/nix-community/home-manager/blob/master/ov...
4: https://github.com/nix-community/home-manager/blob/master/de...
(final: prev: {
nixops = inputs.nixops.defaultPackage.${prev.system};
})
nixops.url = "github:NixOS/nixops";
nixops --version
NixOps @version@
What you wanna do for projects that don't offer a flake.nix or a ready-made overlay is to use a fetcher like in the Emacs example, but write your own overlay function that invokes Nixpkgs' `callPackage` function on whatever Nix expression inside the repo represents the package you want, or imports them from a release or default.nix as appropriate. Home-manager uses this to define its own little overlay[3], the one that gets used in its flake.nix, and its package is just its default.nix[4].
Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope. For example, this works for nix-processmgmt:
(final: prev: {
nix-processmgmt-tools = (import (
(builtins.fetchTarball "https://github.com/svanderburg/nix-processmgmt/archive/6def8584c6b028c922c550859a07b989d21d6f73.tar.gz")
+ "/tools/default.nix")
{ pkgs = prev.pkgs; }
);
})
I think maybe the reason instructions aren't given for some projects is that they see those parts of the guide as mostly for beginners or casual experimenters, and they expect advanced or ‘serious’ users to be able to figure it out without much trouble. To some extent I think this is because different people choose to pin packages in the pre-flakes world through a variety of different mechanisms, and the authors of these packages and tools don't know in advance how you want to do it.
In the case of nix-processmgmt, I think Sander doesn't actually expect to have any users! In such cases, an imperative install that users are expected to play with for a little while and then just throw away is supposed to be enough.
You're right, though, that it's odd and disappointing that instructions for the preferred way of doing things are sometimes simply not given. A polite pull request or issue report would probably be well-received. My recommended strategy, if you get stuck, would be to ask for help getting those packages into scope declaratively on Discourse or Matrix, and then to offer the authors of these out-of-tree packages pull requests to modify their READMEs accordingly. :)
PS: You don't necessarily _have_ to use overlays for these. You can also drop expressions that use builtins.fetchTarball and then use `callPackage` or import from those sources directly into lists of packages like `environment.systemPackages`.
—
1: https://github.com/gytis-ivaskevicius/flake-utils-plus
2: https://github.com/nix-community/emacs-overlay#quickstart
3: https://github.com/nix-community/home-manager/blob/master/ov...
4: https://github.com/nix-community/home-manager/blob/master/de...
Wow, a lot to digest here, thanks for the super detailed response! Definitely going to be coming back to this over the next week or two and trying some things out.
> Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope
This has been my approach, I always kind of assumed that there must be a better way but at least I feel a little better knowing that there isn't really. I haven't spent much time with flakes because I mostly only use Nix as a package manager/operating system but this helped me see how even if I'm not personally writing packages that I want to re-use I can still get a lot of value from flakes, so getting that setup is probably my next step!
> Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope
This has been my approach, I always kind of assumed that there must be a better way but at least I feel a little better knowing that there isn't really. I haven't spent much time with flakes because I mostly only use Nix as a package manager/operating system but this helped me see how even if I'm not personally writing packages that I want to re-use I can still get a lot of value from flakes, so getting that setup is probably my next step!
I don't think removing it is a good idea. Nix is a paradigm shift and things are done completely different, nix-env though is still one piece that might be somewhat similar to what people are used to.
I like to use nix-env (or more often, now, `nix profile`) for a persistence level in between nix-shell and really adding something to my home-manager or NixOS config. I let my profile build up 10-20 things installed, then every few weeks I decide what belongs in my declared config and uninstall everything in the profile.
Imo entirely removing imperative package management would be a mistake, although imperatively managing a file that gets sourced in your declarative config (a bit like /var/dpkg/selections on ol' Debian) instead of putting the whole profile manifest in the Nix store and leaving it at that would be better.
Imo entirely removing imperative package management would be a mistake, although imperatively managing a file that gets sourced in your declarative config (a bit like /var/dpkg/selections on ol' Debian) instead of putting the whole profile manifest in the Nix store and leaving it at that would be better.
> I let my profile build up 10-20 things installed, then every few weeks I decide what belongs in my declared config and uninstall everything in the profile.
I'm curious what you get out of this that you don't get from just adding/removing packages in your home-manager config? Is it just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch? Or is there some other benefit?
I'm curious what you get out of this that you don't get from just adding/removing packages in your home-manager config? Is it just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch? Or is there some other benefit?
> Is [the benefit of this approach] just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch?
That's definitely a factor. I think since I keep my Nix configurations in source control, somehow modifying the configuration feels more ‘official’, and it also usually comes with extra steps like committing and pushing.
The other thing I like is that it makes it very easy to tell if I actually want/need something: if I find myself installing something over and over (because I periodically purge my profile), I know for sure that it's time to add it to my config. This way I end up pulling less crap I don't actually use into my setup in an enduring way.
Maybe I'd also feel the same way about invoking things via `nix run` or `nix-shell` over time, and that would motivate me to incorporate them into my config ‘for realsies’ by declaring them.
> instead of updating your config and running home-manager switch?
I'm not currently a home-manager user on NixOS. Before home-manager was a thing, I used to define groups of packages using buildEnv and store them in an overlay for very simple de-facto declarative package management, so something like:
I'm switching to home-manager in my new setup, but one thing I still like about `nix profile install ...` (the flakes-based/next-gen replacement for nix-env, currently) is that it's user-mode/unprivileged, and it doesn't involve rebuilding my whole system (or anything other than the dependencies of just the package I want), even if my nixpkgs checkout/channel/flake registry or whatever has changed. `home-manager rebuild switch` is also unprivileged and also doesn't involve updating my whole system, but unfortunately home-manager doesn't support flakes just yet. You can use it on flakes-based setups on NixOS and macOS via the home-manager NixOS and nix-darwin modules, respectively... but then you're giving up the other benefits I like, because you have to invoke nixos-rebuild after all!
If `nix profile` were some day removed along with `nix-env`, but I had a user-level declarative environment management tool (like home-manager or something more tightly integrated), I could probably get by with just a little discipline about how I choose to edit my configurations and manage sources of Nix expressions and be pretty happy.
But I think the problems with `nix-env`/`nix profile` are pretty solvable, and I think lacking any imperative solution at all will likely put some ‘winnable’ new users off.
I do agree that `nix-env` itself sucks and needs to go, though. `nix-env --upgrade` doesn't really do what people expect, and there's no real reason to use/prefer it over just removing/reinstalling a package. The way that `nix-env` thinks about versions is basically insane, since Nixpkgs doesn't have any real version metadata and `nix-env` just parses attribute names to get the versions back out. `nix-env --query` is clunky and slow (but the new `nix search` is awesome and crazy fast!). `nix-env` was a cool thing for its time, and it's actually how `home-manager` manages its profiles on the backend (which is why flake support is still lacking; enabling flakes disables `nix-env` for your user). But it's like an imitation of `rpm`, and it was made before Nix had a real userbase and opportunities to think about what operations/abstractions/metadata were desirable at that level.
One of the things that's cool about Nix's design is that its design allows you to just bypass the hardest and most annoying problem that faces traditional package managers: dependency resolution. If you want a package manager that's guaranteed to give you solutions that are correct and complete, you need a SAT solver for dependency resolution, and that's NP-complete. By leveraging its quasi-content-addressable derivation approach, Nix gets to avoid resolving dependencies like traditional package managers do— the thing you need is the thing you were built against, and that's that! Similarly, by leaning hard into the Nixpkgs monorepo so that almost all packages live on it and all third-party package sources are built as de facto overlays on top of it, Nix has been able to totally avoid having to think about versions.
Compared to formats like DEB or RPM, Nix packages have very, very little metadata. Nix packagers don't have to declare things like acceptable version ranges for dependencies, what packages provide, what package names ought to be considered equivalent, what other packages it's incompatible with, or even what version number a package has. But `nix-env`'s `-q`, -i`, and `-u` options all imitate the `rpm` CLI, where all of that kind of metadata is necessary and present. And `nix-env` makes all that stuff work by assuming the structure of Nixpkgs and operating on Nix attributes representing packages directly. And it doesn't really make sense in the Nix world as it exists.
Flakes is the first attempt to revisit all these questions the community has basically punted on like ‘how do we want to relate packages to each other in a way that's not monorepo-centric?’, ‘what kind of metadata do we actually want to have for publishable Nix source package artifacts?’, ‘how should Nix code repositories advertise what features/tooling (packages, shell environments, modules, overlays, whatever) they support?’, ‘do we want Nix to actually be able to reason about versions?’, etc. (This also possibly reintroduces the question of dependency resolution. Hopefully not?) I think once we have real, considered answers for questions of that kind, grounded in the experience of the community so far, we can build an imperative frontend to Nix that makes sense and is nice to use.
That's definitely a factor. I think since I keep my Nix configurations in source control, somehow modifying the configuration feels more ‘official’, and it also usually comes with extra steps like committing and pushing.
The other thing I like is that it makes it very easy to tell if I actually want/need something: if I find myself installing something over and over (because I periodically purge my profile), I know for sure that it's time to add it to my config. This way I end up pulling less crap I don't actually use into my setup in an enduring way.
Maybe I'd also feel the same way about invoking things via `nix run` or `nix-shell` over time, and that would motivate me to incorporate them into my config ‘for realsies’ by declaring them.
> instead of updating your config and running home-manager switch?
I'm not currently a home-manager user on NixOS. Before home-manager was a thing, I used to define groups of packages using buildEnv and store them in an overlay for very simple de-facto declarative package management, so something like:
nix-env -iA pxc-tui-apps
I'm switching to home-manager in my new setup, but one thing I still like about `nix profile install ...` (the flakes-based/next-gen replacement for nix-env, currently) is that it's user-mode/unprivileged, and it doesn't involve rebuilding my whole system (or anything other than the dependencies of just the package I want), even if my nixpkgs checkout/channel/flake registry or whatever has changed. `home-manager rebuild switch` is also unprivileged and also doesn't involve updating my whole system, but unfortunately home-manager doesn't support flakes just yet. You can use it on flakes-based setups on NixOS and macOS via the home-manager NixOS and nix-darwin modules, respectively... but then you're giving up the other benefits I like, because you have to invoke nixos-rebuild after all!
If `nix profile` were some day removed along with `nix-env`, but I had a user-level declarative environment management tool (like home-manager or something more tightly integrated), I could probably get by with just a little discipline about how I choose to edit my configurations and manage sources of Nix expressions and be pretty happy.
But I think the problems with `nix-env`/`nix profile` are pretty solvable, and I think lacking any imperative solution at all will likely put some ‘winnable’ new users off.
I do agree that `nix-env` itself sucks and needs to go, though. `nix-env --upgrade` doesn't really do what people expect, and there's no real reason to use/prefer it over just removing/reinstalling a package. The way that `nix-env` thinks about versions is basically insane, since Nixpkgs doesn't have any real version metadata and `nix-env` just parses attribute names to get the versions back out. `nix-env --query` is clunky and slow (but the new `nix search` is awesome and crazy fast!). `nix-env` was a cool thing for its time, and it's actually how `home-manager` manages its profiles on the backend (which is why flake support is still lacking; enabling flakes disables `nix-env` for your user). But it's like an imitation of `rpm`, and it was made before Nix had a real userbase and opportunities to think about what operations/abstractions/metadata were desirable at that level.
One of the things that's cool about Nix's design is that its design allows you to just bypass the hardest and most annoying problem that faces traditional package managers: dependency resolution. If you want a package manager that's guaranteed to give you solutions that are correct and complete, you need a SAT solver for dependency resolution, and that's NP-complete. By leveraging its quasi-content-addressable derivation approach, Nix gets to avoid resolving dependencies like traditional package managers do— the thing you need is the thing you were built against, and that's that! Similarly, by leaning hard into the Nixpkgs monorepo so that almost all packages live on it and all third-party package sources are built as de facto overlays on top of it, Nix has been able to totally avoid having to think about versions.
Compared to formats like DEB or RPM, Nix packages have very, very little metadata. Nix packagers don't have to declare things like acceptable version ranges for dependencies, what packages provide, what package names ought to be considered equivalent, what other packages it's incompatible with, or even what version number a package has. But `nix-env`'s `-q`, -i`, and `-u` options all imitate the `rpm` CLI, where all of that kind of metadata is necessary and present. And `nix-env` makes all that stuff work by assuming the structure of Nixpkgs and operating on Nix attributes representing packages directly. And it doesn't really make sense in the Nix world as it exists.
Flakes is the first attempt to revisit all these questions the community has basically punted on like ‘how do we want to relate packages to each other in a way that's not monorepo-centric?’, ‘what kind of metadata do we actually want to have for publishable Nix source package artifacts?’, ‘how should Nix code repositories advertise what features/tooling (packages, shell environments, modules, overlays, whatever) they support?’, ‘do we want Nix to actually be able to reason about versions?’, etc. (This also possibly reintroduces the question of dependency resolution. Hopefully not?) I think once we have real, considered answers for questions of that kind, grounded in the experience of the community so far, we can build an imperative frontend to Nix that makes sense and is nice to use.
> or even what version number a package has
In the context of within nix packages that's correct. In the context of nix usage that is not.
I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.
Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.
> and it doesn't involve rebuilding my whole system
True. One thing I'm 100% sure is that nix-env -i will do just that and nothing else, whereas nixos-rebuild switch might include something else pending I might have forgotten about because it relies on globals.
In the context of within nix packages that's correct. In the context of nix usage that is not.
I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.
Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.
> and it doesn't involve rebuilding my whole system
True. One thing I'm 100% sure is that nix-env -i will do just that and nothing else, whereas nixos-rebuild switch might include something else pending I might have forgotten about because it relies on globals.
> I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.
> Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.
Right, right. That's a real problem. Gentoo Portage has mostly a big monorepo kinda like Nixpkgs, and in it you can find multiple versions of many pieces of software. Maybe the sensible future in the Nix world would be:
1. Our package attributes include version metadata, perhaps through something like ‘subflakes’ within the repo.
2. Nixpkgs includes multiple versions of major pieces of software.
3. Nixpkgs' top-level remains basically as it is now, in that for the most part only the latest version of something is used as a dependency. Alternatively, we use some kind of a ‘lock file’ that gets published with Nixpkgs. This way `nix profile install` still doesn't have to perform any dependency resolution for packages inside nixpkgs, so its behavior stays fast and predictable.
4. You can add version constraints in defining packages for use outside Nixpkgs, in `nix shell` environments, etc.
Does that seem like the way to go for you, or is your ideal picture something else?
> Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.
Right, right. That's a real problem. Gentoo Portage has mostly a big monorepo kinda like Nixpkgs, and in it you can find multiple versions of many pieces of software. Maybe the sensible future in the Nix world would be:
1. Our package attributes include version metadata, perhaps through something like ‘subflakes’ within the repo.
2. Nixpkgs includes multiple versions of major pieces of software.
3. Nixpkgs' top-level remains basically as it is now, in that for the most part only the latest version of something is used as a dependency. Alternatively, we use some kind of a ‘lock file’ that gets published with Nixpkgs. This way `nix profile install` still doesn't have to perform any dependency resolution for packages inside nixpkgs, so its behavior stays fast and predictable.
4. You can add version constraints in defining packages for use outside Nixpkgs, in `nix shell` environments, etc.
Does that seem like the way to go for you, or is your ideal picture something else?
I do have that use case as well. Taking a parallel from git, it's kind of a working/staging area/stash. There's a feeling of zero commitment. It could be I try out some unknown package, or that I have a temporary one-shot need that still lasts longer than a nix-shell -p.
But mostly I use it as a beachhead for people to eventually jump ship, helping them climb the first step of the ladder:
"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"
Then I bait them with e.g shell.nix just enough to tease their curiosity.
So instead of feeling overwhelmed by a whole new arch and contractually tied by configs they feel free, empowered, and curious.
But mostly I use it as a beachhead for people to eventually jump ship, helping them climb the first step of the ladder:
"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"
Then I bait them with e.g shell.nix just enough to tease their curiosity.
So instead of feeling overwhelmed by a whole new arch and contractually tied by configs they feel free, empowered, and curious.
> If you want something "installed" use home-manager.
I hear you. I understand nix-env as it exists needs to go, and I can only trust you on the support side. I did not know about home-manager.
Doing my homework, from home-manager README, I can read:
> Before attempting to use Home Manager please read the warning below.
> Unfortunately, it is quite possible to get difficult to understand errors when working with Home Manager, such as infinite loops with no clear source reference. You should therefore be comfortable using the Nix language and the various tools in the Nix ecosystem.
> Home Manager targets NixOS unstable and NixOS version 20.09 (the current stable version), it may or may not work on other Linux distributions and NixOS versions.
> Also, the home-manager tool does not explicitly support rollbacks at the moment so if your home directory gets messed up you'll have to fix it yourself.
On top of being third party (for now?), all of this really does not bode confidence in the tool and severely raises the bar for adoption when all one wants is to install tmux globally for their user (IOW a bunch of symlinks in ~/.nix-profile/bin that "just works").
But the best one is this in the manual:
> This manual will eventually describes how to install, use, and extend Home Manager.
There is no section in the manual describing the usage of the tool. The Getting Started seems to be solely about development and contributing. The terse examples in the README is what made me search for the manual, and I could only get a grasp of what they entail because I have NixOS experience. Seriously, if this is deemed "general public availability" quality, this is borderline user hostile.
So, I do note the envisioned deprecated-ness of nix-env but I will continue to use that as a first rung to help people climb the ladder when I introduce people to nix until there is a suitably accessible and reliable replacement.
Indeed I successfully used it as a beachhead for people to eventually jump ship:
"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"
Then I bait them with e.g shell.nix (which feels like Gemfile) or nix-shell -p --run (which feels like docker run) just enough to make it relatable while teasing their curiosity.
So instead of feeling overwhelmed by a whole new arch complete with a foreign config system and an alien language, and contractually tied by configs they feel confident, uncommitted, free, empowered, and curious, and that much more likely to transition to the next rung up.
> If you just want something for quick dev use nix-shell.
That I 100% agree with, which is why I immediately described it as well. It's the hook to the next rung.
> In some cases Home Manager cannot detect whether it will overwrite a previous manual configuration.
That, to me, is a problem. I fully acknowledge the limitations and caveats of nix-env, and duly highlight them when introducing people to nix, but this means home-manager is not the tool I need, in git parlance it's way too much "porcelain" and not enough "plumbing". It feels invasive when you're still in the process of getting acquainted.
I do not want home-manager to handle launchd, nor .gitconfig. I do not use nix-darwin or NixOS, _on purpose_, as I use nixpkgs as an additional tool, an extension of existing systems.
I am fine if nix-env has some of its features revamped, or is deprecated and replaced by a better tool but in terms of some of its existing use cases it is exactly the tool needed, i.e merely allow users or root to make a package persistent and widely available for the user or for root in their PATH, as if it were a native package, and with no other side effects.
I hope you understand these are legitimate needs.
I hear you. I understand nix-env as it exists needs to go, and I can only trust you on the support side. I did not know about home-manager.
Doing my homework, from home-manager README, I can read:
> Before attempting to use Home Manager please read the warning below.
> Unfortunately, it is quite possible to get difficult to understand errors when working with Home Manager, such as infinite loops with no clear source reference. You should therefore be comfortable using the Nix language and the various tools in the Nix ecosystem.
> Home Manager targets NixOS unstable and NixOS version 20.09 (the current stable version), it may or may not work on other Linux distributions and NixOS versions.
> Also, the home-manager tool does not explicitly support rollbacks at the moment so if your home directory gets messed up you'll have to fix it yourself.
On top of being third party (for now?), all of this really does not bode confidence in the tool and severely raises the bar for adoption when all one wants is to install tmux globally for their user (IOW a bunch of symlinks in ~/.nix-profile/bin that "just works").
But the best one is this in the manual:
> This manual will eventually describes how to install, use, and extend Home Manager.
There is no section in the manual describing the usage of the tool. The Getting Started seems to be solely about development and contributing. The terse examples in the README is what made me search for the manual, and I could only get a grasp of what they entail because I have NixOS experience. Seriously, if this is deemed "general public availability" quality, this is borderline user hostile.
So, I do note the envisioned deprecated-ness of nix-env but I will continue to use that as a first rung to help people climb the ladder when I introduce people to nix until there is a suitably accessible and reliable replacement.
Indeed I successfully used it as a beachhead for people to eventually jump ship:
"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"
Then I bait them with e.g shell.nix (which feels like Gemfile) or nix-shell -p --run (which feels like docker run) just enough to make it relatable while teasing their curiosity.
So instead of feeling overwhelmed by a whole new arch complete with a foreign config system and an alien language, and contractually tied by configs they feel confident, uncommitted, free, empowered, and curious, and that much more likely to transition to the next rung up.
> If you just want something for quick dev use nix-shell.
That I 100% agree with, which is why I immediately described it as well. It's the hook to the next rung.
> In some cases Home Manager cannot detect whether it will overwrite a previous manual configuration.
That, to me, is a problem. I fully acknowledge the limitations and caveats of nix-env, and duly highlight them when introducing people to nix, but this means home-manager is not the tool I need, in git parlance it's way too much "porcelain" and not enough "plumbing". It feels invasive when you're still in the process of getting acquainted.
I do not want home-manager to handle launchd, nor .gitconfig. I do not use nix-darwin or NixOS, _on purpose_, as I use nixpkgs as an additional tool, an extension of existing systems.
I am fine if nix-env has some of its features revamped, or is deprecated and replaced by a better tool but in terms of some of its existing use cases it is exactly the tool needed, i.e merely allow users or root to make a package persistent and widely available for the user or for root in their PATH, as if it were a native package, and with no other side effects.
I hope you understand these are legitimate needs.
I use nix-shell for dev environments. Once I've got it working it's wonderful - full replicable, across multiple development machines and CI. It's painful though. Some problems I've encountered recently on different projects:
Old version of fwup packaged. Can override to get newer version, but needed to add another dependency to make tests pass (and it gets built locally rather than being cached)
Pain getting my environment to include the right Python packages so I can run the gigalixir command line tool.
Have to workaround an oddity to do with how Erlang is packaged so that I don't get lots of warnings when anything is compiled:
Old version of fwup packaged. Can override to get newer version, but needed to add another dependency to make tests pass (and it gets built locally rather than being cached)
Pain getting my environment to include the right Python packages so I can run the gigalixir command line tool.
Have to workaround an oddity to do with how Erlang is packaged so that I don't get lots of warnings when anything is compiled:
shellHook =
''
export ERL_LIBS=""
'';Just a hint - extra attributes in the `mkShell` appear as environment variables in the built shell. So instead of a `shellHook` for this you can just have an attribute `ERL_LIBS="";`
I'm not completely sure but I think this approach is preferred because shellHook only runs in `nix-shell` but if you ever wanted to `nix-build` a release it might not run.
I'm not completely sure but I think this approach is preferred because shellHook only runs in `nix-shell` but if you ever wanted to `nix-build` a release it might not run.
There tools like direnv, that "enter" your env environment as soon as you cd to the directory. This is implemented by changing environment variables like PATH.
Because shellHook allows on executing any code, there's no standardized way to undo what it executes, so those tools don't execute that part.
Because shellHook allows on executing any code, there's no standardized way to undo what it executes, so those tools don't execute that part.
My experience, a while ago I will admit, is that your experience will vary wildly based on what you’re doing with it.
Writing Haskell in it was a joy, but Ruby was pure pain. I actually never could get RoR to run (this might have changed), so I gave up and went to OSX for that stuff.
Writing Haskell in it was a joy, but Ruby was pure pain. I actually never could get RoR to run (this might have changed), so I gave up and went to OSX for that stuff.
I haven't used Ruby, but Python also was a huge pain and there were many attempts at fixing it, but all had some issues for me. That is until I found poetry2nix.
My understanding is that Python packages are directly installable via nix, at least some of them. This is a completely parallel way of installing packages from the regular way, which sucks, but at least it works in theory. I never personally tried it though.
Stack (Haskell) was awesome because it’s Nix aware, enabling it to create its own env to handle C deps on it’s own.
Stack (Haskell) was awesome because it’s Nix aware, enabling it to create its own env to handle C deps on it’s own.
How does poetry2nix compare to mach-nix, in your opinion? The way mach-nix handles native (non-Python) dependencies strikes me as really slick.
So I haven't used mach-nix, but from what I see it provides its own way of specifying dependencies. While in case of poetry2nix you are using poetry for dependency management and poetry2nix just translates it to Nix.
The biggest benefit for using poetry is the dependency resolver which is much smarter than in pip and actually finds packages that match the version requirements (although looks like mach-nix also has a smart resolver.
Other benefits are:
* nice CLI
* ability to package application and upload to PyPI
* poetry can still be used by people who don't use Nix
Regarding non-python dependencies, this is solved by overrides file[1]. That way most of the time you don't have to think about it. You can also override the override in your project as well.
[1] https://github.com/nix-community/poetry2nix/blob/master/over...
The biggest benefit for using poetry is the dependency resolver which is much smarter than in pip and actually finds packages that match the version requirements (although looks like mach-nix also has a smart resolver.
Other benefits are:
* nice CLI
* ability to package application and upload to PyPI
* poetry can still be used by people who don't use Nix
Regarding non-python dependencies, this is solved by overrides file[1]. That way most of the time you don't have to think about it. You can also override the override in your project as well.
[1] https://github.com/nix-community/poetry2nix/blob/master/over...
Long time Arch user here but I'm growing more interested in NixOS by the day. I'm mainly “utilitarian” in my OS choice and other things being equal, Arch's AUR packages is what keeps me using it.
How does the Nix ecosystem compare with AUR? Would you still recommend making the switch?
How does the Nix ecosystem compare with AUR? Would you still recommend making the switch?
I was able to make the switch over cold turkey after ~11 years of ArchLinux.
By sheer happenstance, I blogged earlier this week about one particular killer feature that doesn't get enough air time: https://news.ycombinator.com/item?id=27344677
My not-flake-yet configuration can be found at https://github.com/rraval/nix
By sheer happenstance, I blogged earlier this week about one particular killer feature that doesn't get enough air time: https://news.ycombinator.com/item?id=27344677
My not-flake-yet configuration can be found at https://github.com/rraval/nix
If you don't mind the storage penalty, it looks pretty convenient to set up an Arch Linux chroot and use packages from the AUR when you need to: https://github.com/lemniskett/archbox/blob/master/NIXOS_INST...
Nixpkgs itself is several times the size of the base Arch Linux package collection, and by ‘non-unique’ package count, Nixpkgs is also much larger than the AUR. In addition to Nixpkgs, you can find Nix packages in several community ‘overlays’ for Nixpkgs as well as Nix's own user repositories.
You can check to see whether everything you currently use/need is conveniently available for NixOS in a comprehensive-ish way through the combination of these two web search tools:
• for Nixpkgs/NixOS: https://search.nixos.org/packages?channel=21.05
• for NUR: https://nur.nix-community.org/
NixOS also includes native Flatpak support.
Fwiw, packaging most things for Nix is very easy. I left Arch in ~2010 because at the time the package management stack and default repos on Arch basically sucked compared to most distros I'd used and liked, and from then on I decided that if I wanted software that wasn't in my distro's repos I'd just package it myself. After taking a little time to learn the tools on whatever distro I was using, I never missed Arch or the AUR. Compared to other distros, packaging normal software from source is usually exceptionally easy on NixOS.
If I were you I'd just dive right in and hit Nix's channel on Matrix with the Nixpkgs manual in hand if I found something I wanted to use that wasn't already packaged. But you can fall back on the options outlined above.
Nixpkgs itself is several times the size of the base Arch Linux package collection, and by ‘non-unique’ package count, Nixpkgs is also much larger than the AUR. In addition to Nixpkgs, you can find Nix packages in several community ‘overlays’ for Nixpkgs as well as Nix's own user repositories.
You can check to see whether everything you currently use/need is conveniently available for NixOS in a comprehensive-ish way through the combination of these two web search tools:
• for Nixpkgs/NixOS: https://search.nixos.org/packages?channel=21.05
• for NUR: https://nur.nix-community.org/
NixOS also includes native Flatpak support.
Fwiw, packaging most things for Nix is very easy. I left Arch in ~2010 because at the time the package management stack and default repos on Arch basically sucked compared to most distros I'd used and liked, and from then on I decided that if I wanted software that wasn't in my distro's repos I'd just package it myself. After taking a little time to learn the tools on whatever distro I was using, I never missed Arch or the AUR. Compared to other distros, packaging normal software from source is usually exceptionally easy on NixOS.
If I were you I'd just dive right in and hit Nix's channel on Matrix with the Nixpkgs manual in hand if I found something I wanted to use that wasn't already packaged. But you can fall back on the options outlined above.
Almost everything I used on Arch is available on NixOS. In fact, for a lot of software I was able to use a more advanced configuration because NixOS makes it easier than Arch to set up.
I only ran into one hiccup: NixOS has an open pull request for supporting plymouth for silent boot with a LUKS encrypted drive. For now, I am stuck with an ugly password prompt when I turn my computer on.
I only ran into one hiccup: NixOS has an open pull request for supporting plymouth for silent boot with a LUKS encrypted drive. For now, I am stuck with an ugly password prompt when I turn my computer on.
Great. That won't be a problem, as I love ugly password prompts.
Edit: Relocating /boot to a flash or SD-card is also a way to accomplish deniability of a cryptroot.
Edit: Relocating /boot to a flash or SD-card is also a way to accomplish deniability of a cryptroot.
It should be pointed out that you can install the nix package manager in Arch (or whatever other Linux, or macOS, etc.), so you can try out nix's declarative package management without actually switching to NixOS.
Doh! You're right, I completely forgot that. Thanks for pointing it out.
I tried this on my previous laptop and ran into a number of issues once I tried to install anything with a GUI. It's fine for shells and CLI tools though. I migrated some configs over to a Nix configs under Arch, and while it was a pain to initially set up NixOS (unfamilarity), it's a lot easier doing everything else now.
Yeah, I’m not sure how up-to-date my knowledge is, but opengl and the like are exceptions to the usual deterministic handling of dependencies on non-NixOS distros (not because it is unable to do so, I think it is mainly to avoid storing everything n-times with nvidia/amd), and one has to specify them. It was quite a time I ran nix on a non-nixos distro but there is this tool https://github.com/guibou/nixGL that meant to solve the issue of graphical programs.
nixGL works without issue on my laptop's Ubuntu install, and the project also includes an equivalent wrapper for Vulkan rather than OpenGL.
nixpkgs is of a very similar size to the aur (or even larger depending how you count), but is significantly more up to date: https://repology.org/
So if I understand correctly, I could declaratively specify my workstation, say I want vim, podman, codium, kubectl, curl, tmux, firefox, zettlr. I could put this in a configuration and have my workstation anywhere I can grab that from?
Is there some way I can boot-to-git-configuration? So I could do this in RAM on someone else's hardware, like a network boot?
Is there some way I can boot-to-git-configuration? So I could do this in RAM on someone else's hardware, like a network boot?
>Is there some way I can boot-to-git-configuration?
Yes, the NixOS installer is a live disk with all the features of NixOS. I have booted configurations directly from IPFS in the installer, and many nix commands and functions support git (or GitHub) natively.
You can declaratively specify your whole infrastructure from software configuration to remote machines with their own VMs.
For example here's my nix function for matrix: https://0x0.st/-2CA.nix
It does a whole bunch of stuff.
First it reads in the hostname and domain from the system configuration and put that in a variable to be used when configuring the subdomains later.
Then it opens up your firewall, starts a postgres service, creates the matrix db/user, configures nginx for matrix.hortname.domain, gets your SSL certificate (and sets up automatic renewal), sets up matrix, and lastly a hosted web client at element.host.domain. All services run under their own users on the host system.
It's zero configuration, I just import that to any machine that I want and it's running a matrix server with all appropriate ancillary configuration done.
Yes, the NixOS installer is a live disk with all the features of NixOS. I have booted configurations directly from IPFS in the installer, and many nix commands and functions support git (or GitHub) natively.
You can declaratively specify your whole infrastructure from software configuration to remote machines with their own VMs.
For example here's my nix function for matrix: https://0x0.st/-2CA.nix
It does a whole bunch of stuff.
First it reads in the hostname and domain from the system configuration and put that in a variable to be used when configuring the subdomains later.
Then it opens up your firewall, starts a postgres service, creates the matrix db/user, configures nginx for matrix.hortname.domain, gets your SSL certificate (and sets up automatic renewal), sets up matrix, and lastly a hosted web client at element.host.domain. All services run under their own users on the host system.
It's zero configuration, I just import that to any machine that I want and it's running a matrix server with all appropriate ancillary configuration done.
i hope you aren't still using that shared secret
Nix's IPFS support is for everything in the Nix store, which is world-readable and in which NixOS places no secrets. Options for managing secrets in/with Nix projects either encrypt them in the store or never have them touch the store (encrypted or otherwise).
Mutable state like database contents, including their password databases, don't go in the Nix store either.
Mutable state like database contents, including their password databases, don't go in the Nix store either.
Or the password for that Postgres user...
Thanks I'll have a read through it, I can lots of uses for this already!
> So if I understand correctly, I could declaratively specify my workstation, say I want vim, podman, codium, kubectl, curl, tmux, firefox, zettlr. I could put this in a configuration and have my workstation anywhere I can grab that from?
Kind of. It works very well if you own multiple computers that you want to be configured similarly (like a dotfiles repo on steroids).
It doesn't work so well for I borrowed my coworker's computer for 5 minutes and want to use my own Vim configuration.
However, Nix does also supply the "nix run" command, which allows you to use software without installing it permanently (but without NixOS' configuration support).
> Is there some way I can boot-to-git-configuration? So I could do this in RAM on someone else's hardware, like a network boot?
Yes. You could either build an image with a service that git clones your configuration and switches to it, or you could build an image containing a prebuilt snapshot of your configuration.
Either approach should be possible for basically any medium (USB drive, DVD, netboot, whatever). That said, for the build-and-switch setup I'd recommend something that can give at least /nix/store a writable partition (to avoid having to rebuild/redownload the whole world on every reboot).
Kind of. It works very well if you own multiple computers that you want to be configured similarly (like a dotfiles repo on steroids).
It doesn't work so well for I borrowed my coworker's computer for 5 minutes and want to use my own Vim configuration.
However, Nix does also supply the "nix run" command, which allows you to use software without installing it permanently (but without NixOS' configuration support).
> Is there some way I can boot-to-git-configuration? So I could do this in RAM on someone else's hardware, like a network boot?
Yes. You could either build an image with a service that git clones your configuration and switches to it, or you could build an image containing a prebuilt snapshot of your configuration.
Either approach should be possible for basically any medium (USB drive, DVD, netboot, whatever). That said, for the build-and-switch setup I'd recommend something that can give at least /nix/store a writable partition (to avoid having to rebuild/redownload the whole world on every reboot).
>It doesn't work so well for I borrowed my coworker's computer for 5 minutes and want to use my own Vim configuration
You could use something like nixos-shell[1] to spin up a headless VM of your machine into your current shell.
[1]https://github.com/Mic92/nixos-shell
You could use something like nixos-shell[1] to spin up a headless VM of your machine into your current shell.
[1]https://github.com/Mic92/nixos-shell
Well that is pretty neat. Thanks for the mention of nixos-shell. I've been a NixOS daily-driver for the past 6 months or so, but I am still picking up new ideas on a daily basis. :)
It's been my daily for not much longer than that, and almost every time I play with NixOS itself I get the sense of wonder and possibility in computers that I haven't had since I was a teenager. It's definitely a force multiplier.
Ive seen mention to "switching" to a configuration a few times now, what does that mean technically? Is it a reboot and select a different grub option, does it take time/have to download things? What really is "switching"?
Each NixOS machine has a current configuration, symlinked from /run/current-system. For example, my NixOS laptop currently has this:
Let's say I want to change something. I edit /etc/nixos/configuration.nix, then run "nixos-rebuild build", which produces this:
When I'm ready to change, I run "/nix/store/0fhxj.../bin/switch-to-configuration". This step actually changes the /run/current-system symlink, updates /etc, restarts any daemons that were updated, etc. Obviously, if your configuration has a new kernel, that won't be applied until you reboot.
In practice, we almost always just run "nixos-rebuild switch", which does the build and switch in one go.
This whole process is entirely idempotent and reversible; I can switch back to any configuration I've ever built (unless I've since removed them to save space, of course). GRUB also displays a list of every configuration you've ever built, so you can always boot into a stable configuration even if you've somehow made a royal mess of things.
/run/current-system -> /nix/store/95n5xr5n8bw6xbab8fiz6cqc6mydryg8-nixos-system-Sweetpea-21.11pre291991.ea7d4aa9b82
Let's say I want to change something. I edit /etc/nixos/configuration.nix, then run "nixos-rebuild build", which produces this:
/nix/store/0fhxjqbv20psbkjvw70wni9ww2cxfqgm-nixos-system-Sweetpea-21.11pre292748.6933d068c5d
When I'm ready to change, I run "/nix/store/0fhxj.../bin/switch-to-configuration". This step actually changes the /run/current-system symlink, updates /etc, restarts any daemons that were updated, etc. Obviously, if your configuration has a new kernel, that won't be applied until you reboot.
In practice, we almost always just run "nixos-rebuild switch", which does the build and switch in one go.
This whole process is entirely idempotent and reversible; I can switch back to any configuration I've ever built (unless I've since removed them to save space, of course). GRUB also displays a list of every configuration you've ever built, so you can always boot into a stable configuration even if you've somehow made a royal mess of things.
Thanks for taking the time to describe that, it makes sense now.
I'm impressed that it manages restarting everything on the fly without a reboot on a running system.
I'm impressed that it manages restarting everything on the fly without a reboot on a running system.
On rare occasions, there can be problems restarting a service, like if the service in the new configuration in fact relies on a newer kernel module than what you've got in your running configuration. In addition, sometimes Nix's garbage collector won't dispose of something because it can tell you've still got some processes running that use it or something like that, and so there's a bit of space you can't free up until after reboot. And of course, kernel changes still require reboots on NixOS.
All in all, the service management stuff is handled very gracefully imo. I've never run into real trouble with it (and one time I had a power outage in the middle of a `nixos-rebuild switch`).
All in all, the service management stuff is handled very gracefully imo. I've never run into real trouble with it (and one time I had a power outage in the middle of a `nixos-rebuild switch`).
Building the new configuration (and downloading or building dependencies), adding the bootloader entry, and activating the new configuration (stopping removed services, starting new services, reloading changed services, that kind of thing).
You don't need to do all three: you can skip the bootloader if you want to try something temporarily, and you can skip activation if you want to wait with that until the next reboot. There are also some changes that can't be activated without a reboot, such as changing the kernel.
> Does it take time/have to download things?
Building has to download or build stuff that isn't in the cache. But contrary to, say, Docker it understands the build graph, so it only rebuilds actual dependees.
Activation depends on how many services you've changed, and how quickly they restart. Updating the bootloader is nearly instant.
You don't need to do all three: you can skip the bootloader if you want to try something temporarily, and you can skip activation if you want to wait with that until the next reboot. There are also some changes that can't be activated without a reboot, such as changing the kernel.
> Does it take time/have to download things?
Building has to download or build stuff that isn't in the cache. But contrary to, say, Docker it understands the build graph, so it only rebuilds actual dependees.
Activation depends on how many services you've changed, and how quickly they restart. Updating the bootloader is nearly instant.
Thats pretty incredible. So I can work on configs, switch back and forth until it's exactly right and then switch to it permanently.
Where's a good place to start? Some of the configs people are throwing about are quite involved and look like they need a bunch of specialist knowledge.
Thanks for taking the time to explain. Can't wait to start trying this out.
Where's a good place to start? Some of the configs people are throwing about are quite involved and look like they need a bunch of specialist knowledge.
Thanks for taking the time to explain. Can't wait to start trying this out.
Just take a few minutes to skim the NixOS manual, then keep it in front of you on another device while you do an install. Enable a web browser, a desktop environment, and NetworkManager in your initial configuration.nix.
Then just start adding software that you want to be able to use. Search for it first in the NixOS options[1]. If you see it there, enable it and configure it using the options that make sense to you. If you don't see it there, search for it in the package collection[2], then use the name you find for it there to add it to `environment.systemPackages` in your `configuration.nix` file.
Just kinda take it easy and continue with that process until you have an environment that's more or less comfy. Then you're free to revisit your config and think about stuff like tying it in with your dotfiles, using Nix to manage your home directory, services you might want to run, etc.
—
1: https://search.nixos.org/options?channel=21.05&
2: https://search.nixos.org/packages?channel=21.05&
Then just start adding software that you want to be able to use. Search for it first in the NixOS options[1]. If you see it there, enable it and configure it using the options that make sense to you. If you don't see it there, search for it in the package collection[2], then use the name you find for it there to add it to `environment.systemPackages` in your `configuration.nix` file.
Just kinda take it easy and continue with that process until you have an environment that's more or less comfy. Then you're free to revisit your config and think about stuff like tying it in with your dotfiles, using Nix to manage your home directory, services you might want to run, etc.
—
1: https://search.nixos.org/options?channel=21.05&
2: https://search.nixos.org/packages?channel=21.05&
> However, Nix does also supply the "nix run" command, which allows you to use software without installing it permanently (but without NixOS' configuration support).
What do you mean without config support? It runs a nix derivation which can be arbitrarily complex. Do you mean your /etc/nixos/configuration.nix file? Because you are not really using it either, only for creating a derivation that will become your system. A nix-shell of a user will by default use the user’s channel and may not do anything with the global config.
What do you mean without config support? It runs a nix derivation which can be arbitrarily complex. Do you mean your /etc/nixos/configuration.nix file? Because you are not really using it either, only for creating a derivation that will become your system. A nix-shell of a user will by default use the user’s channel and may not do anything with the global config.
Yeah, I think they just mean without the conveniences of the NixOS module system. You can definitely bundle config with an app and store it in a repository's shell.nix or a flake or whatever.
Alternative to what others have said, you can use your config to build a custom bootable ISO and use it to setup your new system. Comes very handy if you can't guarantee nice network for setup.
Last time I tried this, the NixOS Live USB still requires network connectivity to install. Is there some way to create an ISO or Live USB that installs completely offline?
Nix kind of expects to be always connected.
It is possible though, if you have the same config and use channels at the same revision (or use flakes) nix will know all packages it needs. You could fetch them and then either provide that machine as a cache, or use nix copy to copy the packages.
It is possible though, if you have the same config and use channels at the same revision (or use flakes) nix will know all packages it needs. You could fetch them and then either provide that machine as a cache, or use nix copy to copy the packages.
This sounds like a great idea. Do you happen to know of any documentation to assist with this? I'd love to create a custom, bootable USB which I can upgrade periodically, keeping my SSH keys on a Yubikey.
One way is to use this project: https://github.com/nix-community/nixos-generators
It's just a small wrapper but it shows you most of the targets available.
It's just a small wrapper but it shows you most of the targets available.
Great, thanks!
> So if I understand correctly, I could declaratively specify my workstation, say I want vim, podman, codium, kubectl, curl, tmux, firefox, zettlr. I could put this in a configuration and have my workstation anywhere I can grab that from?
Yes, and you can specify your .vimrc, your vim plugins, the users you want setup, their SSH keys, your systemd config, etc.
> Is there some way I can boot-to-git-configuration?
Bootstrapping a system is the one place where you still need to do a bit of work, but it's not bad.
I personally have a script in my Nix config repo which pulls my config down to disk, sets up symlinks, and then switches to the new config. From there everything then goes through standard Nix tooling.
Yes, and you can specify your .vimrc, your vim plugins, the users you want setup, their SSH keys, your systemd config, etc.
> Is there some way I can boot-to-git-configuration?
Bootstrapping a system is the one place where you still need to do a bit of work, but it's not bad.
I personally have a script in my Nix config repo which pulls my config down to disk, sets up symlinks, and then switches to the new config. From there everything then goes through standard Nix tooling.
What you can relatively easily do is extend the 'installer' config with whatever you want to add there, for instance applications and a desktop environment.
It should be relatively straightforward to also mount /home from a separate USB stick partition, but I haven't tried.
It should be relatively straightforward to also mount /home from a separate USB stick partition, but I haven't tried.
Oh yeah, that's not a bad idea and definitely what I'd do if I was bootstrapping a bunch of hosts at once. As it is I set up maybe one new host a year, which means that it's at least as much work to rebuild an installer boot media + my config as it is to do a base install and then pipe a script through bash to finish setup.
Amazing thanks. I'm going to start taking a look at getting a simple configuration started with my basic setup, keys, shell, tmux config etc.
With nixops you also define define your configurations in nix (and can have interdependencies between them) then start machines or virtual machines as blank nixos installs and add their ssh key and address for nixops to reconfigure and manage.
I ultimately find it easier to have one host that is keeping track of all configurations and pushing my changes to all hosts since I otherwise end up with a lot of machines with different changes I've been working on.
I ultimately find it easier to have one host that is keeping track of all configurations and pushing my changes to all hosts since I otherwise end up with a lot of machines with different changes I've been working on.
Almost any system can do that.
Package managers typically have a concept of a “selected set” or something o that nature which is often simply a file that contains all the packages the user wants to install on a new line at some places. Simply copy that file and do a world update with the package manager and it will install all those packages an their dependencies and typically uninstall what is not needed.
Package managers typically have a concept of a “selected set” or something o that nature which is often simply a file that contains all the packages the user wants to install on a new line at some places. Simply copy that file and do a world update with the package manager and it will install all those packages an their dependencies and typically uninstall what is not needed.
NixOS also covers package configuration, not just which packages are installed. And because it owns all of the configuration, it can semi-intelligently merge package configurations that are managed in multiple independent places.
It can also be used to apply patches to packages (which are then automatically carried over when the upstream package is updated).
It can also be used to apply patches to packages (which are then automatically carried over when the upstream package is updated).
> NixOS also covers package configuration, not just which packages are installed. And because it owns all of the configuration, it can semi-intelligently merge package configurations that are managed in multiple independent places.
Do you mean system configuration in /etc, or also user configuration in /home?
In the latter case I do not see how this can ever be implemented, even if an exhaustive list be provided with each package in it's description, quite a bit of software will arbitrarily add new files under new names as part of it's confguration.
Usually one simply copies one's entire `~/.config` directory along.
> It can also be used to apply patches to packages (which are then automatically carried over when the upstream package is updated).
True, but this is also a feature of any source-based system.
The real advantage to me seems to be that it is a source-based system but with a central repository for identically configured packages to avoid compilation in many cases.
Do you mean system configuration in /etc, or also user configuration in /home?
In the latter case I do not see how this can ever be implemented, even if an exhaustive list be provided with each package in it's description, quite a bit of software will arbitrarily add new files under new names as part of it's confguration.
Usually one simply copies one's entire `~/.config` directory along.
> It can also be used to apply patches to packages (which are then automatically carried over when the upstream package is updated).
True, but this is also a feature of any source-based system.
The real advantage to me seems to be that it is a source-based system but with a central repository for identically configured packages to avoid compilation in many cases.
> Do you mean system configuration in /etc, or also user configuration in /home?
NixOS covers /etc, and Home-Manager covers /home. But in practice many applications support configuration from /etc as well, and you will tend to prefer just managing it from there when using NixOS.
> In the latter case I do not see how this can ever be implemented, even if an exhaustive list be provided with each package in it's description, quite a bit of software will arbitrarily add new files under new names as part of it's confguration.
Hm? If you use Nix to manage the application's configuration then Nix is the only program allowed to change the configuration directly.
> Usually one simply copies one's entire `~/.config` directory along.
Not the same thing. Nix allows you to express relationships between different application configurations (why does program A try to connect to localhost:9267? oh, ripgrep says that program B is binding that port for RPC...), and lets you stack configurations (so that defaults can be updated automatically, or you can change a specific value for a specific machine without having to maintain two completely separate copies).
It's like the difference between giving a new employee access to your source code, vs saying "here's an optimized stripped build, and here's a copy of Ghidra, have fun!".
> True, but this is also a feature of any source-based system.
In the same sense that you can copy your dotfiles around manually, sure. The difference is that Nix allows you to patch the package definition itself dynamically. AUR/PKGBUILD (the only other source-based system I'm familiar with to be fair, not sure if Gentoo handles this differently) allows you to patch the /application/, but you still need to copy the PKGBUILD and backport all upstream changes.
NixOS covers /etc, and Home-Manager covers /home. But in practice many applications support configuration from /etc as well, and you will tend to prefer just managing it from there when using NixOS.
> In the latter case I do not see how this can ever be implemented, even if an exhaustive list be provided with each package in it's description, quite a bit of software will arbitrarily add new files under new names as part of it's confguration.
Hm? If you use Nix to manage the application's configuration then Nix is the only program allowed to change the configuration directly.
> Usually one simply copies one's entire `~/.config` directory along.
Not the same thing. Nix allows you to express relationships between different application configurations (why does program A try to connect to localhost:9267? oh, ripgrep says that program B is binding that port for RPC...), and lets you stack configurations (so that defaults can be updated automatically, or you can change a specific value for a specific machine without having to maintain two completely separate copies).
It's like the difference between giving a new employee access to your source code, vs saying "here's an optimized stripped build, and here's a copy of Ghidra, have fun!".
> True, but this is also a feature of any source-based system.
In the same sense that you can copy your dotfiles around manually, sure. The difference is that Nix allows you to patch the package definition itself dynamically. AUR/PKGBUILD (the only other source-based system I'm familiar with to be fair, not sure if Gentoo handles this differently) allows you to patch the /application/, but you still need to copy the PKGBUILD and backport all upstream changes.
*Almost any system can attempt that.
Nix lets you pin your whole dependency tree, and Nixpkgs includes facilities for managing the configurations of many applications, including integrations with their plugin systems that explicitly manage external runtime dependencies, like stuff your vim configuration might call at the CLI. Without that stuff, reproduction can often fail in practice.
And in the case of NixOS, you can also reproduce things like running services and their configurations, which users exist, etc. Those are OS and configuration management features, not package management.
Nix lets you pin your whole dependency tree, and Nixpkgs includes facilities for managing the configurations of many applications, including integrations with their plugin systems that explicitly manage external runtime dependencies, like stuff your vim configuration might call at the CLI. Without that stuff, reproduction can often fail in practice.
And in the case of NixOS, you can also reproduce things like running services and their configurations, which users exist, etc. Those are OS and configuration management features, not package management.
Could you share a link to your repo? This setup sounds interesting. Currently I have two separate repos for NixOS and home-manager.
I use nixpkgs on my osx laptop for most everything over homebrew these days, I've upgraded nix to 2.4, but home-manager isn't compatible. Hopefully that will change soon.
Good thing about nix package manager is that you can have same path using it on macOS and on Linux, so your shell config files are easier to maintain without if's.
Linuxbrew installing at /home/linuxbrew/.linuxbrew/ is a bit of a joke too.
Linuxbrew installing at /home/linuxbrew/.linuxbrew/ is a bit of a joke too.
Nix is a great replacement for homebrew, but the lack of native support for Apple Silicon at the moment is a blocker for me.
https://github.com/NixOS/nixpkgs/pull/105026 this was recently merged so that should start rapidly improving.
The show's happening here now:
https://github.com/NixOS/nixpkgs/issues/95903
https://github.com/NixOS/nixpkgs/pull/125184
https://github.com/NixOS/nix/pull/4867
There's also not-directly-related cross-functional need here on the install process:
https://github.com/NixOS/nix/pull/4289
https://github.com/NixOS/nixpkgs/issues/95903
https://github.com/NixOS/nixpkgs/pull/125184
https://github.com/NixOS/nix/pull/4867
There's also not-directly-related cross-functional need here on the install process:
https://github.com/NixOS/nix/pull/4289
This excellent Linux distribution has the honour of not needing to recommend a clean install. It deserves to be far more popular. I've run it since 2014 with so few problems, that I can't really claim to "know" Linux anymore.
I still can’t quite figure out why it’s not more popular... then again, I don’t use it either, sadly, because it’s not as popular as I feel it deserves; I feel like there must be some reason it’s not used more, so in turn haven’t dedicated the time to learn it. Perhaps that’s cyclic...
IMO the biggest blocker is documentation. The best way to figure out how to do something is typically to search around in the nixpkgs/NixOS options for what you're trying to do, then go read the code.
There's very little in the way of "here's a common task that new users want to do and here's the blessed way to do it." Even something that should be Nix's MAJOR selling point, like "create a single repo that configures a bunch of machines, including some NixOS and some other OS" doesn't have a recommended way to do it.
Another challenge with the docs is the fact that Nix is
* An OS * A package manager * A way to manage dev environments
A lot of times I find someone who solves my problem but they solve it in the context of building a dev environment and I want to do it in the context of setting up an OS package. It means that even with all the good tutorials and blog posts out there, you might still have a bunch of work to do to get things to work how you want. This translates into tons & tons of opportunities for users to give up.
Also, for some reason, so many examples are done in the imperative manner, and it's not always super clear how to translate them to a declarative setup. Again, just defeating the strengths of Nix and clouding the benefits. If a new user sees a bunch of examples that involve starting nix-shell and installing some dependencies it's both hard to understand the benefits and hard to translate that to a declarative setup.
There's very little in the way of "here's a common task that new users want to do and here's the blessed way to do it." Even something that should be Nix's MAJOR selling point, like "create a single repo that configures a bunch of machines, including some NixOS and some other OS" doesn't have a recommended way to do it.
Another challenge with the docs is the fact that Nix is
* An OS * A package manager * A way to manage dev environments
A lot of times I find someone who solves my problem but they solve it in the context of building a dev environment and I want to do it in the context of setting up an OS package. It means that even with all the good tutorials and blog posts out there, you might still have a bunch of work to do to get things to work how you want. This translates into tons & tons of opportunities for users to give up.
Also, for some reason, so many examples are done in the imperative manner, and it's not always super clear how to translate them to a declarative setup. Again, just defeating the strengths of Nix and clouding the benefits. If a new user sees a bunch of examples that involve starting nix-shell and installing some dependencies it's both hard to understand the benefits and hard to translate that to a declarative setup.
> Even something that should be Nix's MAJOR selling point, like "create a single repo that configures a bunch of machines, including some NixOS and some other OS" doesn't have a recommended way to do it.
NixOps[0]? But yeah, it isn't featured as prominently anymore as it used to be...
[0]: https://github.com/NixOS/nixops
NixOps[0]? But yeah, it isn't featured as prominently anymore as it used to be...
[0]: https://github.com/NixOS/nixops
NixOps is fine for pushing changes to machines (well, actually I think it has a lot of problems, and somehow the NixOps docs are even worse than the regular Nix docs as far as getting started and bootstrapping and doing basic tasks, but that's not the point). But it's all about the mechanics of pushing changes to your hosts.
It certainly doesn't give you any kind of opinion on how to, say, structure a repo that has your hardware configs, Home Manager on a non-NixOS machine, Home Manager on a NixOS machine, some systemd services, etc. And there's no reason to need NixOps to maintain a repo of three or four physical machines that you have around your house.
It certainly doesn't give you any kind of opinion on how to, say, structure a repo that has your hardware configs, Home Manager on a non-NixOS machine, Home Manager on a NixOS machine, some systemd services, etc. And there's no reason to need NixOps to maintain a repo of three or four physical machines that you have around your house.
> And there's no reason to need NixOps to maintain a repo of three or four physical machines that you have around your house.
You don't. You can use `nixos-rebuild test --target-host=red --flake .#red`. If you want to build locally, add --build-host=localhost.
NixOps gives you the advantage of not having to repeat yourself, but you could easily whip up a shell script for `rebuild-switch red`.
You don't. You can use `nixos-rebuild test --target-host=red --flake .#red`. If you want to build locally, add --build-host=localhost.
NixOps gives you the advantage of not having to repeat yourself, but you could easily whip up a shell script for `rebuild-switch red`.
It sounds like nix provides an escape hatch [1] out of the declarative framework into something imperative, e.g. running `apt install $package`. And that seems like it would really ease adoption, since I would assume that it would allow a gradual adoption of the declarative mindset.
[1] https://wiki.c2.com/?EscapeHatch
[1] https://wiki.c2.com/?EscapeHatch
IMO the escape hatch doesn't get you any real benefits. It's not like you can install packages from apt/yum/aur so you don't get access to any new software (plus nixpkgs is bigger than all those). All it does is result in confusing tutorials that hide the benefits of declarative management and make it difficult for new users who just want to get things working.
> It's not like you can install packages from apt/yum/aur
Yes you can. Here's a Nix derivation which makes a Debian chroot and runs 'apt-get update; apt-get install -y chromium' https://github.com/Warbo/warbo-utilities/blob/7087c3b46a97f3...
Here a Nix derivation for a helper script that runs the above chromium: https://github.com/Warbo/warbo-utilities/blob/7087c3b46a97f3...
On a similar note, here's a Nix derivation for Mozilla's Firefox binary (in an FHS env), along with a warning if it's not the latest version: http://chriswarbo.net/git/warbo-packages/git/branches/master...
I wrote these once cache.nixos.org stopped providing "non-core" i686 binaries a few years ago, and I can't be bothered to build those large applications from source.
Yes you can. Here's a Nix derivation which makes a Debian chroot and runs 'apt-get update; apt-get install -y chromium' https://github.com/Warbo/warbo-utilities/blob/7087c3b46a97f3...
Here a Nix derivation for a helper script that runs the above chromium: https://github.com/Warbo/warbo-utilities/blob/7087c3b46a97f3...
On a similar note, here's a Nix derivation for Mozilla's Firefox binary (in an FHS env), along with a warning if it's not the latest version: http://chriswarbo.net/git/warbo-packages/git/branches/master...
I wrote these once cache.nixos.org stopped providing "non-core" i686 binaries a few years ago, and I can't be bothered to build those large applications from source.
Oh yeah, great point! I was thinking of installing packages from apt/yum/aur directly with something like nix-env, but you're right, there's no reason you can't just use apt-get directly.
nix-env isn't meaningfully an escape hatch, since you can still only use it with stuff that's packaged in Nix's peculiar way.
Here's an overview of the escape hatches that are currently around in NixOS for package management, meaning you can use them to avoid packaging something natively for Nix, or to run pre-built software for other distros or targeting multiple distros:
• flatpak — use flatpak to install stuff just like you would on any other distro: https://nixos.org/manual/nixos/stable/index.html#module-serv...
• FHS user environments — create a usermode chroot that puts Nix packages where naive `./configure; make; make install` programs will expect to see their dependencies
• steam-run — run whatever in the pre-built FHSUserEnv that Nixpkgs uses for Steam: https://nixos.org/manual/nixpkgs/stable/#sec-steam-run
• AppImage — launch an AppImage using `appimage-run` or create a quick and dirty Nix package that wrapps an existing AppImage: https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-appimageTo...
• Docker — you can run stuff with Docker if ya want: https://search.nixos.org/options?channel=unstable&show=virtu...
NixOS also includes little escape hatches on the configuration side for every service. Basically every module also has an ‘extraConfig’ type of option, which lets you append to the config file that NixOS generates for the service in its original/raw format.
Otherwise, you're generally looking at setting up a chroot environment. I can think of some ways that NixOS could be a little more generous with escape hatches, but I don't think there's much developer interest in the community.
Here's an overview of the escape hatches that are currently around in NixOS for package management, meaning you can use them to avoid packaging something natively for Nix, or to run pre-built software for other distros or targeting multiple distros:
• flatpak — use flatpak to install stuff just like you would on any other distro: https://nixos.org/manual/nixos/stable/index.html#module-serv...
• FHS user environments — create a usermode chroot that puts Nix packages where naive `./configure; make; make install` programs will expect to see their dependencies
• steam-run — run whatever in the pre-built FHSUserEnv that Nixpkgs uses for Steam: https://nixos.org/manual/nixpkgs/stable/#sec-steam-run
• AppImage — launch an AppImage using `appimage-run` or create a quick and dirty Nix package that wrapps an existing AppImage: https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-appimageTo...
• Docker — you can run stuff with Docker if ya want: https://search.nixos.org/options?channel=unstable&show=virtu...
NixOS also includes little escape hatches on the configuration side for every service. Basically every module also has an ‘extraConfig’ type of option, which lets you append to the config file that NixOS generates for the service in its original/raw format.
Otherwise, you're generally looking at setting up a chroot environment. I can think of some ways that NixOS could be a little more generous with escape hatches, but I don't think there's much developer interest in the community.
> Basically every module also has an ‘extraConfig’ type of option
This is actually changing so that the additional config can be merged/validated even if it can't be typed
https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-o...
This is actually changing so that the additional config can be merged/validated even if it can't be typed
https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-o...
This is pretty exciting! In this case, users will retain an 'escape hatch' in the sense that they'll be free to descend from whatever higher-level configuration options a module author wants to offer (which could set multiple settings, or generate settings based on other settings) and use all the configuration primitives offered by the underlying software being configured. We just won't have to manage it as a big, dumb string anymore. :D
I suppose this could still be annoying for someone who really just wants to copy/paste, though.
I suppose this could still be annoying for someone who really just wants to copy/paste, though.
It does for some things. For example you can run `nix-env -iA firefox` to install firefox (for your user) without changing the config. (Or even better `nix run nixos.firefox -c firefox` to run it without installing) However some things like adding/editing files in /etc is a lot harder as those paths are basically recreated on boot.
A few things which might count as "escape hatches":
The `runCommand` function lets us define a "package" by just executing bash code. This avoids all the "phase" machinery that most nixpkgs definitions use (those phases are great for fine-grained overriding; but we don't usually needed to override our own definitions). It takes 3 arguments: the output name, a set of env vars (including `buildInputs` which will populate $PATH) and a string of bash code (which can be multi-line using ''two quotes''):
We can avoid having to hard-code hashes for "fixed output derivations" (like downloaded URLs) by overriding their `outputHash`, `outputHashAlgo`, `outputHashMode` and `sha256` to `null`.
I wrote up a bunch of these sorts of hacks at http://chriswarbo.net/projects/nixos/useful_hacks.html (although its a few years old, so might not be up to date)
Of course, these things aren't advisable; but they're very useful to get up and running quickly, and things can be made "more Nixy" later.
The `runCommand` function lets us define a "package" by just executing bash code. This avoids all the "phase" machinery that most nixpkgs definitions use (those phases are great for fine-grained overriding; but we don't usually needed to override our own definitions). It takes 3 arguments: the output name, a set of env vars (including `buildInputs` which will populate $PATH) and a string of bash code (which can be multi-line using ''two quotes''):
$ nix-build -E '(with import <nixpkgs> {}; runCommand "hello.txt" { buildInputs = [ foo bar baz ]; } "echo hello > $out")'
these derivations will be built:
/nix/store/z6ffwc1jb70zwz2ly42gaalxnvm7q0gk-hello.txt.drv
building '/nix/store/z6ffwc1jb70zwz2ly42gaalxnvm7q0gk-hello.txt.drv'...
/nix/store/l6sd75hj4854pdzv4044ma42axc2v5v4-hello.txt
with import <nixpkgs> {};
runCommand "my-app"
{
buildInputs = [ (python3.withPackages (p: [ p.pip ])) ];
}
''
echo "Making mutable copy of app" 1>&2
cp -r ${./.} "$out"
chmod +x -R "$out"
echo "Putting dependencies in place" 1>&2
pip3 install -r ${./requirements.txt} --target "$out"/
''
We can avoid having to hard-code hashes for "fixed output derivations" (like downloaded URLs) by overriding their `outputHash`, `outputHashAlgo`, `outputHashMode` and `sha256` to `null`.
I wrote up a bunch of these sorts of hacks at http://chriswarbo.net/projects/nixos/useful_hacks.html (although its a few years old, so might not be up to date)
Of course, these things aren't advisable; but they're very useful to get up and running quickly, and things can be made "more Nixy" later.
For the use case of developer machines, NixOS is much more friction compared to other OSs. It's not about how nice Nix stuff is when it works. It's about how much of a pain it is to get things working when things aren't nice and you don't know what to do.
With other popular distributions, you can rely on being able to find a StackOverflow answer with whatever your error message is. Most other OSs and package managers are malleable enough that you can easily beat them into shape should anything go wrong. It's also usually easy enough to adapt how things are done in one Linux distribution to another, but Nix is weird and takes a long time to figure out. (Nix stuff itself being expressive enough that there's often more than one way to do things).
With other popular distributions, you can rely on being able to find a StackOverflow answer with whatever your error message is. Most other OSs and package managers are malleable enough that you can easily beat them into shape should anything go wrong. It's also usually easy enough to adapt how things are done in one Linux distribution to another, but Nix is weird and takes a long time to figure out. (Nix stuff itself being expressive enough that there's often more than one way to do things).
> It's about how much of a pain it is to get things working when things aren't nice and you don't know what to do.
Thank you for highlighting this. I love elegance, safety, reproducibility, declarative specifications, etc. But the minute I get the sense of "this philosophy is getting in the way. I am stuck, even though I know exactly what to do below this layer of abstraction", I get very annoyed.
That's why I asked about "escape hatches" in a sibling comment.
Thank you for highlighting this. I love elegance, safety, reproducibility, declarative specifications, etc. But the minute I get the sense of "this philosophy is getting in the way. I am stuck, even though I know exactly what to do below this layer of abstraction", I get very annoyed.
That's why I asked about "escape hatches" in a sibling comment.
Worst case just run chroots, or docker for the rare case something really don’t want to run under Nix. But it rarely happens anymore, it has a huge ecosystem with almost everything packaged so depending on how niche is the thing you work on, it can be a really comfortable environment.
There would be less friction if say NPM and package managers like these weren't installing files in random places, assuming locations of files/binaries instead of querying, let libraries and binaries be configurable, and/or not following the XDG spec. This is probably what developers should be doing, but many can't be asked to understand Ubuntu isn't the only Linux distro.
Software that isn't already packaged is hard to use at all. Often possible -- steam-run is handy; despite the name it basically provides a Ubuntu env -- but many forms of software development are difficult-to-impossible without first learning about Nix internals.
It can be a superb experience after you go through that trouble. Or it might not be. It depends on what you're trying to do, exactly, but you have to first spend a lot of effort learning before you'll have your answer.
Personally, I found it worth my while.
It can be a superb experience after you go through that trouble. Or it might not be. It depends on what you're trying to do, exactly, but you have to first spend a lot of effort learning before you'll have your answer.
Personally, I found it worth my while.
Could you briefly mention an example of something that is difficult/impossible without first learning about Nix internals, and allude to what kind of internals need to be learned?
As one example, IIRC, I can run "npm install -g" on other Linux distributions, but can't with Nixpkgs (since npm will try to install the package to the nix store). Yarn would install it to the home folder.
Fwiw you can use `npm install -g` if you configure NPM to use your homedir for globally installed packages, as well. It's just not the default behavior.
Yea this can be very annoying when a package is not already in nixpkgs or it's out of date.
There was a similar issue if you didn't use Python virtualenvs properly, which was an early source of frustration for me personally.
There was a similar issue if you didn't use Python virtualenvs properly, which was an early source of frustration for me personally.
For starters, the recommended way for really understanding how to setup any package or option is to read the source in nixpkgs. Depends what you mean by "Nix internals" but for me that qualifies.
As another example, binary blobs are usually hard to get them to run. But steam-run (a base debian chroot, but you can even specifically build custom ones with only specific libraries available), or in the worst case just running a docker container can solve everything.
Maybe figure out why none of the cloud providers provide Nix as their install OS image.
If that happens, users will easily double and maybe more hands might fix the docs or perhaps it's a chicken egg problem but if it's the doc, it's easily solvable.
If that happens, users will easily double and maybe more hands might fix the docs or perhaps it's a chicken egg problem but if it's the doc, it's easily solvable.
Running something like nixos-infect to convert an Ubuntu machine into NixOS is going to be a new user’s easiest part of using NixOS. After that, it’s a pretty steep learning curve (or a tutorial) to do anything interesting. I’m doubtful that deleting that first step would bring more users.
That is easy. It is unpoplar and not compatible with popular images. The default image for a cloud provider is going to target the common use case and easiest way for a novice to get started. If you already know what you are doing you will select a custom image anyways.
It is great, but a bit painful to learn and get working for you. I like to use Nix to provide my development environments but that's often painful due to conflicts between the Nix approach and how the language package manager wants to work.
Yes this. I tried this a few times but always ended in me banging against some esoteric issue. For example I tried to nixify a project I had which needs ruby and node. If you think you can spin up any version of ruby you are mistaken. The project removes older versions from time to time. The wrapper to fetch the correct gems from nix to setup the nix-env had really hard problems when gems or npm packages started their own native builds.
I’m a huge fan of the project and will give it a try again at one point. But I decided for now to go fully manual with arch to get a better understanding of most components involved in Linux before using the automatic that is nix.
I’m a huge fan of the project and will give it a try again at one point. But I decided for now to go fully manual with arch to get a better understanding of most components involved in Linux before using the automatic that is nix.
> But I decided for now to go fully manual with arch to get a better understanding of most components involved in Linux before using the automatic that is nix.
I think one can do this separately, e.g. run NixOS for the host OS and then do LFS or Gentoo or Arch in a VM where you can easily blow things up and revert to a prior snapshot.
Also, flakes takes care of the channels shenanigans by pinning input package versions.
The hardest thing about Nix is that there is very little in the way of "beginner to intermediate" documentation. There is a huge gap between tutorials about just tweaking configuration.nix and a bit a people who have insanely abstract / modularized configs.
The key insight for me was that although they sell the "it's a lazily-evaluated language" thing really hard, at the end of the day there are a few things forcing side effects:
- There are certain key attributes that nix will call forceAttrs on (in the interpreter). You can think of these as the "roots" that will act as a starting point for evaluating all other attributes as needed. This is the stuff you usually see like nixosConfigurations, shell, etc. The sad part is that you really have to read the C++ source to understand this it seems. - At the "bottom", there are a bunch of things built in to write to files in the sandboxed result/ directory, but there are also these monstrous bash scripts to do a lot of the heavy lifting, e.g. the mkDerivation one. It's important to read over these scripts to understand what's being written.
I've found this youtube series [0] to be a good way to figure out how to go from beginner to intermediate.
[0] https://www.youtube.com/playlist?list=PL-saUBvIJzOkjAw_vOac7...
I think one can do this separately, e.g. run NixOS for the host OS and then do LFS or Gentoo or Arch in a VM where you can easily blow things up and revert to a prior snapshot.
Also, flakes takes care of the channels shenanigans by pinning input package versions.
The hardest thing about Nix is that there is very little in the way of "beginner to intermediate" documentation. There is a huge gap between tutorials about just tweaking configuration.nix and a bit a people who have insanely abstract / modularized configs.
The key insight for me was that although they sell the "it's a lazily-evaluated language" thing really hard, at the end of the day there are a few things forcing side effects:
- There are certain key attributes that nix will call forceAttrs on (in the interpreter). You can think of these as the "roots" that will act as a starting point for evaluating all other attributes as needed. This is the stuff you usually see like nixosConfigurations, shell, etc. The sad part is that you really have to read the C++ source to understand this it seems. - At the "bottom", there are a bunch of things built in to write to files in the sandboxed result/ directory, but there are also these monstrous bash scripts to do a lot of the heavy lifting, e.g. the mkDerivation one. It's important to read over these scripts to understand what's being written.
I've found this youtube series [0] to be a good way to figure out how to go from beginner to intermediate.
[0] https://www.youtube.com/playlist?list=PL-saUBvIJzOkjAw_vOac7...
Thanks for the resources!
I switched from macOS to linux for my personal machine last summer. I played around with nix on some machines before and also used nix-pkg on macOS but ran into issues. There is a great community behind it and I read a lot about nix (the language) etc.
My decision to go with arch was mainly practical. I don't want a huge desktop etc. I wanted to see what gnome etc actually automate for me. I have already a way better understanding what kind of features which I normally take for granted coming from macOS are not part in a base system.
I will give it a go again when I have time as I like the declaritive nature of nixOS.
edit spelling
I will give it a go again when I have time as I like the declaritive nature of nixOS.
edit spelling
I recently moved back to Arch from NixOS. I really love the idea, and the execution isn't necessarily bad, but it didn't make sense for me on the desktop. Straying off the beaten path quickly led to the sense that I was going to have to learn way more about Nix than I wanted to. I hope the documentation and community grows, because I definitely got the sense that it is the future arriving early!
The documentation and the nix language are the worst things about the project IMO. I really hope Guix succeeds because Scheme is, in my opinion, a better DSL and language than this weird, not very well documented, Haskell-like derivative.
Unless you've drank the kool-aid, you have to learn a completely new operating system and way of doing things, a weird language that's lacking on documentation, and a ton of system or package-specific options that are even less documented, and requires you to spend too much time reading the source to understand what's going on and how to achieve something a little off the beaten path.
Feels a bit cultish IMO, and that only the illuminated ones are able to grok it. That it's my fault I'm not prepared to do the work and see for myself why NixOS is better than any other distro.
Unless you've drank the kool-aid, you have to learn a completely new operating system and way of doing things, a weird language that's lacking on documentation, and a ton of system or package-specific options that are even less documented, and requires you to spend too much time reading the source to understand what's going on and how to achieve something a little off the beaten path.
Feels a bit cultish IMO, and that only the illuminated ones are able to grok it. That it's my fault I'm not prepared to do the work and see for myself why NixOS is better than any other distro.
I agree on every single point. I'd love to make GuixSD my permanent home.
But, Guix is not just Nix with better language. It comes with its own baggage elsewhere. Guix repository has way wayy fewer packages. Some of the most common ones like the plasma-desktop are absent. Proprietary applications are a no-go. Even if I set up nonguix, I'm warned that that cuts off my support in official channels. As an Optimus laptop owner (because my Master's required running Cuda and pytorch) I'm simply left hanging on Guix.
I understand and often appreciate Guix's stand on everything above. No. of packages is because of strong stance on bootstrapping, proprietary package unavailability is self a explanatory.
But, that doesn't solve my problems, and creates few new ones. Even Debian allows using proprietary firmware blobs with some fiddling and doesn't turn people away for asking. These barriers are so much that I bit the bullet and (mostly) learned the weird Nix DSL.
As much as I'd like to go for Guix, for my personal use case, it is a nonstarter.
But, Guix is not just Nix with better language. It comes with its own baggage elsewhere. Guix repository has way wayy fewer packages. Some of the most common ones like the plasma-desktop are absent. Proprietary applications are a no-go. Even if I set up nonguix, I'm warned that that cuts off my support in official channels. As an Optimus laptop owner (because my Master's required running Cuda and pytorch) I'm simply left hanging on Guix.
I understand and often appreciate Guix's stand on everything above. No. of packages is because of strong stance on bootstrapping, proprietary package unavailability is self a explanatory.
But, that doesn't solve my problems, and creates few new ones. Even Debian allows using proprietary firmware blobs with some fiddling and doesn't turn people away for asking. These barriers are so much that I bit the bullet and (mostly) learned the weird Nix DSL.
As much as I'd like to go for Guix, for my personal use case, it is a nonstarter.
Yes, if Guix were "Nix but with Scheme" I'd go for it in a second. Unfortunately, although both Nix and Guix make hard choices about things (e.g. the init system), at least the choices made by Nix are more livable day-to-day.
As to why Nix wasn't some system written using Haskell (for actual language features that is also lazy) or Prolog (for its whole thing about solving constraint-based problems all over the place), I will never understand...
As to why Nix wasn't some system written using Haskell (for actual language features that is also lazy) or Prolog (for its whole thing about solving constraint-based problems all over the place), I will never understand...
Guix and Nix don’t actually solve constraint problems, though. By design, they omit the dependency resolution that other package managers do, and humans manage the dependencies. If you want to change versions, you need to do some hacking to update things and propagate requirements through the package graph.
You could think of Spack (github.com/spack/spack) as a nix/guix-like model, but with a solver. Spack uses ASP (which is prolog-like but reduces to SAT with optimization) but packages are written in an embedded python DSL.
You could think of Spack (github.com/spack/spack) as a nix/guix-like model, but with a solver. Spack uses ASP (which is prolog-like but reduces to SAT with optimization) but packages are written in an embedded python DSL.
Would it be possible to write a CLI tool that translated Scheme into Nix-compatible config files? Presumably then you could alias Nix commands so they piped config 8n through the translator.
It might be possible, but a better approach (and the one Guix used initially, iirc) is to just be compatible with the derivation format (kinda like object files in C/C++ compilation). The end result is basically a big dependency graph, so one may be able to write a different generator in front of it.
Anyone who has done the basics in Prolog could "logically" pull this off (e.g. some rule requires another rule, or however Prolog works; it's been a long time since I looked at it), but all of the extra stuff in the library (e.g. all of the systemd tweaking, interface, etc) is a LOT of man hours.
I heard about Spack [0] on a recent cppcast episode, which seems like a competitor to Nix (but from the supercomputing side of things), but I don't know if they do the OS-level stuff (vs. nix shell / conda / pip / gem replacement stuff).
[0] https://spack.io/
Anyone who has done the basics in Prolog could "logically" pull this off (e.g. some rule requires another rule, or however Prolog works; it's been a long time since I looked at it), but all of the extra stuff in the library (e.g. all of the systemd tweaking, interface, etc) is a LOT of man hours.
I heard about Spack [0] on a recent cppcast episode, which seems like a competitor to Nix (but from the supercomputing side of things), but I don't know if they do the OS-level stuff (vs. nix shell / conda / pip / gem replacement stuff).
[0] https://spack.io/
Nix and similar projects are all going to be eaten alive by immutable distributions, Flatpak and new projects - that might work just like Nix but get the user experience right, and make packaging and maintenance easier. And no I'm not saying they do the exact same thing, have all the same features or target the same type of users.
For the developer tools side I like what Shopify did but I would never recommend that to my employer. Too many moving parts and not enough people using it that I would feel safe about it.
For the developer tools side I like what Shopify did but I would never recommend that to my employer. Too many moving parts and not enough people using it that I would feel safe about it.
As a new NixOS user currently set up with home manager and flakes - i'd probably switch in a heartbeat if someone gave me a nice nix-like package manager that also handled dotfiles.
As much as i love the feeling of NixOS, i really want something like a Lockfile as seen in Rust (and other languages).
Ie if i could define a `Cargo.toml`, which includes versions of packages or `*` if i don't care - and then it gets built into a `Cargo.lock`, i'd be in heaven. Combine that with a great language backing the distro and i'd switch immediately.
As it stands, I hate nixlang, and while flakes is amazing i still dislike the single primary input approach to NixOS _(ie all the packages bundled together)_.
As an example of why i dislike that, i'm stuck on an old version of NixOS currently because when i tried to update the repo date - lots of packages changed in difficult to manage ways. X was crazy slow, Firefox was being wonky, XFCE was janky, etc. All my flakes.lock told me was that the hash of the repo was different.. yay.. hundreds of dependencies were different i'm sure, but no clue which ones, and no easy way to isolate the problems and just incrementally update.
Luckily flakes allowed me to rollback perfectly. Well, i still had some userland state from the newer applications that i needed to nuke, but i'm ignoring that for now.
Being able to more easily incrementally update specific dependencies would be amazing for me. As it stands i have no clue when or how i'll update my NixOS input version. Which is not a promising sign for my continued usage of NixOS.
As much as i love the feeling of NixOS, i really want something like a Lockfile as seen in Rust (and other languages).
Ie if i could define a `Cargo.toml`, which includes versions of packages or `*` if i don't care - and then it gets built into a `Cargo.lock`, i'd be in heaven. Combine that with a great language backing the distro and i'd switch immediately.
As it stands, I hate nixlang, and while flakes is amazing i still dislike the single primary input approach to NixOS _(ie all the packages bundled together)_.
As an example of why i dislike that, i'm stuck on an old version of NixOS currently because when i tried to update the repo date - lots of packages changed in difficult to manage ways. X was crazy slow, Firefox was being wonky, XFCE was janky, etc. All my flakes.lock told me was that the hash of the repo was different.. yay.. hundreds of dependencies were different i'm sure, but no clue which ones, and no easy way to isolate the problems and just incrementally update.
Luckily flakes allowed me to rollback perfectly. Well, i still had some userland state from the newer applications that i needed to nuke, but i'm ignoring that for now.
Being able to more easily incrementally update specific dependencies would be amazing for me. As it stands i have no clue when or how i'll update my NixOS input version. Which is not a promising sign for my continued usage of NixOS.
I don’t really understand your problem as in NixOS it is precisely the easiest to handle some packages from this repo and anothers from another. You can just add a single line of an alternative repo like altNixpkgs = “reference repo either by url or path” in the let in “block” and instead of pkgs.something just use altNixpkgs.pkgs.something.
Also, I usually manage my home packages separately from my global config. The easiest way to do that is probably by adding a different channel/path to your user’s nix-channels. I believe home-manager uses that. Alternatively you can also do the trick I mentioned in your home.nix file.
Also, I usually manage my home packages separately from my global config. The easiest way to do that is probably by adding a different channel/path to your user’s nix-channels. I believe home-manager uses that. Alternatively you can also do the trick I mentioned in your home.nix file.
> I don’t really understand your problem as in NixOS it is precisely the easiest to handle some packages from this repo and anothers from another. You can just add a single line of an alternative repo like altNixpkgs = “reference repo either by url or path” in the let in “block” and instead of pkgs.something just use altNixpkgs.pkgs.something.
Yea, i actually do that for a couple of my dependencies. My problem is there are dozens or hundreds in my OS that instantly upgrade if i change my `nixpkgs` ref. And i have no clue what version _any_ of those are at, off hand.
Eg lets say i want to fix my X - because when i updated my `nixpkgs` pointer Xorg was using a lot of CPU for some reason. There's no lockfile that says what version X is at, or what dependencies X is using or associated with. Is it one package? Is it a half dozen? I could open the repo and dig through the configs to locate it and _try_ to infer what the hell version it's at, and what dependencies it has.. and then find what they're at and so forth - but it's convoluted imo. Much easier in Cargo.lock, for example - something i've done dozens of times before.
In short, basically everything in my OS broke when i updated my nixpkgs ref, and it's non-obvious and maybe non-trivial to understand what versions everything is at. To incrementally try to un-screw my system when i change my nixpkgs.
As i move forward, any packages i install i'm tempted to bind them to very specific inputs. So that when i want to upgrade one thing, only that one thing changes. But this workflow seems difficult in NixOS. Despite it being the workflow that i want.
Well, maybe not difficult - but lots of LOC. Adding dozens of inputs just to change versions of stuff without breaking other things seems like a lot of work for a dependency management system.. imo.
Yea, i actually do that for a couple of my dependencies. My problem is there are dozens or hundreds in my OS that instantly upgrade if i change my `nixpkgs` ref. And i have no clue what version _any_ of those are at, off hand.
Eg lets say i want to fix my X - because when i updated my `nixpkgs` pointer Xorg was using a lot of CPU for some reason. There's no lockfile that says what version X is at, or what dependencies X is using or associated with. Is it one package? Is it a half dozen? I could open the repo and dig through the configs to locate it and _try_ to infer what the hell version it's at, and what dependencies it has.. and then find what they're at and so forth - but it's convoluted imo. Much easier in Cargo.lock, for example - something i've done dozens of times before.
In short, basically everything in my OS broke when i updated my nixpkgs ref, and it's non-obvious and maybe non-trivial to understand what versions everything is at. To incrementally try to un-screw my system when i change my nixpkgs.
As i move forward, any packages i install i'm tempted to bind them to very specific inputs. So that when i want to upgrade one thing, only that one thing changes. But this workflow seems difficult in NixOS. Despite it being the workflow that i want.
Well, maybe not difficult - but lots of LOC. Adding dozens of inputs just to change versions of stuff without breaking other things seems like a lot of work for a dependency management system.. imo.
Thank, I get it now. Not a solution but perhaps try git bisecting the working and non-working version. It’s never a good thing to hunt down a bug but when one has to, the immutable nature of nixos actually makes it a very handy process.
You can use the Nix tooling to show the diff between two derivations' closure.
I have a script which I use [1] after updating my Flakes to do just that : know what has been upgraded.
[1] https://github.com/nix-community/nur-combined/tree/master/re...
I have a script which I use [1] after updating my Flakes to do just that : know what has been upgraded.
[1] https://github.com/nix-community/nur-combined/tree/master/re...
Given that you already have flakeified everything, it's not that hard to pull in a different version of nixpkgs and cherry-pick packages from it. I have a (somewhat complex) setup utilizing this: https://github.com/casept/nixos-config
Yea, i actually do that now. My problem is the dozens of packages i didn't hand pick all broke. So basically i need to hand pick everything. And in doing so, it feels like a lot more work than simply specifying what versions i want during the package install.
I’m not too afraid of a niche being eaten alive by a slightly larger niche :D
Also, I don’t really see the value of flatpaks, they don’t solve the dependency hell problem (nix does) nor they are adequate sandboxes (be honest, linux has non-existent user-space security, it’s frankly terrible. And for some reason even flatpak is written in goddamn C in 2000+ something)
Also, I don’t really see the value of flatpaks, they don’t solve the dependency hell problem (nix does) nor they are adequate sandboxes (be honest, linux has non-existent user-space security, it’s frankly terrible. And for some reason even flatpak is written in goddamn C in 2000+ something)
Why do you consider flatpak as not having adequate sandboxing?
> For the developer tools side I like what Shopify did but I would never recommend that to my employer. Too many moving parts and not enough people using it that I would feel safe about it.
I'm not sure I understand you. In their case this became the standard dev environment. Everyone in the company is using it and the true benefit is that it makes development predictable. You only need one command to set up dev environment. He also mentioned that it is much faster than the old way.
I'm not sure I understand you. In their case this became the standard dev environment. Everyone in the company is using it and the true benefit is that it makes development predictable. You only need one command to set up dev environment. He also mentioned that it is much faster than the old way.
I'm out of the loop, what did Shopify do?
EDIT: ah, you're probably talking about https://shopify.engineering/shipit-presents-how-shopify-uses...
EDIT: ah, you're probably talking about https://shopify.engineering/shipit-presents-how-shopify-uses...
I hate that they mentioned about ru.nix, published it on github for short time and then unpublished it[1].
[1] https://github.com/Shopify/runix/
[1] https://github.com/Shopify/runix/
I think they wanted to avoid having people build on it before they stabilize its interfaces, since they still had some big changes planned. As I understand it, the plan is supposedly still to publish it at some point.
I hope that if that's true, they publish it soon, because its design is very relevant to ongoing changes in the ecosystem, and it could be a useful reference point. :(
I hope that if that's true, they publish it soon, because its design is very relevant to ongoing changes in the ecosystem, and it could be a useful reference point. :(
I don't think it is. The talk was a year ago and it still isn't available.
I created my own version, based on (I think they did that too) on NixOS modules system.
But I'm wondering how it compares.
Actually looking at https://gist.github.com/burke/72ca46c80e57a25907a75611ee5eb6... I think their goal is slightly different than mine.
There's also https://github.com/hercules-ci/project.nix but it wasn't touched for a year now.
I created my own version, based on (I think they did that too) on NixOS modules system.
But I'm wondering how it compares.
Actually looking at https://gist.github.com/burke/72ca46c80e57a25907a75611ee5eb6... I think their goal is slightly different than mine.
There's also https://github.com/hercules-ci/project.nix but it wasn't touched for a year now.
> Straying off the beaten path quickly leads
to a terrifying abyss. The contrast is remarkable. When everything you need is packaged and it just works, it's amazing. Hands down the best system management experience I've had. And then one day you learn you've been dancing on the edge of a cliff.
I'm hooked; there's no going back now for me. But there's at least one project (TidalCycles) that I simply gave up on ever running, because installing it requires peering deep inside the machine, and my psyche cannot handle it.
to a terrifying abyss. The contrast is remarkable. When everything you need is packaged and it just works, it's amazing. Hands down the best system management experience I've had. And then one day you learn you've been dancing on the edge of a cliff.
I'm hooked; there's no going back now for me. But there's at least one project (TidalCycles) that I simply gave up on ever running, because installing it requires peering deep inside the machine, and my psyche cannot handle it.
J/w, have you ever packaged software for other distros? If yes, what made Nix more difficult here? If no, have you tried asking for help packaging TidalCycles on Matrix or Discourse?
I'm wondering whether that's some especially oddly-behaved software, or if what's lacking here is some onboarding (that maybe can solved for you with a little community engagement or something).
I'm wondering whether that's some especially oddly-behaved software, or if what's lacking here is some onboarding (that maybe can solved for you with a little community engagement or something).
Nix is the only OS I've used where the standard installation instructions available in, say, the READMEs of things I want to install are a bad idea. Granted, whenever something is in nixpkgs it's a piece of cake, and almost everything I need is. But to install something not part of nixpkgs, I had to look under the hood in a way that no other platform has made me do.
I've never packaged software on another platform.
I packaged serialoscd and monomeserial, and the Discourse channel was critically helpful. It looks like I never tried engaging the community about TidalCycles. I found an old package, tried to fix it, couldn't.Thanks for sharing your experience in more detail.
I think Nixpkgs is easier to package for than a lot of ecosystems/distributions, but the obligation of _having_ to package for the distro in order to get things working in an ideal way is (reasonably!) more than many people want to have to deal with. (There is also a gap, kinda like across the built-in vs. have-to-package barrier, inside the Nixpkgs world, between software with reasonable/standard build systems and software with unique or ill-designed build processes.)
I've been daydreaming about how this could be better for a little while now, and wondering what compromises it will require to the ‘purity’ of the system, how worthwhile they'll be for novice and advanced users, etc.
I think Nixpkgs is easier to package for than a lot of ecosystems/distributions, but the obligation of _having_ to package for the distro in order to get things working in an ideal way is (reasonably!) more than many people want to have to deal with. (There is also a gap, kinda like across the built-in vs. have-to-package barrier, inside the Nixpkgs world, between software with reasonable/standard build systems and software with unique or ill-designed build processes.)
I've been daydreaming about how this could be better for a little while now, and wondering what compromises it will require to the ‘purity’ of the system, how worthwhile they'll be for novice and advanced users, etc.
I've been smitten by NixOS ever since I first tried it 2 years ago. Its the first carefree experience I've had with my system. As a side note, this result includes first tiny contributions of mine!
That said, I really hope the community gets their stuff together on Flakes. They're designated as experimental and official document doesn't recommend them, but all the cool kids are already using them everywhere and it is fracturing the efforts and alienating newcomers.
That said, I really hope the community gets their stuff together on Flakes. They're designated as experimental and official document doesn't recommend them, but all the cool kids are already using them everywhere and it is fracturing the efforts and alienating newcomers.
I really can't believe they didn't finalize Flakes in this release. The difference between it and the "standard" way of doing things is night-and-day.
these sort of "secret better ways of doing things" that are not the default really frustrate and drive away new users
these sort of "secret better ways of doing things" that are not the default really frustrate and drive away new users
By the way things are, I doubt they'll be stable in next release either. There were some pretty fancy things in last year's NixCon so devs might just push on that.
But the problem is, many many new shiny things are so comingles with flakes that they are now stuck in limbo, thanks to the one amazing feature that was showcased back in 2019 and then rotted away in implementation hell. Initial RFC was rushed, barely discussed (there are lots of comments, but very little progress, as is the way with flakes) and then simply merged as experimental akin to pushing mess under the carpet. Well the mess was useful, but now the carpet is uneven and the first timers can barely walk straight. I just want a conclusion on flakes, any conclusion would be better than this stupid silent march.
But the problem is, many many new shiny things are so comingles with flakes that they are now stuck in limbo, thanks to the one amazing feature that was showcased back in 2019 and then rotted away in implementation hell. Initial RFC was rushed, barely discussed (there are lots of comments, but very little progress, as is the way with flakes) and then simply merged as experimental akin to pushing mess under the carpet. Well the mess was useful, but now the carpet is uneven and the first timers can barely walk straight. I just want a conclusion on flakes, any conclusion would be better than this stupid silent march.
I wonder if there are any contentious design decisions left in flakes. I've still new to Nix, but it seems pretty solid (the only thing I've seen is that `nix flake info` is now `nix flake metadata` or something minor). The only missing thing I've seen is how to load a repl with flake contents easily, but I usually do:
nix-repl> builtins.getFlake (toString ./.)
when I'm in the flake directory. It would be nice to have `nix flake repl` as well.
nix-repl> builtins.getFlake (toString ./.)
when I'm in the flake directory. It would be nice to have `nix flake repl` as well.
As another learner, I'm pretty upset about this as well. Standard install gets you nix 2.3.12 (https://nixos.org/download.html), but to get flakes, you have to be on nix-2.4pre20210503_6d2553a:
https://nixos.wiki/wiki/Flakes#Non-NixOS
(And yeah, obviously all the docs are non-flake too, which leads to confusion about flake.nix vs default.nix...)
https://nixos.wiki/wiki/Flakes#Non-NixOS
(And yeah, obviously all the docs are non-flake too, which leads to confusion about flake.nix vs default.nix...)
The problem with this is that flakes are currently an experiment, and one which many people in the community think has flaws that need another iteration.
Making it stable now locks Nix into the current flakes model forever - it's too early for that.
Making it stable now locks Nix into the current flakes model forever - it's too early for that.
What part of flakes might be up for discussion?
The only quirky part I see is how one has to "rewire" inputs of your own inputs (e.g. nixpkgs from an input to one's current flake) via that `follows` directive, but overall it's been rather pleasant compared to channels!
The only quirky part I see is how one has to "rewire" inputs of your own inputs (e.g. nixpkgs from an input to one's current flake) via that `follows` directive, but overall it's been rather pleasant compared to channels!
Agree. I think the core of the problem was that we didn't get a discussion before implementation.
We got an implementation and then were given chance to discuss upon. So when the discussion pointed out flaws, since so much work was already put in, it was decided to push as experimental instead of starting from square 1. But now that implementation was available for everyone with simple flag change (a strength of Nix, if I might say), adventurers started using them. And since these adventurers have outsized impact in terms of mind share projection, the community looks divided and confuses the hell out of newcomers. I know I struggled with this because when I started learning 'The Nix Way™', flakes were the hotness and everywhere, but the documentation sucked, but still looked like they were the future so I had to wade my through anyway.
We got an implementation and then were given chance to discuss upon. So when the discussion pointed out flaws, since so much work was already put in, it was decided to push as experimental instead of starting from square 1. But now that implementation was available for everyone with simple flag change (a strength of Nix, if I might say), adventurers started using them. And since these adventurers have outsized impact in terms of mind share projection, the community looks divided and confuses the hell out of newcomers. I know I struggled with this because when I started learning 'The Nix Way™', flakes were the hotness and everywhere, but the documentation sucked, but still looked like they were the future so I had to wade my through anyway.
I'm glad we didn't yet stabilize because I am skeptical of a number of aspects to Nix. I suspect it's unduly popular because pure eval mode is a food idea, but there is little way to use it outside of flakes.
s/to nix/of flakes/, to be clear
what do you think of the strategy of stabilizing just the input schema for Nix flakes, which is all that's required for pure evaluation, then stabilizing each component of the output schema separately as those components are called for by specific features of the Nix/NixOS CLI?
That's a neat idea and improvement for sure. I'm pretty pessimistic about the outcome here, and perhaps what you said is a most realistic turn.
There is a larger issue that still irks me: all this stuff is based off Cargo/Cabal/etc., but I think that's fundamentally the wrong model for our purposes. Historically every C package is a snowflake and we do need to package them individually. But as language-based ecosystems take off, we increasingly can auto generate everything we need, and our Nix code is not the bricks but the mortar.
More concretely, the things I do with nix-thunk / Niv today don't really correspond to package boundaries that much, and if they do those repos might not contain any Nix code because I callCabal2nix them. callPackage-style "I am a function of my dependencies" is right, but when those dependency derivations don't align 1-1 with my nix-thunk / niv usage or need extra preprocessing à la cabal2nix, the fundamental concept of flakes breaks down.
There is a larger issue that still irks me: all this stuff is based off Cargo/Cabal/etc., but I think that's fundamentally the wrong model for our purposes. Historically every C package is a snowflake and we do need to package them individually. But as language-based ecosystems take off, we increasingly can auto generate everything we need, and our Nix code is not the bricks but the mortar.
More concretely, the things I do with nix-thunk / Niv today don't really correspond to package boundaries that much, and if they do those repos might not contain any Nix code because I callCabal2nix them. callPackage-style "I am a function of my dependencies" is right, but when those dependency derivations don't align 1-1 with my nix-thunk / niv usage or need extra preprocessing à la cabal2nix, the fundamental concept of flakes breaks down.
I agree with the general orientation in favor of the monorepo and transforming package definitions from repos that don't know/care about Nix, then aggregating them. I think the vision where we use Nix mostly by way of Nix code that has propagated into the source repos of various projects, and we simply use that code directly, probably gets it backwards.
> callPackage-style "I am a function of my dependencies" is right
What did you think of Eelco's presentation at the most recent NixCon where he considers defining packages as NixOS-like modules instead of bare functions?
> callPackage-style "I am a function of my dependencies" is right
What did you think of Eelco's presentation at the most recent NixCon where he considers defining packages as NixOS-like modules instead of bare functions?
The TOML one? That might be might be nice, but we should fix the global namespace issues of NixOS modules first. (Sometimes there are scarse global resources to multiplex access too, but that doesn't mean we should only be able to e.g. run only one postgres instance.)
I must admit, I'm curious about the idea but I don't yet "get" it.
Reproducible builds sound like a great idea at first, but this has the same problem as Docker containers: you inadvertently become a package/distro maintainer. How does one update the system regularly with security updates? Are we locking by semver somehow (like node, etc.)? And who decides which version ranges to lock to, etc. etc.?
Honestly, I'd probably much rather just take an existing Debian/Redhat/etc. LTS distro and do a custom install script, which isn't that hard in the first place.
And, in case of cloud server environments, you can just take a snapshot of your VM with a simple click of a button. You can't get much more reproducible than an actual snapshot of your entire system.
Reproducible builds sound like a great idea at first, but this has the same problem as Docker containers: you inadvertently become a package/distro maintainer. How does one update the system regularly with security updates? Are we locking by semver somehow (like node, etc.)? And who decides which version ranges to lock to, etc. etc.?
Honestly, I'd probably much rather just take an existing Debian/Redhat/etc. LTS distro and do a custom install script, which isn't that hard in the first place.
And, in case of cloud server environments, you can just take a snapshot of your VM with a simple click of a button. You can't get much more reproducible than an actual snapshot of your entire system.
Usually you have a declarative package/system config file that is seldom changes and you point it to a nixpkgs repository, which has definitions for other packages. So an update is as easy as changing that repo to a newer/modified branch. Of course you can even depend on multiple such repos, or override them in any way so you can use a specific lib with an older shared library while use everything else with the latest.
As for versioning, it is hash-based. Packages specify a source (eg github repo and a specific commit) and the hash of the downloaded source files, and build instructions inside the nixpkgs repo. The resulting program will depend on each listed property. And since they are referenced by hash, a program can use any possible version of another program. Hope it answers your questions!
As for versioning, it is hash-based. Packages specify a source (eg github repo and a specific commit) and the hash of the downloaded source files, and build instructions inside the nixpkgs repo. The resulting program will depend on each listed property. And since they are referenced by hash, a program can use any possible version of another program. Hope it answers your questions!
Within a release there are no version bumps, only if you live on unstable or switch to a new release you would get them.
The problem of an actual snapshot of your entire system is that you don't really now what should be in there and it collects a lot of cruft/state. With NixOS you can write little modules for services and move those between VM/Machines, whatever floats your boat. IMHO this type of config management sucks less than ansible/chef etc because you are not fighting against the package manager. The configuration holds the ground truth on what is installed on the system.
It's like an OS that has builtin salt/ansible/chef/puppet.
Because Nix language describe the OS instead of what to change/configure it's superior to these tools, and solves the problem that supposedly same machines are drifting apart.
For example if in the CM you tell it to install a package, then change your mind and remove the entry that does it. The package will remain installed.
With NixOS if you remove the package from configuration, it's gone.
I personally really like Nix's building capability. For example I can use it to generate a minimal docker container. It requires some knowledge, but I can also modify compilation options in dependencies (like remove unneeded functionality).
It looks like there's also an option to similarly build lightweight OS images[1]. I haven't tried it yet but looks cool.
[1] https://github.com/cleverca22/not-os
Because Nix language describe the OS instead of what to change/configure it's superior to these tools, and solves the problem that supposedly same machines are drifting apart.
For example if in the CM you tell it to install a package, then change your mind and remove the entry that does it. The package will remain installed.
With NixOS if you remove the package from configuration, it's gone.
I personally really like Nix's building capability. For example I can use it to generate a minimal docker container. It requires some knowledge, but I can also modify compilation options in dependencies (like remove unneeded functionality).
It looks like there's also an option to similarly build lightweight OS images[1]. I haven't tried it yet but looks cool.
[1] https://github.com/cleverca22/not-os
> but this has the same problem as Docker containers: you inadvertently become a package/distro maintainer. How does one update the system regularly with security updates? Are we locking by semver somehow (like node, etc.)? And who decides which version ranges to lock to, etc. etc.?
That is not true in the case of NixOS.
NixOS is an ordinary distribution: it has a release (which seems to be maintained for a year, receiving security patches) and an unstable branch.
So if you want to maintain your whole system in Nix, my recommendation is something like:
(1) use nix flakes. Flakes basically means you expressly specify the versions of your inputs, following the well known pattern of a human-edited but loosely specify source specification and a lockfile.
(2) One of your inputs will be github:NixOS/nixpkgs/release-21.05, which means the release-21.05 branch of github.com/NixOS/nixpkgs. The lockfile will ensure this follows the latest version
(3) In your flake, include your system configuration: it should build the whole system (systemd jobs, your reverse proxy, whatever it is you want each machine/class of machines in your network to do), not merely your application.
(4) In your flake, include your system tests. They can run on a virtual machine (or a network of virtual machines) whose configurations are derived from the configurations you created in (3).
If you've done that, you can now be reasonable confident that you can automatically receive security updates, run your tests against them, and deploy them.
That is not true in the case of NixOS.
NixOS is an ordinary distribution: it has a release (which seems to be maintained for a year, receiving security patches) and an unstable branch.
So if you want to maintain your whole system in Nix, my recommendation is something like:
(1) use nix flakes. Flakes basically means you expressly specify the versions of your inputs, following the well known pattern of a human-edited but loosely specify source specification and a lockfile.
(2) One of your inputs will be github:NixOS/nixpkgs/release-21.05, which means the release-21.05 branch of github.com/NixOS/nixpkgs. The lockfile will ensure this follows the latest version
(3) In your flake, include your system configuration: it should build the whole system (systemd jobs, your reverse proxy, whatever it is you want each machine/class of machines in your network to do), not merely your application.
(4) In your flake, include your system tests. They can run on a virtual machine (or a network of virtual machines) whose configurations are derived from the configurations you created in (3).
If you've done that, you can now be reasonable confident that you can automatically receive security updates, run your tests against them, and deploy them.
(which seems to be maintained for a year, receiving security patches)
Releases are maintained until one month after the next release. So, 21.05 will be maintained until a month after 21.11 is released.
Releases are maintained until one month after the next release. So, 21.05 will be maintained until a month after 21.11 is released.
What do experienced nix-ers tend to do when they need to use software that's not up to date or available in nixpkgs? Are you guys knowledgeable enough to package everything yourselves, or do you just use buildFHSUserEnv until someone else works out the kinks? I didn't get a chance to figure out how to do the latter, but I'm curious because I've struggled as a new user.
I fork the repo, update the package in my fork, add the fork as a flake to my inputs and test my changes. When I'm happy, I create a pull request to the main repo.
I started using NixOS in all my machines last winter, and slowly decided it is for me when I understood the basics. It was a rough start for me, but I have a few decades of linux knowledge, that helped me past the starting point.
You can read my config and see how to create overlays from different sources. E.g. if your fork updates vim, your overlay overwrites vim in the definition. Then referring to pkgs.vim points to your fork. See flake.nix in the repo.
https://github.com/pimeys/nixos
I started using NixOS in all my machines last winter, and slowly decided it is for me when I understood the basics. It was a rough start for me, but I have a few decades of linux knowledge, that helped me past the starting point.
You can read my config and see how to create overlays from different sources. E.g. if your fork updates vim, your overlay overwrites vim in the definition. Then referring to pkgs.vim points to your fork. See flake.nix in the repo.
https://github.com/pimeys/nixos
As a side note, thank you for posting this. I've been trying to get flakes set up on my system and this is more readable than the other dotfiles I've looked through.
You're welcome. It is still not perfect, and I hit my head to the wall enough when writing that. hlissner is my biggest influence here, but I simplified their code a lot so I can understand it better for now. My nix-fu is still not that high level.
Next thing I want to learn is how to package a complex js/rust package into a flake setup. I have the whole next friday to that.
Next thing I want to learn is how to package a complex js/rust package into a flake setup. I have the whole next friday to that.
I would just add that more often than not it is as easy as changing the version string in the somePackage/default.nix file and changing the corresponding hash of the source. Of course if the build changes in the new version or it depends on some new lib it can become a bit more difficult, but with a bit of experience it is not that hard to solve.
And to get the hash of some file you download, you can do `nix-prefetch-url https://us.download.nvidia.com/XFree86/Linux-x86_64/460.80/N...` that'll cache the file for you and give the sha256. Remember to add `--unpack` if the file is an archive.
I run [nix-update](https://github.com/Mic92/nix-update/) to update stuff. If you want to package something new, first identify the programming language/buildsystem the project is using and than look for a similar project in nixpkgs.
I typically try to package it and push it upstream. It usually isn't much work and it helps others. That being said it is rare that something I want isn't in nixpkgs. After all by some metrics nixpkgs-unstable is the most up to date package repository, and only slightly smaller than the AUR https://repology.org/repositories/graphs
In some cases I do settle for using `steam-run` or `buildFHSUserEnv` but I don't think I am using those for anything right now.
In some cases I do settle for using `steam-run` or `buildFHSUserEnv` but I don't think I am using those for anything right now.
We nixify all in-house development packages at work, it isn’t that bad once you get the hang of it, and comes with an awesome amount of ‘just works’ when integrating packages on custom devices down the road.
If you currently use something like Ansible to orchestrate changes (such as failover of a PostgreSQL node), how would you do this with Nix? Do you still use Ansible, or can you use NixOps for this?
In general Nix is a huge step above Ansible for config management. They aren't even playing the same game. Ansible is an awkward syntax to generate a shell script, NixOps is a declarative deployment tool. (I wrote more about this in [1] if you are interested).
However one place that NixOps doesn't handle well is state. So for example if you want to run state mutating commands such as boot read-only, kill replication, then start replication on the other node it is a bit tedious. You could do this with multiple states. For example generate the "readonly" state and deploy it, then generate the "failed over" state. However there isn't great support sequencing these types of things as far as I am aware. But also a disclaimer that I haven't actually been in the situation where I needed to manage PostgreSQL with Nix so I haven't looked too closely at the options. I would love to hear someone with experience to share what they do.
[1] https://kevincox.ca/2015/12/13/nixos-managed-system/
However one place that NixOps doesn't handle well is state. So for example if you want to run state mutating commands such as boot read-only, kill replication, then start replication on the other node it is a bit tedious. You could do this with multiple states. For example generate the "readonly" state and deploy it, then generate the "failed over" state. However there isn't great support sequencing these types of things as far as I am aware. But also a disclaimer that I haven't actually been in the situation where I needed to manage PostgreSQL with Nix so I haven't looked too closely at the options. I would love to hear someone with experience to share what they do.
[1] https://kevincox.ca/2015/12/13/nixos-managed-system/
Ahhhh, Nix/NixOS. The package manager/OS that I so desperately wanted to love.
I stuck with Nix for about six months. I even became a package maintainer.
In the end, I gave up for three reasons:
1. Documentation is really bad. It's often confusing, incomplete or just plain wrong. What drove me crazy is the extreme imprecision with which certain fundamental concepts are referred to. For example, "derivations" are among the basic building blocks of Nix's package management system, yet I counted at least three unrelated ways in which they're referred to within different parts of the documentation. There were also many times when the only way to solve routine problems was to get in contact with Nix's developers.
2. I never appreciated how much package management is intertwined with individual ecosystems' cultures until I tried using a "universal" package manager like Nix. Take Node packages. The norm is to install in a project's node_packages directory, and many packages rely on that idiom just to function (e.g., hardwired to import modules with that particular directory structure in mind). Nix attempts to translate each ecosystem's idioms into its own system by introducing many helper functions (all written in the Nix language, very few of which are documented) for each ecosystem. The problem is more fundamental than the fact that this approach is only partially successful and many packages remain subtly broken. The real problem is that there's no unified meta-approach or design language to rationalize how each ecosystem's idiosyncrasies are assimilated. The API's for each ecosystem's Nix helper functions are totally different. You end up as a "Nix Node package developer" just to do your job as a web developer, and then a "Nix Python package developer" just to add a Python package, and so on.
3. The developers don't seem to have any interest in solving these UX problems. They are quick to respond and welcoming people, but the impression I was left with is that they're mostly academics or just interested in whatever subsystem they're working on. For example, when I was working on Nix, the canonical mechanism for updating a single Node package in the Nix repository was to run a script that updated all (yes, all) Node packages in the repo. When I pointed out all the ways that this was an impractical approach, the response was that it's actually a good thing because that way the Node packages in the repo don't get get stale (!) This was one of a few examples I encountered where the devs didn't seem to be aware of how unusable Nix was in real world cases.
My mini-review is from the perspective of someone who was willing to put in the time to jump through all the hoops. I've left out obvious issues that would be a showstopper for many (e.g., obscurity of the Nix language, poor support on macOS). The issues that I've listed are "fit and finish" issues that at a glance may appear superficial, but having worked in the community for a while, I came to believe are more fundamental. They're about governance and what the community wants from the project. I don't believe that their goals are aligned with those of most devs who probably want a system that solves one of the biggest pains in tooling, package management.
It's a pity because Nix's design is really ingenious. Making a "functional" package manager in and of itself is a huge achievement. What I didn't appreciate when I started using Nix is how it normalizes the act of programming your package manager. It's like shifting from Heroku to CloudFront except with much worse docs. The idea is brilliant, the promise is there, but the team doesn't seem interested in turning the whole package from a project into a cohesive product.
I stuck with Nix for about six months. I even became a package maintainer.
In the end, I gave up for three reasons:
1. Documentation is really bad. It's often confusing, incomplete or just plain wrong. What drove me crazy is the extreme imprecision with which certain fundamental concepts are referred to. For example, "derivations" are among the basic building blocks of Nix's package management system, yet I counted at least three unrelated ways in which they're referred to within different parts of the documentation. There were also many times when the only way to solve routine problems was to get in contact with Nix's developers.
2. I never appreciated how much package management is intertwined with individual ecosystems' cultures until I tried using a "universal" package manager like Nix. Take Node packages. The norm is to install in a project's node_packages directory, and many packages rely on that idiom just to function (e.g., hardwired to import modules with that particular directory structure in mind). Nix attempts to translate each ecosystem's idioms into its own system by introducing many helper functions (all written in the Nix language, very few of which are documented) for each ecosystem. The problem is more fundamental than the fact that this approach is only partially successful and many packages remain subtly broken. The real problem is that there's no unified meta-approach or design language to rationalize how each ecosystem's idiosyncrasies are assimilated. The API's for each ecosystem's Nix helper functions are totally different. You end up as a "Nix Node package developer" just to do your job as a web developer, and then a "Nix Python package developer" just to add a Python package, and so on.
3. The developers don't seem to have any interest in solving these UX problems. They are quick to respond and welcoming people, but the impression I was left with is that they're mostly academics or just interested in whatever subsystem they're working on. For example, when I was working on Nix, the canonical mechanism for updating a single Node package in the Nix repository was to run a script that updated all (yes, all) Node packages in the repo. When I pointed out all the ways that this was an impractical approach, the response was that it's actually a good thing because that way the Node packages in the repo don't get get stale (!) This was one of a few examples I encountered where the devs didn't seem to be aware of how unusable Nix was in real world cases.
My mini-review is from the perspective of someone who was willing to put in the time to jump through all the hoops. I've left out obvious issues that would be a showstopper for many (e.g., obscurity of the Nix language, poor support on macOS). The issues that I've listed are "fit and finish" issues that at a glance may appear superficial, but having worked in the community for a while, I came to believe are more fundamental. They're about governance and what the community wants from the project. I don't believe that their goals are aligned with those of most devs who probably want a system that solves one of the biggest pains in tooling, package management.
It's a pity because Nix's design is really ingenious. Making a "functional" package manager in and of itself is a huge achievement. What I didn't appreciate when I started using Nix is how it normalizes the act of programming your package manager. It's like shifting from Heroku to CloudFront except with much worse docs. The idea is brilliant, the promise is there, but the team doesn't seem interested in turning the whole package from a project into a cohesive product.
> never appreciated how much package management is intertwined with individual ecosystems' cultures until I tried using a "universal" package manager like Nix.
Unfortunately it is not a problem for nix to solve, they can’t really mandate upstream projects to not include hardcoded executables.
Also, the problem with npm/pip etc is that they are themselves package managers more or less trying to solve the same thing. So I would even wager that without some language-specific help nix can’t really work seamlessly across other dependency managers. Also, it is probably not even the best idea to replicate npm’s every dep inside nixpkgs (there are third-party nix repos).
Also, as an example Haskell has it sort of solved with a specific program that can translate a haskell specific dependency specification (analogous to package.json) to a .nix build file. There may very well be something similar for npm as well.
Unfortunately it is not a problem for nix to solve, they can’t really mandate upstream projects to not include hardcoded executables.
Also, the problem with npm/pip etc is that they are themselves package managers more or less trying to solve the same thing. So I would even wager that without some language-specific help nix can’t really work seamlessly across other dependency managers. Also, it is probably not even the best idea to replicate npm’s every dep inside nixpkgs (there are third-party nix repos).
Also, as an example Haskell has it sort of solved with a specific program that can translate a haskell specific dependency specification (analogous to package.json) to a .nix build file. There may very well be something similar for npm as well.
>Also, the problem with npm/pip etc is that they are themselves package managers more or less trying to solve the same thing.
The problem is that they're trying to solve different problems. NPM/pip etc are mostly about letting the developer of an internal application choose and acquire dependencies for their project. Nixpkgs/dpkg etc are about letting the developers of a system acquire and maintain dependencies for a whole system.
These two goals pull in different directions.
NPM/pip want to have available every released version of every package, so that when I can keep using legacy-package with a well-tested version of web-dependency. This way, I can focus my attention on the important matters of writing new code for contemporary-project. I can be confident that legacy-package will keep running the same way it used to, security bugs and all, until I make a decision to attend to it.
Nixpkgs/dpkg want to have available as many packages as possible with a target that every api incompatible library is included, so that any two programs which use the same api use the same version of a package. This way, if they need to patch a well-tested but insecure version of web-dependency, all programs which use it will be kept up-to-date.
Because nix is an agnostic build tool, it can run in both modes simultaneously. You can create a nix expression that uses nixpkgs to build a system that provides a runtime environment and a system that provides a buildtime environment and then uses internal project dependency managers to fulfil the lockfile and installs the product of that, node_modules and all, onto the runtime system. You will lose certain guarantees by doing it this way, and perhaps some parts will require awkward, but that is an engineering decision. For instance, nix won't be able to verify that you aren't depending on changeable remote systems, because you'll have chosen to do that: Nix doesn't trust that npm is add-only, but you might. (Also, some aspects of this will be poorly documented. That's a fixable problem, but the maintainers of a package distribution are probably less interested in the case that optimises for concerns they've chosen not to optimise for, so the documentation for working that way will naturally be worse.)
The problem is that they're trying to solve different problems. NPM/pip etc are mostly about letting the developer of an internal application choose and acquire dependencies for their project. Nixpkgs/dpkg etc are about letting the developers of a system acquire and maintain dependencies for a whole system.
These two goals pull in different directions.
NPM/pip want to have available every released version of every package, so that when I can keep using legacy-package with a well-tested version of web-dependency. This way, I can focus my attention on the important matters of writing new code for contemporary-project. I can be confident that legacy-package will keep running the same way it used to, security bugs and all, until I make a decision to attend to it.
Nixpkgs/dpkg want to have available as many packages as possible with a target that every api incompatible library is included, so that any two programs which use the same api use the same version of a package. This way, if they need to patch a well-tested but insecure version of web-dependency, all programs which use it will be kept up-to-date.
Because nix is an agnostic build tool, it can run in both modes simultaneously. You can create a nix expression that uses nixpkgs to build a system that provides a runtime environment and a system that provides a buildtime environment and then uses internal project dependency managers to fulfil the lockfile and installs the product of that, node_modules and all, onto the runtime system. You will lose certain guarantees by doing it this way, and perhaps some parts will require awkward, but that is an engineering decision. For instance, nix won't be able to verify that you aren't depending on changeable remote systems, because you'll have chosen to do that: Nix doesn't trust that npm is add-only, but you might. (Also, some aspects of this will be poorly documented. That's a fixable problem, but the maintainers of a package distribution are probably less interested in the case that optimises for concerns they've chosen not to optimise for, so the documentation for working that way will naturally be worse.)
> Unfortunately it is not a problem for nix to solve, they can’t really mandate upstream projects to not include hardcoded executables.
Yet it's normal for Linux distros to patch upstream so that it works well inside of the distro. You may say "but that's crazy! Downstream patching Node packages opens up a whole can of worms!" Maybe you would be right--or maybe not. My point is that when I would raise these kinds of issues, I was met with a collective shrug or various justifications for why brokenness is actually a good thing.
That's what I call a kind of academic/purist mentality that's really great for a research project and not practical for a product. If you ship a Linux distro (and note, I'm talking here about NixOS, not just Nix) that comes with software in the repo, the universal expectation is that the software works as intended, and if it doesn't that it's a bug. There can be an argument over whether the bug is upstream or downstream, but the fault is recognized. That recognition doesn't seem to happen in Nix world in cases where the package comes from an ecosystem that has assumptions that contradict Nix's (or NixOS') assumptions.
> So I would even wager that without some language-specific help nix can’t really work seamlessly across other dependency managers.
The thing is, Nix provides various helper functions to aid in writing derivations for packages from the different ecosystems. They actually try to adapt to the various idioms, which is admirable.
My gripe is that each set of adaptations for each different ecosystem has its own entirely unique API. There's no meta-framework or set of common practices for writing makeNodePackage vs. makePythonPackage vs. makeHaskellPackage even though in many cases they share similar challenges (e.g., X and Y ecosystem's packages import dependencies from hardwired relative paths). It's a question of abstraction; makeXYZPackage exists, but there's no supra-ecosystem abstraction for writing it.
That sounds like an obscure concern until you try using Nix as your daily driver for dev work. As long as you stick to one ecosystem, it's passable. Still a bad UX because nothing is properly documented and really bad anti-patterns (such as forcing package maintainers to update hundreds of thousands of packages to update a single package) are championed, but at least you only need to learn one ecosystem's quirks and idiosyncrasies. The situation changes very quickly as soon as you try to mesh NPM packages with Python with Ruby, etc.
I actually raised this concern a while ago with the Nix devs, and their response was basically that it would take too much effort to harmonize all the different approaches. I thought that was a valid response, but it's unfortunate since I think it's the project's Achilles' heel.
Yet it's normal for Linux distros to patch upstream so that it works well inside of the distro. You may say "but that's crazy! Downstream patching Node packages opens up a whole can of worms!" Maybe you would be right--or maybe not. My point is that when I would raise these kinds of issues, I was met with a collective shrug or various justifications for why brokenness is actually a good thing.
That's what I call a kind of academic/purist mentality that's really great for a research project and not practical for a product. If you ship a Linux distro (and note, I'm talking here about NixOS, not just Nix) that comes with software in the repo, the universal expectation is that the software works as intended, and if it doesn't that it's a bug. There can be an argument over whether the bug is upstream or downstream, but the fault is recognized. That recognition doesn't seem to happen in Nix world in cases where the package comes from an ecosystem that has assumptions that contradict Nix's (or NixOS') assumptions.
> So I would even wager that without some language-specific help nix can’t really work seamlessly across other dependency managers.
The thing is, Nix provides various helper functions to aid in writing derivations for packages from the different ecosystems. They actually try to adapt to the various idioms, which is admirable.
My gripe is that each set of adaptations for each different ecosystem has its own entirely unique API. There's no meta-framework or set of common practices for writing makeNodePackage vs. makePythonPackage vs. makeHaskellPackage even though in many cases they share similar challenges (e.g., X and Y ecosystem's packages import dependencies from hardwired relative paths). It's a question of abstraction; makeXYZPackage exists, but there's no supra-ecosystem abstraction for writing it.
That sounds like an obscure concern until you try using Nix as your daily driver for dev work. As long as you stick to one ecosystem, it's passable. Still a bad UX because nothing is properly documented and really bad anti-patterns (such as forcing package maintainers to update hundreds of thousands of packages to update a single package) are championed, but at least you only need to learn one ecosystem's quirks and idiosyncrasies. The situation changes very quickly as soon as you try to mesh NPM packages with Python with Ruby, etc.
I actually raised this concern a while ago with the Nix devs, and their response was basically that it would take too much effort to harmonize all the different approaches. I thought that was a valid response, but it's unfortunate since I think it's the project's Achilles' heel.
Thanks for expanding on it!
I feel the all-platform abstraction would be very difficult and leaky but at least some minor convention would indeed be great. Though I’m not sure how perfect the nix language is for very high-level abstractions —- I think the documentation problem is also in part due to not having static types.
I feel the all-platform abstraction would be very difficult and leaky but at least some minor convention would indeed be great. Though I’m not sure how perfect the nix language is for very high-level abstractions —- I think the documentation problem is also in part due to not having static types.
I ran Nixos for a few months from Dec 25 to Mar 12 of this year. During that time there were 2 zero-day exploits published for Chrome that went unpatched for weeks on Nixos. (Specifically the package named google-chrome went unpatched.)
This was on the "unstable" branch of Nixos, the branch that became the stable branch today, which is the occasion of this story on HN we are commenting on.
Digression: the reason I chose the unstable branch instead of the then-stable branch (named Nixos-20.09) was that I wanted to achieve an "all-Wayland" environment as quickly as practical (and in fact all of the apps running on my machine right now and for the past few weeks talk directly to Wayland without the interposition of Xwayland). And the reason I wanted to jump ahead to an all-Wayland system is that although I knew Linux pretty well, I knew very little about X11, and I reasoned that since X11 is slowly going away anyways, I would prefer to avoid spending any time learning about it and would prefer to avoid having to get used to its quirks. (End of digression.)
You cannot just install Chrome from Google's web site on Nixos: unless someone has compiled it for you, you have to compile it from source and more distressingly you have to configure the compilation extensively to have any hope of the resulting binary actually running. For example, glibc on Nixos is not in a standard location, but rather all of the files in the glibc package are under a directory with a name like /nix/store/8f9f6724fd2341d9f2bc758cf9e43830d23d37a3-glibc. Well, of course the code that compiles Chrome needs to be informed of that location and of the locations of all of the other packages it relies on, e.g., the system's libraries for drawing GUIs on the screen. This job of configuring the compilation is mostly easily done by writing something similar to a declarative build script in the Nix language, so of course that means I would've had to learn a new programming language in order to patch the zero-day exploit in Chrome before the maintainer of the google-chrome package got around to it (which I never did).
Sadly, I did not have time back then in early March to install the stable version of Nixos to determine whether these zero-day exploits had remained unpatched there, too. (I did verify that the package "chromium" was at the same vulnerable version as the package "google-chrome" was.) So I guess the purpose of this comment is that there are preliminary signs (namely, the fact that nowhere was I warned that the unstable branch might have gaping security holes) that Nixos does not get as much attention from security-knowledgeable maintainers as some of the other Linux distros do -- or at least that the security attention it gets is concentrated on server use cases with desktop use cases being relatively neglected.
This was on the "unstable" branch of Nixos, the branch that became the stable branch today, which is the occasion of this story on HN we are commenting on.
Digression: the reason I chose the unstable branch instead of the then-stable branch (named Nixos-20.09) was that I wanted to achieve an "all-Wayland" environment as quickly as practical (and in fact all of the apps running on my machine right now and for the past few weeks talk directly to Wayland without the interposition of Xwayland). And the reason I wanted to jump ahead to an all-Wayland system is that although I knew Linux pretty well, I knew very little about X11, and I reasoned that since X11 is slowly going away anyways, I would prefer to avoid spending any time learning about it and would prefer to avoid having to get used to its quirks. (End of digression.)
You cannot just install Chrome from Google's web site on Nixos: unless someone has compiled it for you, you have to compile it from source and more distressingly you have to configure the compilation extensively to have any hope of the resulting binary actually running. For example, glibc on Nixos is not in a standard location, but rather all of the files in the glibc package are under a directory with a name like /nix/store/8f9f6724fd2341d9f2bc758cf9e43830d23d37a3-glibc. Well, of course the code that compiles Chrome needs to be informed of that location and of the locations of all of the other packages it relies on, e.g., the system's libraries for drawing GUIs on the screen. This job of configuring the compilation is mostly easily done by writing something similar to a declarative build script in the Nix language, so of course that means I would've had to learn a new programming language in order to patch the zero-day exploit in Chrome before the maintainer of the google-chrome package got around to it (which I never did).
Sadly, I did not have time back then in early March to install the stable version of Nixos to determine whether these zero-day exploits had remained unpatched there, too. (I did verify that the package "chromium" was at the same vulnerable version as the package "google-chrome" was.) So I guess the purpose of this comment is that there are preliminary signs (namely, the fact that nowhere was I warned that the unstable branch might have gaping security holes) that Nixos does not get as much attention from security-knowledgeable maintainers as some of the other Linux distros do -- or at least that the security attention it gets is concentrated on server use cases with desktop use cases being relatively neglected.
For anyone willing to help the Nix project, there's a script[1] you can use in the Nixpkgs repo to update the official Nix package for Chrome. If you ever feel that Chrome needs to be updated immediately, you can help out by running this script and opening a PR.
[1]: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...
---
Now, back to point, security issues are not "neglected" in NixOS, and most of the points made are simply not true. Security vulnerabilities are rounded up on a regular basis. However, like many open source projects, Nix is mostly being worked on by volunteers and I don't believe it's fair to spread doubts just because they don't get fixed in a day or two.
Let me also clarify the remaining points.
First, the parent comment alleges that the Chrome package in Nixpkgs has been neglected. This is not the case. Looking at the commit logs[2], the Chrome package has been updated on a frequent basis "from Dec 25 to Mar 12 of this year."
Second, the parent comment argues that installing your preferred version of Chrome is "distressingly" impossible. Again, this is false. Installing your preferred version of Chrome using Nix is as simple as supplying an alternative upstream URL for the google-chrome package. This happens to be one of the major selling point of Nix, how it makes it easier to reuse and customize existing package definitions without friction.
[2]: https://github.com/NixOS/nixpkgs/commits/master/pkgs/applica...
[1]: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...
---
Now, back to point, security issues are not "neglected" in NixOS, and most of the points made are simply not true. Security vulnerabilities are rounded up on a regular basis. However, like many open source projects, Nix is mostly being worked on by volunteers and I don't believe it's fair to spread doubts just because they don't get fixed in a day or two.
Let me also clarify the remaining points.
First, the parent comment alleges that the Chrome package in Nixpkgs has been neglected. This is not the case. Looking at the commit logs[2], the Chrome package has been updated on a frequent basis "from Dec 25 to Mar 12 of this year."
Second, the parent comment argues that installing your preferred version of Chrome is "distressingly" impossible. Again, this is false. Installing your preferred version of Chrome using Nix is as simple as supplying an alternative upstream URL for the google-chrome package. This happens to be one of the major selling point of Nix, how it makes it easier to reuse and customize existing package definitions without friction.
[2]: https://github.com/NixOS/nixpkgs/commits/master/pkgs/applica...
>I don't believe it's fair to spread doubts
I think it is fair as long as I make it clear (which I hope I have) that I didn't investigate fully after I saw some warning signs.
>spread doubts just because they don't get fixed in a day or two.
I am not a security expert, but tptacek is, and in 2017 he wrote that "You don't want your browser to be any number of days behind the Chromium patch cycle."[1] IIRC, tptacek or someone of similar credibility said that the browser and the kernel are the primary targets of zero-days exploits on a desktop Linux install.
[1] https://news.ycombinator.com/item?id=13517227
Also let's see how another party handles patches for Chrome: the Arch User Repository's Chrome package (not an official part of Arch Linux) "will automatically get updated as soon as the Debian package is available. This is checked at least once per hour."[2]
There are many many things a distro needs to get right to compete with, e.g., Debian or Fedora on all the criteria desktop Linux users care about. It is not easy. My thanks to Nixos's maintainers for investing their time in trying to advance the state of the art. But Linux users invest a lot of time, too, and in chiming in today I did so because I thought I could save Linux users some time by presenting my conclusions after spending many 10s of hours installing and exploring Nixos.
[2] https://aur.archlinux.org/packages/google-chrome/
>Installing your preferred version of Chrome using Nix is as simple as supplying an alternative upstream URL for the google-chrome package.
Please correct me if I am wrong: this requires building Chrome from source, which will spike the user's CPUs for hours and is impossible with less than 8 megs of RAM ("More than 16GB is highly recommended")[3]
[3] https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...
I think it is fair as long as I make it clear (which I hope I have) that I didn't investigate fully after I saw some warning signs.
>spread doubts just because they don't get fixed in a day or two.
I am not a security expert, but tptacek is, and in 2017 he wrote that "You don't want your browser to be any number of days behind the Chromium patch cycle."[1] IIRC, tptacek or someone of similar credibility said that the browser and the kernel are the primary targets of zero-days exploits on a desktop Linux install.
[1] https://news.ycombinator.com/item?id=13517227
Also let's see how another party handles patches for Chrome: the Arch User Repository's Chrome package (not an official part of Arch Linux) "will automatically get updated as soon as the Debian package is available. This is checked at least once per hour."[2]
There are many many things a distro needs to get right to compete with, e.g., Debian or Fedora on all the criteria desktop Linux users care about. It is not easy. My thanks to Nixos's maintainers for investing their time in trying to advance the state of the art. But Linux users invest a lot of time, too, and in chiming in today I did so because I thought I could save Linux users some time by presenting my conclusions after spending many 10s of hours installing and exploring Nixos.
[2] https://aur.archlinux.org/packages/google-chrome/
>Installing your preferred version of Chrome using Nix is as simple as supplying an alternative upstream URL for the google-chrome package.
Please correct me if I am wrong: this requires building Chrome from source, which will spike the user's CPUs for hours and is impossible with less than 8 megs of RAM ("More than 16GB is highly recommended")[3]
[3] https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...
That's wrong. The google-chrome packages in nixpkgs are "built" by downloading the official releases (in .deb format I think) and massaging them a little. It takes a few seconds.
When the zero-days came out, I just updated the upstream-info.json file in my local nixpkgs checkout (look at previous chrome version bump commits for how to do it) and rebuilt my system. I didn't have to wait for any PRs or merges or CI/hydra. Granted, it requires a little nix knowledge, but if you've been running nixos you can probably figure out how to do it in a few minutes.
When the zero-days came out, I just updated the upstream-info.json file in my local nixpkgs checkout (look at previous chrome version bump commits for how to do it) and rebuilt my system. I didn't have to wait for any PRs or merges or CI/hydra. Granted, it requires a little nix knowledge, but if you've been running nixos you can probably figure out how to do it in a few minutes.
For anyone reading along, if you visit https://www.google.com/chrome/ from a Linux box, then click the big blue "Download" button, you are offered the choice of a .deb or an .rpm. I take it that that is what parent means by "the official releases".
>massaging them a little. It takes a few seconds.
Interesting and surprising. When I run ldd on my Chrome binary (which comes straight from Google, not my distro), it lists 101 dynamically-linked libraries. I believe that that means that the binary contains references (i.e., file names) to those 101 libraries. Does the massaging process just replace those references (file names) with the appropriate names beginning with "/nix/store/"? I.e., does the massaging process edit the binary?
(If so, I'm a little surprised, what with how paranoid Google is, that Chrome doesn't refuse to start because the binary has a different checksum!)
>massaging them a little. It takes a few seconds.
Interesting and surprising. When I run ldd on my Chrome binary (which comes straight from Google, not my distro), it lists 101 dynamically-linked libraries. I believe that that means that the binary contains references (i.e., file names) to those 101 libraries. Does the massaging process just replace those references (file names) with the appropriate names beginning with "/nix/store/"? I.e., does the massaging process edit the binary?
(If so, I'm a little surprised, what with how paranoid Google is, that Chrome doesn't refuse to start because the binary has a different checksum!)
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...
I believe this is the package you are talking about. The significant part is basically bash, so it should be readable without nix knowledge as well. And yeah it basically creates both a wrapper for setting some env variables, but it also replaces some dynamic lib locations with patchelf. So apparently it does startup with a different checksum/validation happens at another level.
I believe this is the package you are talking about. The significant part is basically bash, so it should be readable without nix knowledge as well. And yeah it basically creates both a wrapper for setting some env variables, but it also replaces some dynamic lib locations with patchelf. So apparently it does startup with a different checksum/validation happens at another level.
Yes, exactly: https://github.com/NixOS/patchelf
It's a key part of how nix works.
It's a key part of how nix works.
The claim that Chrome was not updated on the unstable branch for weeks is wrong.
The two releases in question are:
88.0.4324.96 on Jan 19: https://chromereleases.googleblog.com/2021/01/stable-channel...
88.0.4324.146 on Feb 2: https://chromereleases.googleblog.com/2021/02/stable-channel...
The commits updating them in nixpkgs are:
https://github.com/NixOS/nixpkgs/commit/5b6d3c4b13e50540d044...
https://github.com/NixOS/nixpkgs/commit/4fe69d33aeb6dac647e4...
The first landed in master on Jan 19 (the same day), the second landed in master on Feb 3 (one day late).
The way nix works, the CI has to build and verify a large chunk of packages before it gets promoted to unstable. So when did they hit nixos-unstable? It's not possible to tell from git alone, but the history of the channel is archived here: https://channels.nix.gsc.io/
I did the search and the first release hit nixos-unstable on Jan 21 18:20:45 (UTC), and the second on Feb 6 08:30:17 (UTC). So around two days and four days.
The backports to nixos-20.09 hit that branch on Jan 26 16:55:17 and Feb 6 13:45:20, so seven days and four days.
I'm not sure what the parent commentor was doing that led to them missing these updates.
To be clear, this is not a great record. I was running nixos-20.09 and I updated my system ahead of the channels (which is fairly easy, not at all difficult as the parent comment implies). But it's not "weeks", and it's not that bad for a volunteer project.
The two releases in question are:
88.0.4324.96 on Jan 19: https://chromereleases.googleblog.com/2021/01/stable-channel...
88.0.4324.146 on Feb 2: https://chromereleases.googleblog.com/2021/02/stable-channel...
The commits updating them in nixpkgs are:
https://github.com/NixOS/nixpkgs/commit/5b6d3c4b13e50540d044...
https://github.com/NixOS/nixpkgs/commit/4fe69d33aeb6dac647e4...
The first landed in master on Jan 19 (the same day), the second landed in master on Feb 3 (one day late).
The way nix works, the CI has to build and verify a large chunk of packages before it gets promoted to unstable. So when did they hit nixos-unstable? It's not possible to tell from git alone, but the history of the channel is archived here: https://channels.nix.gsc.io/
I did the search and the first release hit nixos-unstable on Jan 21 18:20:45 (UTC), and the second on Feb 6 08:30:17 (UTC). So around two days and four days.
The backports to nixos-20.09 hit that branch on Jan 26 16:55:17 and Feb 6 13:45:20, so seven days and four days.
I'm not sure what the parent commentor was doing that led to them missing these updates.
To be clear, this is not a great record. I was running nixos-20.09 and I updated my system ahead of the channels (which is fairly easy, not at all difficult as the parent comment implies). But it's not "weeks", and it's not that bad for a volunteer project.
I appreciate your digging into that.
The interval of time during which I was most concerned about unpatched holes in Nixos-unstable's google-chrome was the days leading up to Mar 12.
The interval of time during which I was most concerned about unpatched holes in Nixos-unstable's google-chrome was the days leading up to Mar 12.
Chrome stable 89.0.4389.90 was released on March 12: https://chromereleases.googleblog.com/2021/03/stable-channel...
Version bump in nixpkgs: https://github.com/NixOS/nixpkgs/commit/5f8b95113983c8f31d63... on March 13 (one day later)
Released in nixos-unstable on Mar 16 11:11:22 UTC (four days later)
Backport to 20.09: https://github.com/NixOS/nixpkgs/commit/26ba8cd77b5a4408799f... on March 13 (one day later)
Released in nixos-20.09 on Mar 16 16:31:31 UTC (four days later)
Again, four days is not great for a Chrome zero-day. NixOS doesn't have a professional security team, and if you need that assurance, maybe you can't use it. But please don't exaggerate and muddy the facts.
Most of the delay is due to hydra having to build everything that came in ahead of that change on the master branch, and sometimes the master branch is just broken.... I wish there was a fast-track process that could bump urgent security changes ahead of other ones, but it seems like it could complicate things a lot.
Of course, as I said above, it's relatively easy for you to update your local install without waiting for all that, if you're aware of the release and its severity.
Version bump in nixpkgs: https://github.com/NixOS/nixpkgs/commit/5f8b95113983c8f31d63... on March 13 (one day later)
Released in nixos-unstable on Mar 16 11:11:22 UTC (four days later)
Backport to 20.09: https://github.com/NixOS/nixpkgs/commit/26ba8cd77b5a4408799f... on March 13 (one day later)
Released in nixos-20.09 on Mar 16 16:31:31 UTC (four days later)
Again, four days is not great for a Chrome zero-day. NixOS doesn't have a professional security team, and if you need that assurance, maybe you can't use it. But please don't exaggerate and muddy the facts.
Most of the delay is due to hydra having to build everything that came in ahead of that change on the master branch, and sometimes the master branch is just broken.... I wish there was a fast-track process that could bump urgent security changes ahead of other ones, but it seems like it could complicate things a lot.
Of course, as I said above, it's relatively easy for you to update your local install without waiting for all that, if you're aware of the release and its severity.
I found the nix pills to be a good starting point for learning about how to use nixpkgs/nixos https://nixos.org/guides/nix-pills/index.html
What are people's thoughts on running nix without nixos, using a base system such as Debian or Arch?
It seems like it might be a nice way to get reproducible dev environments, without some of the headaches being discussed here.
Is this a common use, or do people typically just do all-or-nothing?
It seems like it might be a nice way to get reproducible dev environments, without some of the headaches being discussed here.
Is this a common use, or do people typically just do all-or-nothing?
That's how I started out with openSUSE + Nix, but I quickly found that I still had to learn about most of Nix's quirks, while only getting a small part of the benefits (namely, not the system configuration).
It may be worth it if you care a lot about having imperative escape hatches, but if you're willing to invest upfront work in getting past the learning curve and want to maximize the benefit you get from it, I would just recommend diving into NixOS directly.
It may be worth it if you care a lot about having imperative escape hatches, but if you're willing to invest upfront work in getting past the learning curve and want to maximize the benefit you get from it, I would just recommend diving into NixOS directly.
I would say that it is a good gateway drug. I used Nix + home-manager on macOS for a while. It was really nice for replacing the broken macOS userland plus having reproducible dev environments.
I liked it so much that I tried NixOS on a Dell Workstation that I still had. Configuring a full machine declaratively was so powerful that I slowly moved from macOS to NixOS pretty much full-time.
I liked it so much that I tried NixOS on a Dell Workstation that I still had. Configuring a full machine declaratively was so powerful that I slowly moved from macOS to NixOS pretty much full-time.
If I wanted to use nixpkgs on macOS instead of my dotfiles manager (yadm) and Brewfile – I hear you can do that – would I have to expect missing software (vs. homebrew)? Just about how much pain should I expect, being completely new to NixOS?
There’s a couple of absent macOS-specific packages, compared to homebrew. So I have `trash` and `pngpaste` declaratively compiled-from-github in my nix config, but that's the only two I'm missing compared to brew. (That said, I never dealt much with brew-cask, so if you have a host of GUI packages from that, you may be missing more)
If you're on an M1 Mac it will use x86 binaries by default, running under Rosetta. The situation is improving - I use almost all of my packages in native aarm64-darwin - but it still requires using bleeding-edge code.
The learning curve has been pretty brutal, but I really like it for both managing my system-wide (well, user-wide) software, and for pinning to explicit package versions in a large ruby/javascript project.
If you're on an M1 Mac it will use x86 binaries by default, running under Rosetta. The situation is improving - I use almost all of my packages in native aarm64-darwin - but it still requires using bleeding-edge code.
The learning curve has been pretty brutal, but I really like it for both managing my system-wide (well, user-wide) software, and for pinning to explicit package versions in a large ruby/javascript project.
I do that for my work machine which is a MacBook Pro.
You still need to install most GUI apps through the normal non-Nix procedure.
You also won't be able to use the Git module in home manager since that will use a version of Git that doesn't understand some MacOS specific customizations, mostly surrounding Keychain if i recall correctly.
Most packages just work on MacOS but it's not a first class citizen and there are always more packages broken for Darwin than Linux.
But I'll say it again, I tend to dabble in plenty of programming languages and often run software that's a bit off the beaten path and Home Manager and Nix on MacOS work 95% of the time for me.
You still need to install most GUI apps through the normal non-Nix procedure.
You also won't be able to use the Git module in home manager since that will use a version of Git that doesn't understand some MacOS specific customizations, mostly surrounding Keychain if i recall correctly.
Most packages just work on MacOS but it's not a first class citizen and there are always more packages broken for Darwin than Linux.
But I'll say it again, I tend to dabble in plenty of programming languages and often run software that's a bit off the beaten path and Home Manager and Nix on MacOS work 95% of the time for me.
Nix has over 80k packages now, so odds are what you need is there. But you should verify everything you need is available before switching. Make a list of everything you need and search the package repository for it:
https://search.nixos.org/packages
https://search.nixos.org/packages
A teeny-tiny orientation for (prospective?) Nix newcomers:
Having spent years on Arch and Gentoo, imo the NixOS community is outstanding even among 'advanced' Linux distros in terms of expertise and kindness.
I'll never forget one time I was having problems with a Python package generation tool and when I mentioned it on IRC, and one of its authors (Rok Garbas, who is so quite active in the community) just hopped on a call to debug the issue I was having right then and there.
If you're on the fence about Nix and that kind of community is valuable to you, make sure to check out the forums [1] and realtime chat [2] (the latter of which is unfortunately still in the process [3] of migrating away from Freenode).
If personal assistance/mentorship of the kind I described at the beginning of this post is appealing to you, several generous and knowledgeable community members host Nix ‘office hours’ [4]. I'm not sure who all are still running them, but I know tomberek is for sure [5] and you can find a link in the footnotes.
There is one serious problem with NixOS and the wider ecosystem right now, namely that the best experience available depends on Nix features that are unreleased. Nix makes it very easy to run bleeding edge builds of Nix and enable these features, but the unofficial status of it all has slowed adoption, integration, and documentation of these tools in the wider community.
That aside— i.e., if you're willing to be a little bit of a pioneer— the community is on the whole pretty rich with good documentation of everything but the bleeding edge, unreleased bits at this point. Those pieces mostly impact the CLI and specifying inputs to the expressions you use to define your system. Everything in the NixOS and Nixpkgs manuals is still accurate. The unreleased components are also fairly mature despite their unofficial status: many folks in the community have been using them for a year or two now.
Finally: if you're interested in dipping your toes in without committing to a system fully managed by Nix and you're a macOS user, nix-darwin [6] provides a pretty NixOS-like experience using a module system and CLI based on NixOS'. There's nothing equally complete in terms of managing system services in a declarative fashion on non-NixOS Linux, but home-manager [7] provides some functionality for enabling user-mode services in a declarative style.
You can check whether your favorite software is packaged for Nix here [8], and additionally NixOS does support several other forms of cross-distro packaging/deployment, including Flatpak, AppImage, and Docker. Steam support is native and works without much fuss, too.
---------
1: https://discourse.nixos.org/
2: https://matrix.to/#/!MKvhXlSTLGJUXpYuWF:nixos.org
3: https://github.com/NixOS/rfcs/pull/94
4: https://discourse.nixos.org/search?q=office%20hours
5: https://discourse.nixos.org/t/nix-office-hours/11945
6: https://github.com/LnL7/nix-darwin
7: https://github.com/nix-community/home-manager
8: https://search.nixos.org/packages
Having spent years on Arch and Gentoo, imo the NixOS community is outstanding even among 'advanced' Linux distros in terms of expertise and kindness.
I'll never forget one time I was having problems with a Python package generation tool and when I mentioned it on IRC, and one of its authors (Rok Garbas, who is so quite active in the community) just hopped on a call to debug the issue I was having right then and there.
If you're on the fence about Nix and that kind of community is valuable to you, make sure to check out the forums [1] and realtime chat [2] (the latter of which is unfortunately still in the process [3] of migrating away from Freenode).
If personal assistance/mentorship of the kind I described at the beginning of this post is appealing to you, several generous and knowledgeable community members host Nix ‘office hours’ [4]. I'm not sure who all are still running them, but I know tomberek is for sure [5] and you can find a link in the footnotes.
There is one serious problem with NixOS and the wider ecosystem right now, namely that the best experience available depends on Nix features that are unreleased. Nix makes it very easy to run bleeding edge builds of Nix and enable these features, but the unofficial status of it all has slowed adoption, integration, and documentation of these tools in the wider community.
That aside— i.e., if you're willing to be a little bit of a pioneer— the community is on the whole pretty rich with good documentation of everything but the bleeding edge, unreleased bits at this point. Those pieces mostly impact the CLI and specifying inputs to the expressions you use to define your system. Everything in the NixOS and Nixpkgs manuals is still accurate. The unreleased components are also fairly mature despite their unofficial status: many folks in the community have been using them for a year or two now.
Finally: if you're interested in dipping your toes in without committing to a system fully managed by Nix and you're a macOS user, nix-darwin [6] provides a pretty NixOS-like experience using a module system and CLI based on NixOS'. There's nothing equally complete in terms of managing system services in a declarative fashion on non-NixOS Linux, but home-manager [7] provides some functionality for enabling user-mode services in a declarative style.
You can check whether your favorite software is packaged for Nix here [8], and additionally NixOS does support several other forms of cross-distro packaging/deployment, including Flatpak, AppImage, and Docker. Steam support is native and works without much fuss, too.
---------
1: https://discourse.nixos.org/
2: https://matrix.to/#/!MKvhXlSTLGJUXpYuWF:nixos.org
3: https://github.com/NixOS/rfcs/pull/94
4: https://discourse.nixos.org/search?q=office%20hours
5: https://discourse.nixos.org/t/nix-office-hours/11945
6: https://github.com/LnL7/nix-darwin
7: https://github.com/nix-community/home-manager
8: https://search.nixos.org/packages
How is managing python packages or ruby gems on NixOS? I remember the experience was not polished a few years ago.
In general it is very nice. A common method is you create a `default.nix` file in the project you are working on and use tools that manage the deps for you. For example:
Rust+Cargo: https://gitlab.com/kevincox/watchlog/-/blob/22c877065f763b3d...
Node+NPM: https://gitlab.com/kevincox/kevincox-web-compiler/-/blob/9fa...
My only Ruby project is private but I just rolled my own with:
And sorry, but I don't do much Python.
For development you then use nix-shell to build the environment with all of the dependencies available. The downside is that you either need to commit the default.nix to the project, or configure it to be ignored. But the advantage is that it handles everything for you. For example you can configure native libraries such as PostgreSQL client or openssl without worrying if they are installed on your system. This means that the entire dev environment is tested in CI (assuming that you run nix-build in CI).
This is also nicer than docker/VM based dev environments as it just prefixes your usual environment with the deps, so you still have your regular commands and development tools available.
Rust+Cargo: https://gitlab.com/kevincox/watchlog/-/blob/22c877065f763b3d...
Node+NPM: https://gitlab.com/kevincox/kevincox-web-compiler/-/blob/9fa...
My only Ruby project is private but I just rolled my own with:
export "GEM_HOME=$out"
bundle install --jobs $NIX_BUILD_CORES
And sorry, but I don't do much Python.
For development you then use nix-shell to build the environment with all of the dependencies available. The downside is that you either need to commit the default.nix to the project, or configure it to be ignored. But the advantage is that it handles everything for you. For example you can configure native libraries such as PostgreSQL client or openssl without worrying if they are installed on your system. This means that the entire dev environment is tested in CI (assuming that you run nix-build in CI).
This is also nicer than docker/VM based dev environments as it just prefixes your usual environment with the deps, so you still have your regular commands and development tools available.
Are you up for explaining these gists? We use nix extensively at work and yet I’m having trouble wrapping my head around these. Node dependencies have been a thorn in our nixification of some packages, C / C++ has come easily.
Sure. NPM is the easy case because the package-lock.json file can easily be read by Nix and contains hashes for all of the packages. This means that simply be importing the file into Nix you can have a reproducible build. No Nix-specific maintenance required.
In the linked case I use this library to manage that https://github.com/nmattia/napalm (in that example I use master but for production I would pin a version). It simply parses the package-lock.json, downloads the packages and uses npm to build the node_modules folder. It also provides some convenient functions for building packages with "bin" files or just linking node_modules inside a build.
Note that this is more for project development. It doesn't use the "system" packages (intentionally) for Node, it fetches whatever versions you have specified from NPM. Nix will only provide the "native" stuff like Node and NPM themselves and any native libraries.
In the linked case I use this library to manage that https://github.com/nmattia/napalm (in that example I use master but for production I would pin a version). It simply parses the package-lock.json, downloads the packages and uses npm to build the node_modules folder. It also provides some convenient functions for building packages with "bin" files or just linking node_modules inside a build.
Note that this is more for project development. It doesn't use the "system" packages (intentionally) for Node, it fetches whatever versions you have specified from NPM. Nix will only provide the "native" stuff like Node and NPM themselves and any native libraries.
IMO (shameless plug, I'm the author) the best way to manage Python environments with Nix is poetry2nix: https://github.com/nix-community/poetry2nix.
I see isolated dev environments in terminal. Can Pycharm (IDE and GUI) pick up these environments?
Services and dev tools want to hog system ports. Can I run multiple dev projects each managing own services in Pycharm?
Services and dev tools want to hog system ports. Can I run multiple dev projects each managing own services in Pycharm?
Would love to see an official Docker image.
There is one: https://github.com/NixOS/docker
But it's more of a Nix docker image than a NixOS one, because half of the things that NixOS gives you don't really make much sense in a docker image (systemd in a container?).
But it's more of a Nix docker image than a NixOS one, because half of the things that NixOS gives you don't really make much sense in a docker image (systemd in a container?).
> it's more of a Nix docker image
Time to update!
I also appreciate that a lot of frequently used configuration is easy to enable. For example, to install podman, the Arch wiki details a number of files in /etc you need to edit. On NixOS, just add `podman.enable = true;` to your configuration and you're done.
At this point, I think NixOS is hugely underrated and I strongly encourage anyone who is an advanced linux user to try it.