Our Roadmap for Nix(tweag.io)

283 pointsby cosmojgvor 4 Jahren256 comments

https://www.tweag.io/blog/2022-08-04-tweag-and-nix-future/

259 comments

Hnusvor 4 Jahren

I have never felt more dumb than when trying to use and learn anything nixos related. Its awesome technology not meant to be used by humans. I am using cs.github.com, open ai chat bot and usually like 4 tabs describing the same thing from different months using different commands + few versions of what seems like official manual while still having zero idea what I am doing. It feels more like I am reverse engineering something rather than using it. There is something seriously wrong with what is acceptable ergonomics by whoever is designing the user interfaces. I really wish they fix all their problems because when it all works its awesome.

I wanted to give guix a shot does anybody know if its any better ignoring their stance on non-free software as I think you can work around it.

grumbelvor 4 Jahren

Guix interface is a bit more friendly by default, but once you enable the 'nix' command on NixOS, there really isn't much of difference in terms of basic CLI experience.

The big different is the language and I think Nix wins here by a mile. Doing everything with Scheme just leads to layers of macro spaghetti that I really did not enjoy to dig through. Error messages that tell you absolutely nothing about what went wrong were pretty common. Nix has those too, but less frequently. Also with Nix you just use regular shell scripts snippets for the building the packages, Guix wants you to do it all in Scheme. Package selection on NixOS is much bigger.

Another big thing, Nix has Flakes, which make it trivially to turn all your Git repositories into Nix packages. Your Git repository becomes essentially a first class citizen in the package manager, making it completely trivial to run different versions of the same software. Guix has none of that, they still treat packages as a separate thing from the software itself and trying to add third party packages involves quite a bit of overhead. Easily up or downgrading individual software isn't possible as far as I can tell, you have have to roll back the complete Guix system to do so.

Basically, after switching from Guix to NixOS, I can't say I missed anything. NixOS just felt like a more polished and feature rich version of what Guix was doing, which given that Guix is basically a NIH version of Nix, is understandable.

jyrkeshvor 4 Jahren

> Another big thing, Nix has Flakes, which make it trivially to turn all your Git repositories into Nix packages.

I have no doubt that this is true, but I still feel dumb about it. I run a Plex server (among a few other workloads) on a NixOS box, and just figuring out how to update the Plex binaries off of the official master branch was a multi-hour back-and-forth endeavor between StackOverflow suggestions and compiler errors.

I really do love Nix though. And once I got it working I felt a LITTLE bit smarter, AND it's worked flawlessly for months.

benreesmanvor 4 Jahren

I’ve said this elsewhere in the thread and might do so again but it bears repeating: use flakes. At first they seem like this weird semi-supported thing, but eventually you realize that Nix is badly broken without them.

Flakes aren’t “experimental”. “Experimental” is what happens to your sanity without them.

yjftsjthsd-hvor 4 Jahren

> Flakes aren’t “experimental”. “Experimental” is what happens to your sanity without them.

Okay, I like flakes, I use flakes, I enable flakes without much worry. But. They literally are an experimental feature, and they are still enabled by setting "experimental-features = nix-command flakes" in nix's config.

benreesmanvor 4 Jahren

I think I was maybe too casual with my language. Flakes are clearly marked experimental in the documentation. But they were a no-brainer a year ago, they’re always “a few months” from being official, they map directly onto users’ intuition around lock files, they enable principled composition of shit that doesn’t live in nixpkgs, the list goes on.

Outside of some (hypothetical to me) edge cases, not using them is masochism.

Flakes have a standard library (“flake-utils”) and and a pimped out version (“flake-utils-plus”).

Making it a weird gymnastics trick for a novice user to even access them is the kind of shit GHC devs say, “no, that’s too user-hostile even for us”.

Use flakes.

ethbr0vor 4 Jahren

> a multi-hour back-and-forth endeavor between StackOverflow suggestions and compiler errors

Sounds like the authentic 90s linux package experience! (except with 's/StackOverflow/mailing list/')

rowanG077vor 4 Jahren

How do you do it? I literally remove the flake from my nix profile and re-add it with a specific commit hash. I had troubles that just specifying master didn't update to the current master.

jyrkeshvor 4 Jahren

I cheated and did a stateful thing (because I couldn't figure out flakes): sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable

and then I added this blurb to my Nix config:

nixpkgs.config = { allowUnfree = true; packageOverrides = pkgs: { unstable = import <nixos-unstable> { config = config.nixpkgs.config; }; }; };

josteinkvor 4 Jahren

> Doing everything with Scheme just leads to layers of macro spaghetti that I really did not enjoy to dig through.

On the flip side, it’s a real language, well understood, with a real specification, working tooling, and community support outside Guix (or in the case of Nix, outside the package manager).

As someone who gave up NixOS exactly because of the incomprehensibleness of the Nix language, I know what I would choose.

There’s no doubt at all.

grumbelvor 4 Jahren

I see nothing especially incomprehensible about Nix, the language itself pretty simple. About the only thing I had a bit of an issue with is that I kept forgetting that functions only take a single argument in Nix. But other than that it is very nice to work with and has all the syntactic sugar you want, string interpolation, sets/map, sane multi-line strings and all that. Syntactic sugar is an area where Scheme has basically nothing to offer, everything needs calls to functions with long names.

For most common uses you barely even have to care about the language, as it's just some JSON-like data with shell scripts in between that do the actual work.

parminyavor 4 Jahren

The language very much generates complaints because it is gives you no real help understanding what argument should be passed to what function and how to get that argument. Error messages are dreadful because there's often a million functions your object has to go through before it produces an error. Dynamic typing and lazy evaluation don't seem to mix. Some kind of compile-time, presumably gradual, type system would really be an incredible improvement, so that type errors were generally eager.

pxcvor 4 Jahren

> Dynamic typing and lazy evaluation don't seem to mix.

Incidentally, this is one of the things that some of the people involved with the blog post in the OP have set out to fix. A few years ago, Théophane (author of the blog post) started an experiment to add gradual typing to Nix, which eventually became the start of Nickel: https://github.com/tweag/nickel

Today, Nickel development continues at Tweag, but now they have someone else, Yann Hamdaoui, driving the implementation forward. (I assume because he's a domain expert in programming languages.) Producing helpful error messages is one of the major goals of Nickel, as a potential successor language to Nix. All of the language design decisions are documented as GitHub issues on the repo, so if the technical side of this kind of problem is interesting to you, you may enjoy browsing that.

It appears that Nickel is very close to a usable state for early experimentation. You might be interested in cloning the repo and messing around with it!

babyvor 4 Jahren

Honestly I’ve done enough scheme to understand that I wouldn’t touch anything related with scheme in a million years. People who think otherwise probably don’t understand why people like javascript and python and will never write any product that catches on.

The nix language is pretty nice on the other hand. The only thing I’m missing is a language server protocol thing so I can go to definition

axlprosevor 4 Jahren

> People who think otherwise probably don’t understand why people like javascript and python and will never write any product that catches on.

Ironically, this has been posted on a popular site written in a dialect of scheme:

http://arclanguage.org/

https://github.com/arclanguage/anarki

meltedcapacitorvor 4 Jahren

Where is the irony? HN does not expose scheme to the user.

Don't think the parent was saying that it's impossible to write a good program in scheme/lisp, more that scheme as user interface for a software system is a hard sell... which seems anecdotally true.

babyvor 4 Jahren

how is this ironic? HN was written by one person, and nobody needs to care what language they used to make it work.

t0astbreadvor 4 Jahren

rnix-lsp is a language server for Nix that provides go to definition. I'm using it with the "Nix IDE" extension in VS Code. It doesn't handle imports though afaict, so it's only useful for local definitions.

_hl_vor 4 Jahren

> Error messages that tell you absolutely nothing about what went wrong were pretty common.

Maybe I'm holding it wrong, but I almost never get useful error messages from Nix (NixOS and NixOps). Almost every error is deep within some module with no trace to the option I set. Just nix barfing its 300-line eval traceback where my own code doesn't even appear.

geddawmvor 4 Jahren

Nix has a debugger now. You can drop into the frames that are causing problems and inspect the state. Miles easier than looking at the trace manually.

benreesmanvor 4 Jahren

I last flashed my flake.lock in June, so I might be out of date, but my rigs can’t reliably include the line number in my code that led to the duck-typing fiasco deep in the core of someLanguage.withPackages.fuck.WTF.this.

Fix the fucking “—show-trace” thing folks. Seriously.

VTimofeenkovor 4 Jahren

Out of all cases when I used --show-trace, I can remember it being useful only once or twice. I gave up and just sprinkled builtins.trace.

IIRC debugger comes with nix 2.9 and nixpkgs for 22.05 still has 2.8.x.

n8henrievor 4 Jahren

I was excited to see the debugger flag to nix build the other day, since I'm really struggling cross-compiling an RPi on BTRFS root sd image.

Of course running with the debugger flag resulted in an unintelligible error message and no debugger.

rtpgvor 4 Jahren

Any link to documentation on how to use it? Would be a godsend for me

babyvor 4 Jahren

I agree. I'm actually trying a hello-world example from the tweag website, and it complains about "gcc" not existing. I try adding

    buildInputs = [ nixpkgs.gcc ];

but it complains that nixpkgs.gcc does not exist. There's nothing helpful. I remember fixing this issue a month ago, and I have no recollection on how I did it.

hypersoarvor 4 Jahren

Are you using flakes? You need to use the legacyPackages output for your system, e.g.

  buildInputs = [ nixpkgs.legacyPackages."x64_64-linux".gcc ];

In practice, it usually looks something like this:

  let
    system = "x86_64-linux";
    pkgs = nixpkgs.legacyPackages.${system};
  in stdenv.mkDerivation {
    ...
    buildInputs = [ pkgs.gcc ];
    ...
  }

benreesmanvor 4 Jahren

So there is more than one way to skin this cat. ‘nixpkgs’ as handed to you as part of a flakes ‘inputs’ is (jargon alert) partially applied, which is to say you have to put a battery in it. In this instance what you have to pass is ‘system’ because nixpkgs supports many systems (e.g. ‘linux-x86_64’ or ‘darwin-aarach64’). It sort of makes sense that it wouldn’t know what GCC you want without that information.

You can do this up top by hand, e.g. P = inputs.nixpkgs { inherit system; }, but more commonly you would use a library like ‘flake-utils’ or ‘flake-utils-plus’ to issue system configuration for all supported packages to all your packages/derivations/devShells.

babyvor 4 Jahren

Ah right, I saw that library but it didn’t have examples and I didn’t manage to use it. Do you know a page that would explain that?

benreesmanvor 4 Jahren

The ‘flake-utils’ readme is a pretty good jumping off point: https://github.com/numtide/flake-utils

I have this or that nitpick with FL and FLP but overall it’s very solid stuff. FLP is a little more “magical”, and that’s not always the best starting out, but you really can’t go wrong with either.

Smaug123vor 4 Jahren

Maybe you got confused (like I did every time until I stopped doing anything non-flakey) about when `nixpkgs` is called `nixpkgs` and when it's called `pkgs`? In the flakes world it's considerably easier because everything is explicitly named, so you take an actual decision about what to call it.

babyvor 4 Jahren

Is the convention to call it pkgs when you instantiate it with a system?

cmmvor 4 Jahren

yes. other possible arguments are overlays and package selection modifiers (like whether to include non-free stuff or stuff that is marked as broken).

kaba0vor 4 Jahren

I don’t know what is the specific file you are working on, but shouldn’t it be pkgs.gcc?

SuperSandro2000vor 4 Jahren

That's strange. GCC is part of the stdenv on Linux and though always available.

To correct your coffee snippet: nativeBuildInputs = [ pkgs.gcc ];

babyvor 4 Jahren

I’m on mac, I managed to fix it with https://mimoo.github.io/nixbyexample/flakes-packaging.html

babyvor 4 Jahren

And now I’m trying to understand how devshells work in flakes and there’s pretty no doc

benreesmanvor 4 Jahren

So a dev shell as consumed by a flake is typically just a derivation. And typically the most important thing in that derivation is ‘buildInputs’, which is just a list of ‘pkgs.thingIWant’. It’s not uncommon to hook other “phases” of the derivation, which are (oversimplified) just little shell scripts that run in some order (‘configurePhase’ before ‘installPhase’) and typically expressed using the nifty “” syntax. For example if Nix is like stuffing paths into a Makefile that you want to use in your dev shell, you might put that in installPhase. Protip: a Nix “package” (derivation) evaluated in a string context (foo = “${pkgs.gcc}”;) gives you a path to where the thing lives in /nix/store.

Smaug123vor 4 Jahren

Ask your questions on discourse.nixos.org please - the documentation situation in Nix-land is generally exactly as you have noted, which means centralised question-asking in well-known locations is much much more helpful for Those Who Come After.

benreesmanvor 4 Jahren

I say this with the utmost respect for those who work hard to answer questions on discourse, but I have found it singularly unhelpful.

I strongly encourage the Nix community to embrace a mainstream Q&A platform like StackOverflow or GitHub Issues.

Nix is “weird” enough for beginners already, the kind of janky interface and uneven moderation/quality on discourse is IMHO self-defeating for Nix.

babyvor 4 Jahren

Agree. I feel the same way about OCaml, and I have forced myself to ask every questions I have on SO to build the list of answers there. Yet, people tend to answer on SO with “you should ask on the OCaml discourse”…

pxcvor 4 Jahren

Aside: I love the Discourse UI. It feels easy and thoroughly modern to me. What's janky about it, in your opinion?

babyvor 4 Jahren

I also do not like it for different reasons (one of them is that they hijack ctrl+F)

pxcvor 4 Jahren

Ah yeah, that behavior is annoying. At least if you hit it twice, you get the normal in-browser thing

rekadovor 4 Jahren

Guix has package transformations, which lets you use different commits or git URIs for a given package definition.

It also doesn't require the definition of a package if all you want is an environment to hack on the code in a git repo. People use `manifest.scm` or `guix.scm` for these purposes.

Guix System services are in fact all built together, but upgrading individual software or services does not require "roll[ing] back the complete Guix system ".

(I want to point out that describing Guix as "a NIH version of Nix" is a symptom of the attitudes that made me almost completely retreat from discussions of software online. It's so grating to deal with dismissive attitudes like that again and again. Sucks all the fun out of hacking.)

f1reflyvor 4 Jahren

> Also with Nix you just use regular shell scripts snippets for the building the packages, Guix wants you to do it all in Scheme

I'm by no means a good schemer, but I'd still take scheme over shell every time. Shell tends to devolve into something completely unreadable with even moderate complexity while scheme can make even the most complex logic nice to work with.

> Guix has none of that, they still treat packages as a separate thing from the software itself

I'm not quite sure what you mean with that, but it's sure possible (and easy) to just define a guix repo that points to all your projects repositories and make them available through the guix package manager. Guix repos are just git repos with a few lines of metadata attached.

> Easily up or downgrading individual software isn't possible as far as I can tell, you have have to roll back the complete Guix system to do so.

It's very easy actually. You declare a package definition as an inferior to a package and can pin that definition to whatever version, commit hash, tag, you want. You basically say "I want the package declared as name X, but I want it while the guix repo is at point Y. That way, the package manager knows what dependencies the package has and provide those with the right versions as well.

> NixOS just felt like a more polished and feature rich version of what Guix was doing, which given that Guix is basically a NIH version of Nix, is understandable.

On the contrary, guix is more like a more polished and dedicated version of nix. Nix's solution to hard-to-package packages is "We take everything that's needed to build this thing and freeze it in the repository" while guix will not accept anything that doesn't build completly from scratch and can thus be distributed as optional source code. The difference is very important because it allows for better packaging and more flexible versioning of the packages. Using guix, you will never run into a situation where the solution has you staring down at a binary blob and questioning your life choices.

Machavor 4 Jahren

Is there an actual tutorial on how to use the nix command? All the official docs for Nix seem to be slightly discouraging it as not ready where they do mention it.

It's one where the community is pretty clear don't use nix-env etc. but the tutorials only explain nix-env, with the only details of the new commands being the CLI reference manual which doesn't cover workflows.

phaervor 4 Jahren

Yes, my understanding is that the official documentation lags behind community usage, because flakes are still an "experimental" feature as details of the implementation are worked out.

For reference documentation, there's "experimental commands" in the manual https://nixos.org/manual/nix/stable/command-ref/experimental...

barkingcatvor 4 Jahren

I'm finding nix-env to be great! I do not understand why nix offers a command and a package index, but then advises not to use it.

If people aren't supposed to use nix-env, why don't nix put a huge deprecated notice on every invocation and delete the nix-env command itself from nix?

Machavor 4 Jahren

I think the problems with nix-env come down to two things:

- nix-env -i and nix-env -u will almost certainly eventually do the wrong thing for you if you use them enough (nix-env -iA is ok): https://ianthehenry.com/posts/how-to-learn-nix/ambiguous-pac...

- It's not reproducible as it depends on the state of your nix-channel setup at that time. How much of a problem this is depends on your use case. nix-as-homebrew-replacemnt or nix-as-shared-package-list-across-osx-and-linux users are probably fine with this, as what these users care about is more portability than reproducibility, but nix-as-predictable-dev-environment types will be let down by it.

I think the fact that the latter affects different users differently and that the new flake world doesn't offer a "modify your global environment without the tool taking over" option (expecting you to use nixOS or nix-darwin instead) has meant that there's not enough motivation to agree on a replacement that nix-env could tell you to use instead.

I guess the eventual replacement will be "nix profile", but while you'll find at least some documentation on "nix develop" and "nix flake", I had to look in the CLI tool docs for this one, which makes me suspect it's even less final than the other two

So instead the community will tell you to avoid nix-env, but there's no agreement to have the tool or docs tell you that.

Smaug123vor 4 Jahren

It's not quite true that it's trivial to turn a Git repo into a flake, because flakes don't do submodules, and https://github.com/NixOS/nix/pull/5497 has languished :(

grumbelvor 4 Jahren

You can fix that by adding '?submodules=1' to the URL. It does get rather annoying when compiling local code, as that turns a simple 'nix build' into:

    nix build  "git+file://$(pwd)?submodules=1"

But it will work with submodules.

Another mild annoyance related to this is that Flake inputs can't be expressions, this can make it a rather difficult to fetch dependencies that aren't packaged in a way understood by the existing fetch methods.

microtonalvor 4 Jahren

as that turns a simple 'nix build' into

Which reveals another annoying thing about using flakes for development. It copies every time you build something the whole darn project into the Nix store. Fine when you have a small project, but with large projects this becomes a significant overhead.

ParetoOptimalvor 4 Jahren

> It copies every time you build something the whole darn project into the Nix store. Fine when you have a small project, but with large projects this becomes a significant overhead.

Isn't copying the repo to a read only location just inherent to the problem of reproducibility?

grumbelvor 4 Jahren

Not necessarily, most build systems allow you to separate source input and build output. So grabbing the source from the current directly is a possibility, if the build script is clean and doesn't do anything weird in the source directory.

In Nix you can get that via 'nix develop', which gives you a shell with access to the individual build phases and with a bit of tweaking, allows you to run them from your source directory without copying the source around first (see 'declare -f genericBuild').

grumbelvor 4 Jahren

When you build into the store, you have to rebuild from scratch. However Nix does allow incremental builds inside your source directory as well. If you spawn a development shell like:

    nix develop .

All the build phases are available as shell functions or variables (unpackPhase, patchPhase, configurePhase, buildPhase, ...). The phases depend on each other, so if you want to skip a phase some additional tweaking or setting of environment variables will be necessary. The "genericBuild" function is the top level entry point for a normal build, the source can be viewed via:

   declare -f genericBuild

and should give a bit of an idea what Nix is doing.

Smaug123vor 4 Jahren

Oh, you're a lifesaver. Thanks! (I spent quite a lot of time Googling how to do this, and never found it.)

civodulvor 4 Jahren

Guix channels (not to be confused with Nix channels) let you turn a Git repo into a package collection:

https://guix.gnu.org/manual/devel/en/html_node/Channels.html

I guess it serves some of the same purposes as Flakes, and it comes with other bells and whistles such as authentication (which I think is kinda important!).

For the rest, your description of Guix is largely incorrect ("has none of that", "you have to roll back the complete Guix system", etc.), but as a Guix hacker I'd be interested in better understanding your experience and perceptions if you'd like to get in touch with us on the mailing lists!

babyvor 4 Jahren

I’ve heard guix uses scheme, so I’m not sure I understand how it can be friendly (you (see (what (I (mean))))))

benreesmanvor 4 Jahren

You aren’t dumb. Nix is that rarest of birds: something damned-near impossible to learn well that’s still worth it.

Learning Nix well from the Internet is a horror film. But… once you do it’s shockingly, actually worth it.

Getting anything complicated working/building under Nix is a PITA, the payoff is that when you fix broken things, they stay fixed.

setheronvor 4 Jahren

I introduced Nix into our build setup as well. We are an acquisition within Google and it's been quite a fun adventure incorporating it alongside Bazel.

Nix has been great but I've run up against a few thorny edges. Luckily I've sent out patches for them; long love OSS :)

oxffvor 4 Jahren

> more like I am reverse engineering something rather than using it

Perfect description.

louibvor 4 Jahren

I tried learning using several different resources, but what did it for me was the nix pills (https://nixos.org/guides/nix-pills/). I'd say it's becoming a bit outdated now with the nix standalone command, but the fundamentals are still mostly the same.

microtonalvor 4 Jahren

Perhaps in contrast to most people's experiences, I haven't found Nix hard to learn. I think it really helps if you have learned Haskell before. Nix (the language) is really a simple functional programming language, but Nix relies quite heavily on functional programming concepts like laziness, fixed points, etc. A lot of things are underdocumented, but if you understand the Nix language well, it isn't hard to look up definitions in nixpkgs. I can understand that it is all very alien and overwhelming if you do not have a grounding in functional programming.

The primary issue for me has been that Nix is a very deep rabbit hole. You can spend enormous amounts of time on making your configuration more functional and declarative. Pretty much like you can spend enormous amounts of time on customizing an Emacs configuration. It's hard to strike a balance. And outside declaratively defined infrastructure (servers), it's probably not really worth it. I could almost fully reproduce my NixOS system (there is always some mutable state left) with a single command. But takes many months of effort to get to that point. On the other hand, I can set up a fresh macOS or Fedora Silverblue systems with all my customizations in 1 or 2 hours and have to do that maybe once of twice a year? So, ¯\_(ツ)_/¯.

I think the balance is different when you manage a lot of servers and most servers can be defined as a function with a small number of varying parameters.

The other part of the rabbit hole is that software breaks frequently in Nix. Upstreams do not really develop things under the assumption of a non-FHS, immutable system. So, you were going to work on something, but before you know it is 30 minutes later because you ended up fixing some package you need and that broke. Similarly, you'll end up packaging a lot of stuff and spending quite some time making it fit the Nix mold (looking at you Python packages that mutate in-place, sigh).

I love Nix as a principle -- it's declarative, immutable, pure, reproducible. But in practice, you can reap many of the same benefits from impure, inferior alternatives, with far less work. Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours. Rust's cargo doesn't allow you to specify every non-Rust dependency exactly, but with a Cargo.lock file you can make most of your build reproducible. NixOS is a clean, immutable, declarative system, but other systems offer a subset of its features, such as atomic upgrades/rollbacks, immutable root, and isolated applications (e.g. Fedora SilverBlue + Flatpaks, Fedora IoT, macOS). These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.

Worse is better.

crabbygrabbyvor 4 Jahren

I know Haskell, regularly write in rust for fun, and nix is by far the most complicated thing I've tried to learn hahaha. Glad you were able to figure it out, and seriously thank you for sharing your opinion of how you view it even after learning it.

I see some younger folks really investing time in it and it worries me. Sometimes smart people fall under this fallacy of "you have to be smart to learn this so it must be good because it's hard" when really it looks more like quicksand than asphalt. Quicksand is hard to get out of, but there's still no reward for falling in.

kaba0vor 4 Jahren

I recommend looking at already existing package descriptions for programs written in the same language you are looking for. Chances are, you can hit it enough times to make it package up your program as well.

I don’t think learning nix from its internals is worth it, especially not for beginners.

pxcvor 4 Jahren

> I don’t think learning nix from its internals is worth it, especially not for beginners.

Nitpick: what you're calling 'Nix internals' are actually Nixpkgs internals.

FWIW, I think that to some extent this is just the way you have to go when you're trying to quickly take up a new (to you) configuration language for practical purposes.

I've somewhat recently started putting together some simple CI/CD pipelines at work with a tool called Dagger, which is a container image builder-and-runner based on BuildKit and the CUE language.

When I first started learning Nix years ago, I had a lot of packaging trouble with some quirky software that wasn't yet packaged for Nix, and I only ended up getting unstuck by asking for help on IRC. One generous person was hitting me with incredibly helpful links to examples from Nixpkgs and the documentation left and right, and I was astonished with their fluency. I asked them how they knew all this, and they basically said:

> Idk, one day I just sat down and started to read through some of the Nixpkgs codebase, and now I `grep -R` through it a lot to look for examples.

At the time it sounded adventurous and crazy to me, as I was pretty intimidated by the prospect. Eventually that became one of my standard tools: I treat the Nixpkgs repo as another source of documentation.

Fast forward to today and my Dagger implementation efforts, and things got immensely easier for me the moment I started treating Dagger like Nix. I cloned the upstream Dagger repository and started searching through the unit tests with `rg`, and all the answers to my questions about what functionality Dagger exposes via CUE modules were right there. Tons of the guesswork disappeared, even though I don't yet have a great grasp of the CUE language.

In some ways, the Dagger case is harder, though. In the name of simplicity and partly for other language design reasons, CUE is emphatically not Turing-complete. This is interesting and attractive in some ways, but one of the consequences is that sometimes you hit bottom in the CUE libraries to find some compiler directive that links in functionality which had to be implemented in a Real™ programming language (in this case, Go). So to really understand, e.g., the custom data types Dagger uses for secrets, you have to go read code in another language.

By contrast, in the Nix case, the language is powerful enough that you can learn everything you need to for even very advanced Nix usage only by reading Nixlang, either in Nixpkgs or in other examples. You never actually have to dive into true 'Nix internals' (the Nix repo, which is C++ code) to figure out how something in a Nix example is achieved. With a more restricted DSL, you sometimes do.

Maybe one valuable 'missing manual' would be something like 'How to read Nixpkgs'! It's such a rich source of examples. I think making earlier reference to it could help a lot of people make faster progress and feel more effective when learning Nix.

yusefnaporavor 4 Jahren

I would love to see a "guided tour of nixpkgs," using real-world packages as examples and building up in complexity as you go.

I feel like so many of the problem I encounter as a new nix user have been neatly solved already and there are probably tons of great examples in nixpkgs, but I have no clue where to look, or even what to grep for.

It would be super helpful if there were a resource with examples of common problems and how they're solved in the real world. For example, say you're packaging something whose unit tests hit the network. There could be a link to a package definition in nixpkgs that overrides the check phase to disable the problematic test.

AndersonTorresvor 4 Jahren

When anything hits the network, there is nothing to do besides

- disabling the code that hits the network, whether by patching or by configuration ("--disable-downloads" or anything similar)

- or emulating it; e.g. if the script downloads a file, we can `fetchurl` it and move the downloaded file to its expected place. 9Of course, it works when the script does not override the downloaded file; otherwise, go to the previous option)

Two examples came to my mind:

- Arcan vendoring required patching its cmake scripts, because cmake does not honor the cache and tries to download things anyway:

https://github.com/NixOS/nixpkgs/blob/901978e1fd43753d56299a...

- cardboard didn't require it because meson honors the cache (and Nixpkgs configures Meson to not download anything):

https://github.com/NixOS/nixpkgs/blob/901978e1fd43753d56299a...

benreesmanvor 4 Jahren

I think the first step to solving problems is acknowledging them: lazy? cool. duck-typed? cool. weird thunk stack traces, eh, workable.

Lazy and pure and duck-typed and broken ass stack traces? Fuck that. Fuck that in particular.

Eelco fucked up. Badly. nixpkgs uses Haskell type signatures as comments in the core ‘lib’s, because duck-typed pure lazy functional is fucking insane.

rkrzrvor 4 Jahren

I agree - Nix, the language, really is at the root of many of Nix' problems.

The bad error messages, the useless stack traces, and the overall un-debuggability of Nix are not just minor issues. These are at the core of the steep learning curve of Nix. These are also the reason why with Nix you are often 'dead in the water' and don't know how to fix a problem, and end up googling for hours or asking somebody else who knows more about Nix.

And these issues cannot be fixed by having "better docs" or a better tutorial.

If somebody were to rewrite the language today then my suggestion would be to make it statically typed, like Haskell. I would probably also make it strict, with optional laziness, although that's less important imo. And providing good error messages and good debuggability should be explicit design goals.

Making it statically typed would necessitate a different overall design, since many of the dynamic tricks would not be possible any more, but I suspect that you would end up with a better overall design in the end.

benreesmanvor 4 Jahren

I think you would just make it Haskell. I don’t know the exact chronology of Nix (the language), so I can’t comment with full confidence, but Real World Haskell (written by my utterly brilliant former colleague Bryan O’Sullivan /humblebrag) was published in 2008, and the Haskell described there has monad transformers and all kinds of nifty stuff. I think Eelco’s thesis was published around the same time? It’s possible that my dates are off and they really needed to rig up their own System F machine, but I suspect it just seemed cool and now we’re stuck with it.

Nix (the language) stomps like JSON or YAML for writing config files, it’s actually not bad at that, but it’s a disaster in its contemporary role.

AndersonTorresvor 4 Jahren

2006, to be accurate.

https://researchr.org/publication/Dolstra2006

More recently there is an attempt to run Nickel as a successor for Nix: https://www.tweag.io/blog/2020-10-22-nickel-open-sourcing/

Smaug123vor 4 Jahren

Oh thank God, I thought I was the only one. A while back, I posted "a lazy dynamically-typed language (in my book) perfectly combines the run-time reasonability of Haskell with all the development-time safety of LISP" on the Discourse, and the reply "I love the lazy functional minimal expression-oriented JSON-like packaging DSL that is nix" got 13 Likes. I thought I was going completely insane.

benreesmanvor 4 Jahren

You’re not crazy. Duck-typing a lazy functional language in a zillion KLOC repo with broken stack traces is crazy.

At my work those “O RLY” meme book covers are in vogue this week, and my last one was “Nix Haskell Overlays for Fun and Murder: Homicidal Ideation for the Working Hacker”.

It was my job to get Clash (the VHDL/Verilog synthesis tool on Haskell) building under Bazel via Tweag’s rules_haskell and playing nice with all our other Haskell shit. That’s 14 hours straight I’m never getting back.

But… because it’s Nix I never have to do that again. Even if I upgrade everything else I can keep nixpkgs at that version in my flake.lock and that will always work forever. So I’m cool with it.

Kaze404vor 4 Jahren

> But… because it’s Nix I never have to do that again.

This is the killer feature for me. My use-case for Nix is vastly different from yours, as I use it for centralized declarative configuration through NixOS, but I have an immense sense of satisfaction knowing that hell can freeze over but I'm still only a single command away from my exact setup running on (almost) any machine on the planet.

This might not matter much for most people, but making my machine work for me is something that took me years, and I can sleep well knowing that it's finally safe under Nix vs. multiple different configuration files for multiple different programs scattered across my filesystem.

benreesmanvor 4 Jahren

Our use cases might not be that different. Whether it’s Mac laptops, physical dev machines, or cloud instances, our domain/scale is such that our machines are “pets” rather than “cattle”, i.e. they have names.

Making NVIDIA drivers work with either Xorg on someone’s desk or CuDNN on the GPU boxes with like two friggin lines in the NixOS stuff is like, doing drugs or something in terms of sheer feeling great after years of chasing that shit all around Ubuntu’s broken-ass standard of living.

pxcvor 4 Jahren

> Nix Haskell Overlays for Fun and Murder: Homicidal Ideation for the Working Hacker

good god pls share the link

benreesmanvor 4 Jahren

Haha sure. I don’t know anything about image hosting on the fly so pardon the sketchy ass link: https://ibb.co/Dr5Jsrx

pxcvor 4 Jahren

> These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.

> Worse is better.

There are some potential alternatives out there that seem inspired by Nix but which are less strict. I think a system that offers and manages Nix's purity but doesn't require it could end up the final winner some day. Shea Levy had some ideas for eventually achieving this in Nix that he called 'Nomia' (probably riffing on the name of Sander van der Burg's Dysnomia, I'd guess), but his life circumstances changed before the project really materialized. But Nix still could be the place where that is first achieved.

Meanwhile new container image-building DSLs like Dagger¹ and Bass² look a lot like a Nix which is impure by default, and bash builders are replaced with BuildKit directives. And there's Denxi³, which looks more like a Nix that lets you reduce its purity when you want to.

For now, though, the Nix ecosystem is still vastly more capable than these alternatives because of all the incredible work that Nixpkgs represents. Nix we know it today could still end up the concrete basis for what succeeds it, and not just an inspiration.

1: https://dagger.io/

2: https://bass-lang.org/

3: https://docs.racket-lang.org/denxi-guide/

AndersonTorresvor 4 Jahren

My experience was seriously different. I started with Slackware, switched to Arch and after a failed attempt to run some AUR scripts, I was tired and switched to NixOS.

Incidentally, I treat both my Emacs config and my NixOS and home-manager configurations as Igors. They are never really finished. In this sense, the "many months of effort to get to that point" does not matter. Also, with NixOS I feel I can be bolder than with other distros, knowing with way more accuracy what is breaking my configuration.

> Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours.

The Docker replit was using before Nix was going hopelessly "uglier": https://blog.replit.com/nix

Also, what Cargo and NPM and `<insert lang here>` lockfile system do for each lang, Nix does in a more general package management level.

rewgsvor 4 Jahren

I’ve religiously updated a new-linux-install.sh. The majority of packages I build from source, and the remaining I use the distro’s package manager. I’m well aware that nix is a very different beast, but in practice this gives me basically the same outcome. Nix is incredibly interesting to me, but all the painful stories have kept me on my very simple and long bash script.

BiteCode_devvor 4 Jahren

Everybody that sold nix to me said the goal was to replace the complicated side effect mess that is our current stack with something simple and pure.

As with FP, eventually, practicality beats purity I guess.

ParetoOptimalvor 4 Jahren

> As with FP, eventually, practicality beats purity I guess.

Until you can take enough advantage of purity that it's practicality is higher.

crabbygrabbyvor 4 Jahren

Probably not worth it. Not trying to be dismissive but genuinely, it's fine to have side effects and if it saves 1000 hrs of engineering over the life cycle of a project I'd wager it's completely worth it.

rowanG077vor 4 Jahren

The keypoint here is that nix requires a larger up front investment. But once done it will save you 1000 hrs of engineering in the long run. It's absolutely faster to start a project without nix and just start coding. But once you have many deps, multiple developers, ci, multiple environments nix becomes a no brainer.

SuperSandro2000vor 4 Jahren

But the big problem in the end will likely be that you have no idea how reproduce the setup from scratch again

crabbygrabbyvor 4 Jahren

It's not just you, the user experience of nix is terrible. It's biggest feature of "pure" builds is mostly unnecessary given the complexity. It's runner up feature of configuring builds with a few scripts can also easily be accomplished by simpler tooling...

I don't get why it's necessary or unfortunately why it is so complicated... Anything that can be done in nix can be done in a tenth the time by any employee in a more commonly used technology. In industry, this is a very bad thing in my opinion

benreesmanvor 4 Jahren

The user experience of Nix is terrible. I’m not sure that it’s even fixable.

But Nix or something like it is going to take over the world.

Dynamic linking by default is absurdly stupid and mostly motivated by GNU politics. It’s a terrible problem. Many (most?) of the users of Docker don’t even realize that this is the problem Docker is solving for them. But they know they have a terrible problem and Docker helps a lot.

Nix is Docker on steroids. It’s Docker that got bitten by a radioactive spider.

Linux namespaces and cgroups and BSD jails have been around. Docker made it Just Work.

When someone does that for Nix, which is basically Docker done by computer scientists, it’s game over for anything else.

lillecarlvor 4 Jahren

You can use Nix to build OCI(images) and run them on Kubernetes if you want to.

Nix is a souped up package manager, Docker is a container runtime.

Nix depends on packages existing in /nix Docker chroots into a "folder" and runs a command(+many more things).

Lets not mix technologies up for the readers too much.

benreesmanvor 4 Jahren

I’m well aware that Nix can produce container images. Xe has a great post about it.

People use Docker for a lot of reasons, but mostly? Same Dockerfile, same outcome, mostly every time. No one is moving /usr/lib/x86_64 around under you. It’s a real sea change, on the order of revision control: we hadn’t even realized that we were living with constant low-level anxiety that someone was going to break our computing environment at any moment. “sudo apt upgrade —whatever”, eh, maybe next week, we’ve got a release coming up.

Calling Nix a souped up package manager is like technically correct maybe?

It’s ‘git reset —hard HEAD^’ for your whole computer or fleet of computers. It’s utterly fearless experimentation, it’s low/zero runtime cost isolation and reproducibility.

It’s early days ‘git’ for systems: pain in the ass to learn and use, frequently and credibly accused of being too hard for mortals, but profoundly game changing.

Whether Nix per se remains the plumbing, someone is going to do good porcelain and end DevOps as a specialization, along with Docker and Canonical and mandatory glibc nonsense and a thousand other things that have overstayed their welcome. Disks are big now, we can have a big directory full of hashes. We can afford the good life.

It’s going to be a big deal.

CMCDragonkaivor 4 Jahren

Nix is a programming language/system for software packaging. Along the way, it turns out OS configuration is also just packaging... As long as you don't have anything too dynamic.

It is the most sophisticated programming language/system for software packaging on the market today.

We use it for all our reproducible software development environments. All of our project templates. And all of our employees are issued NixOS configurations so everybody on the team has the same NixOS, the same Nix package overlay... Etc. This level of consistency ensures that we get a level of reproducibility from hardware to OS to user profile to development environment to CI/CD that just doesn't exist anywhere else atm.

It's not entirely perfect but that's not really Nix's fault. It's actually the fault of the rest of the software industry that persists with bad tech. All of Nix's complexity stems from having to package everybody else's mess. Anyway one day I hope the CI/CD systems in the world will just provide Nix jobs directly instead of proxying through a docker container image that I have to setup atm.

benreesmanvor 4 Jahren

Your shop sounds pretty ahead of the curve. “It works on my box!” “What do you know, mine too!” “Production person over here: I’m getting the same thing!”

It seems to me that people who go through the agony and ecstasy of getting it set up do so because the difficulty of the domain is high and the headcount is low: you need leverage in that setting.

Can you share anything about what motivated your group to get it dialed in?

CMCDragonkaivor 4 Jahren

My company matrix.ai was working a new cloud orchestration platform and Nix was core to how customers would package their applications/containers for deployment. The OS development is halted for now while we are working on a secrets management system.

So it was only natural to fully dog food Nix. We also introduced it to clients during our computer vision machine learning consulting work. It was the only way to get reproducible projects involving a complex set of Python dependencies, Tensorflow, CUDNN, CUDA, Nvidia libraries (there is a very strict set of requirements going all the way to hardware). I actually first tried doing it with Ubuntu and apt, it did not work. Setting up your own nixpkgs overlay is a must in these scenarios.

It is definitely something that is easier to fully dial in when you start from scratch. It's a comprehensive system so it will take time for adoption. I always recommend starting with it as a development environment tool first, then consider automating your OS conf or user profile or VMs... etc.

mvoodarlavor 4 Jahren

What was the orchestration system used for? Was it in the case where there were many models that needed to be run one after another. I know it's a huge problem in video processing to be able to increase speed a ton. My company Sieve (see profile) is building infrastructure specifically for running ML models on video which is why I'm curious.

CMCDragonkaivor 4 Jahren

It was built for AI driven container orchestration, configuration synthesis from high level constraints.

Yes ML workloads is particularly complex, because they have both batch oriented data flows (training), and service oriented data flows (inference). There aren't many systems that can adequately express both.

lillecarlvor 4 Jahren

In the more philosophical sense I think you're right. I've been using NixOS for 2 years now on all my machines* but I still don't know the language.

I use it as a stable base system, but when I need to do something that I'm unable to do with Nix I still drop into a $DISTRO container and do my work there where everything is stateful and "disgusting".

Having worked with DevOps for awhile I can happily tell you that we don't run around building Docker containers all days, at least in my team the developers do so. We provide them with a base to build upon, provide CI and stateful services like databases and storage.

I spend an awful lot of time writing Terraform and Helm charts though, since things needs to run somewhere.

But yes, the immutable nature of Nix is great. Nix was VERY helpful for me when I switched GPU's from NVIDIA to AMD on my desktop, no screen? Reboot, reconfigure, retry.

But yes, I agree. Something that resembles Nix will take over the computing world, as someone said somewhere in the comments of this post. The great thing about Nix is that when you fix something it stays fixed. Even if it was a PITA to get there.

benreesmanvor 4 Jahren

I believe that we’re in violent agreement :)

I used the exact same words “things stay fixed” elsewhere in the thread. Which is big psychologically. I’ll do some really difficult or even painful things in good humor if I get a lasting benefit. But if someone or something is just going to yank the rug underneath me next month? Fuck it, hack up some doofy shell script and call it a day.

Nix aligns the incentives on getting things right. I’m happy to learn the arcana of some weird corner case of some tool or service if I can apply that in a way that is permanent.

Now we just need someone to unfuck the UX nightmare :P

0x457vor 4 Jahren

> Same Dockerfile, same outcome, mostly every time.

Uhm, no? Dockerfile has tons of side effects:

- doing `apt-get update -y`? On some machines it will run, on others it won't be due to caching. - Using `FROM` that isn't locked to sha256? Well, sometimes you will get version 1.2.3 sometimes you will get 1.2.5. Sometimes a new one will get tagged with the same tag. - It literally has network access during the build, unless you include a hash of what you're downloading, there is zero guarantee it will be the same download.

I think the majority of leaf containers rarely get the same result with the same Dockerfile. The only thing that is guaranteed with docker is that the same image will be the same image, but ensuring that different machines pull the same version of an image is another story.

benreesmanvor 4 Jahren

I’m being a bit generous to Docker in the above comment: I believe that this is what users are hoping to achieve and getting closer to achieving than they would otherwise. Docker is basically a roundabout way to get static linking behind Drepper’s back. Almost no one is using it to bin pack such and such CPU/RAM to the request serving process and the batch processing process given finite SKUs.

Modulus the absurdly high barriers to entry, Nix is trivially better for this purpose.

kaba0vor 4 Jahren

I don’t think mixing up the two was parent’s point at all. He/she just made an analogy of where Nix currently is vs where should it go to “take over the world”.

[deleted]vor 4 Jahren

babyvor 4 Jahren

I think we’re used to tools using convention over configuration more and more, which is great, but as nix is trying to solve a different problem it is all about configuration, which makes it really hard to understand. Maybe it would help to have a GUI or a local webapp to sort of show the state of what you have and debug things… not sure.

natrysvor 4 Jahren

I don't see how that addresses OP's problem, which is poor and confusing documentation (I felt the same too). Everytime I hit a snag and searched for an issue, there were never a comprehensive and centralized source of solution, rather a bunch of pages each with different holes in information sprinkled with outdated advices. I share the feeling that I was reverse engineering things that should just be plainly documented.

babyvor 4 Jahren

Agree that documentation could be better, but as someone who has spent reading a lot of it I was still left with a sense of “how are things really glued together here in this nix project?”

infogulchvor 4 Jahren

> convention over configuration

Hmm I guess the nix approach would be "convention as shared configuration"

noidiocyallowedvor 4 Jahren

I'll translate. It's utter garbage bullshit.

benreesmanvor 4 Jahren

I’ve said it before: Nix is advanced alien technology that was badly damaged when it crash landed on Earth.

We’ve got a very nice setup that deeply integrates Nix/NixOS/Bazel (working name “straylight” for the Gibson fans) that builds most major languages well and rounds off the (very) rough corners. I’m dying to open source it but it takes time to detangle that kind of thing from your proprietary stuff. I paid in blood to know Nix inside out and I don’t want any more people to do that than already have. We use Tweag’s shit for some of the Haskell stuff.

Until then: if you’re ready to pay the price of learning this shit: you’ll get a favorable exchange rate by reading the source of nixpkgs. The docs/blogs are just usually wrong. There are exceptions (Xe is a baller), but mostly you’re flipping a coin whether or not an online resource will move you forwards or backwards.

Edit: Flakes aren’t an experimental new feature. They are a desperately needed fix to a badly broken status quo. Do not pass go, go directly to flakes.

runeksvor 4 Jahren

> Edit: Flakes aren’t an experimental new feature.

https://nixos.wiki/wiki/Flakes tells me to enable Flakes by adding “flakes” to the config option named “experimental-features”...

benreesmanvor 4 Jahren

You’re absolutely right.

Exhibit #902353 in “Why Everything About Nix on The Internet is Wrong”.

Haskell used to have a semi-official motto: “Avoid success at all costs”. I think they were joking.

Docs/UX that bad makes one wonder if the Nix people took that to heart. Flakes being some weird shoulder-surgery to even turn on? Why don’t we save time and just kick the ball into our own goal over and over.

Le sigh.

otabdeveloper4vor 4 Jahren

Too many corner cases don't yet work properly in flakes for it to be the default.

I think we can wait a few more months until they become stable enough.

benreesmanvor 4 Jahren

Educate me a little about the corner cases? I’ve long since adopted a philosophy I call “Kool-Aid Man Programming” named after ad advert when I was a kid of the character jumping through walls. If something doesn’t work I write my own derivation and patches if necessary, if the utilities for that ecosystem don’t work I write new ones, I just don’t take no for an answer from Nix. I find this to be faster and less stressful than engaging with maintainers.

Perhaps for more casual users it’s still a bad default? Even that doesn’t tie out with my personal experience.

A year ago flakes were already default amongst the serious pros, now if I don’t see a flake.nix I assume I stumbled into the old repo.

otabdeveloper4vor 4 Jahren

This thing, for example: https://github.com/NixOS/nix/issues/3121 Or the issues surrounding submodules.

benreesmanvor 4 Jahren

There are a lot of remarks on that issue, so maybe the plot thickens somewhere along the way, but if I’m reading that right, the corner case is that the first time a flake is somehow reified into the store, I’m hitting the filesystem with exactly what would happen if I ever referenced it elsewhere?

So in practice I’m eating the cost of “installing” it the first time I use it, which is subsequently amortized across all the other times I use it?

And if I’m really worried about this, my flake hits a default.nix in a directory that can also have a shell.nix I’m it so I can get around this?

I think Einstein once said: “However great your troubles with mathematics are: be of good cheer. I assure you mine are far greater.”

Nix isn’t relativity and I’m no Einstein, but if you work in Nix world and that issue even rises to your notice? Let’s trade jobs because I fucking wish I had Nix problems like that.

wocramvor 4 Jahren

How are you integrating bazel and nix?

My assumption is that bazel is running in an environment created by nix, so I wonder what you mean by 'deeply integrated'.

benreesmanvor 4 Jahren

The devil as always is in the details, but in the broad strokes we have a pretty uniform/composable way to float stuff from the Nix store into Starlark and thereby put all of nixpkgs within Bazel’s reach. Doing like custom Bazel cc_toolchain stuff against Nix wrappers is, uh, an adventure. But you grind away at it and now you’ve got the C toolchain you want. Wash/rinse/repeat for all the other languages you use and before you know it you’ve got Nix’s determinism and Bazel’s crazy fast build times.

When I was at FB there were dozens if not hundreds of serious hackers working on this sort of thing, and they were fucking good at their jobs, but technology usually moves forward with time and this setup smokes where FB was at in 2018. I don’t know what they’re up to now, I bet it’s good.

wocramvor 4 Jahren

I have spent a little bit of time working on a prototype of a setup like this, and have needed to write a lot of (hacky) glue and BUILD files.

I take it you have departed quite a bit from https://github.com/tweag/rules_nixpkgs ? Are you generating BUILD.bazel files for nixpkgs, or are you doing that by hand?

atom_arrangervor 4 Jahren

Learning about these kinds of things is one thing motivating me to try for a big tech company job.

I want to build my own company, but I want it to have this kind of tooling, that I don’t fully understand yet.

Sounds really interesting, would love to see it used more.

benreesmanvor 4 Jahren

I don’t know what the conventional wisdom is these days, but speaking for myself, I felt much more equipped to tackle the huge undertaking of being more entrepreneurial after seeing how a really serious software shop is run.

As for getting the big company job, it’s a weird time over like the last two months and the next two months, but this will soon pass.

Everyone hates leetcode interviews. I hated being interviewed in that format, and I hated interviewing in that format. It is what it is. It’s a test like the SAT or LSAT or GMAT or anything, arguably not as hard. It’s basically nothing to do with the job at the lower seniorities. I think people would be less psyched out about it if they viewed it the same as cramming for any other test. A high score on the LSAT doesn’t make you a good lawyer, it’s a loose proxy metric at best, and you game it by studying.

By virtue of a) hanging out on HN and b) hanging out in a thread about Nix you’re clearly curious and paying attention, so go get that marquee job, and take the experience with you when you start your venture! I’ll be rooting for your success!

tylerjlvor 4 Jahren

Scoped down to the problems that nix solves, I get the same vibes using nix as I did in the early days of docker. It solves painful problems, with a community and suite of tools large enough to be useful, and people who have paid the initial mental investment are happily using nix in lots of ways today.

But when you widen that scope to incorporate the rest of nix's warts, I'm not sure whether the project carries the requisite momentum to become a ubiquitous tool. I hope it does! The project has lofty goals and it really deserves a layer of porcelain to soften its usability and bring its benefits to more developers, because the actual, pragmatic results it delivers border on magic sometimes, and I wish more people were able to experience it. I don't blame people for avoiding that ramp-up time investement. I tried, gave up for many years, and only in the past six months or so devoted enough time to use it proficiently without training wheels. Not everyone has that luxury, nor should we in the nix community expect people to pay down enormous initial learning debt.

I began freelance consulting at the start of the year, and interesingly, more than one of my engagements turned into "let's solve this with nix", which is both surprising and encouraging. If the nix community can trim down the harsh thorns that encircle the language and tools (and provide rock solid learning materials), there's an actual chance that nix could really bloom in the near future.

nextosvor 4 Jahren

I love Nix, I use NixOS and associated tools in many projects and I maintain several packages on NixPkgs.

IMHO Nix is relatively easy if your usecase doesn't involve unpackaged artifacts that require heavy patching to fit them into Nix.

For example, I think it's one of the best distros to run a common Linux desktop setup because you can upgrade without fear and you can test new things without adding clutter.

The biggest hurdle is documentation. Writing documentation is not as sexy as implementing new features, but that's the biggest area to improve. https://nixos.wiki has recently documented many areas that lacked sufficient coverage, but there's a ton more needed.

microtonalvor 4 Jahren

For example, I think it's one of the best distros to run a common Linux desktop setup because you can upgrade without fear and you can test new things without adding clutter.

You can use Fedora Silverblue, which also offers atomic upgrades and rollbacks, offers support for regular RPMs as an escape hatch. It provides meaningful desktop security (verified secure boot, kernel lockdown mode, SELinux, some application sandboxing in Flatpaks). But you don't have to learn a functional language and ecosystem that is completely foreign to most people.

Oh, and Steam doesn't break all the time :).

I think Nix is great, but I wouldn't recommend most desktop users to use NixOS on the desktop, unless you are willing to invest a lot of time.

grumbelvor 4 Jahren

> You can use Fedora Silverblue, which also offers atomic upgrades and rollbacks

The beauty of Nix with Flakes is not just upgrades and rollbacks, but that it can be completely stateless. You can just run software directly from the Git repository:

   nix run github:JUser/your_app

And if you want an old or newer version, just go:

   nix run github:JUser/your_app?rev=01f352202ad

You can even override dependencies if you like:

   nix run --override-input somlib git+file:///yourlocalfork github:JUser/your_app

There isn't a whole lot of software that has turned their Git repository into a Flake yet (i.e. added a flake.nix), so that benefit is a bit theoretical for the average user, but the potential to completely reshaping the Free Software landscape is certainly there, as Nix finally allows upstream to become a first class citizen in the distributions packaging system.

microtonalvor 4 Jahren

There isn't a whole lot of software that has turned their Git repository into a Flake yet (i.e. added a flake.nix)

As long as the project developers are not going to use Nix themselves, there are going to be a lot of unmaintained, broken flakes. All dependencies are pinned, but the project itself may move on from what is specified in the flake. A bunch of my own projects were flakes [1], but they would regularly break, because the project would move on (e.g. requiring a newer version of libtorch), but I forgot to update flake.nix/lock.

Of course, you can still run historical versions, but when the main path doesn't work, many users will just move on. So, including Nix flake files becomes more of a liability to a project than an asset.

(Besides that, it probably fills your store with many, potentially vulnerable versions of glibc and whatnot due to use of many different nixpkgs revisions, a problem that the centralized nixpkgs package set doesn't have.)

[1] I even implemented/contributed the cargoLock functionality, so that you can use buildRustPackage (rather than an external solution) without needing to specify the Cargo vendoring hash to make it easier to use buildRustPackage in your own repositories:

https://nixos.org/manual/nixpkgs/stable/#importing-a-cargo.l...

Bolkanvor 4 Jahren

> because the actual, pragmatic results it delivers border on magic sometimes

I was able to run 10 years old chrome builds on latest nixos using nix. Windows users might read this and go meh but linux people know how big of a deal that is.

ixxievor 4 Jahren

Fix the learning curve.

I love Nix and NixOS and have used them for years, but it always feels like I am stuck in beginner level when it comes to understanding the language.

The biggest obstacle is pedagogic. I think big wins can be make by creating an up-to-date, deep-dive learning resource. I get the suspicion that one must learn Nix in a somewhat different way than one learns other languages. But I don't know what this method should be, and I wish somebody showed it to me.

Admittedly, some of the difficulty relates to the Nix language itself. I got excited when I saw a gist [1] by Eelco Dolstra that 'naturalize' aspects of the Nix language by bringing them down a level of abstraction. I am disappointed this isn't discussed so often in the community. Making configurations and packages explicit parts of the language - as Eelco suggests - is certainly going to help us mere-mortals understand what is going on in Nix. But syntactic improvements that makes Nix feel more like YAML are also huge: familiarity is a cornerstone of intuitiveness.

Please twaeg - make this happen!

[1] https://gist.github.com/edolstra/29ce9d8ea399b703a7023073b0d...

mboesvor 4 Jahren

We're onto the pedagogy thing. Check out the Nix book efforts: https://discourse.nixos.org/t/documentation-team-flattening-....

Regarding the language and "configurations" specifically, you might like what we do with Nickel: https://github.com/tweag/nickel. Research project showing a potential future for Nix.

shashurupvor 4 Jahren

Interesting. I don't find the Nix language itself especially complex, however, the whole system built with it is not that easy. For instance, packaging process is very well documented, making a derivation looked simple and reasonable until I decided package some code for myself. At this point things stopped being smooth. According to the documentation I needed to explicitly import my nix file from the top-level list of all packages which was not what I wanted, I'm unsure anybody except me would want my package (at least at the early stage of development). Intuitively, I was sure it wasn't necessary, there definitely must be a simpler way to toy with packaging but skimming through the documentation didn't reveal any answer. Looking at the source code left the impression that I needed a lot more time to understand how things worked behind the hood. On the other hand replacing pyenv with Nix was very pleasant experience. Overriding an existing package with my fork on github was almost a one-liner.

runeksvor 4 Jahren

> According to the documentation I needed to explicitly import my nix file from the top-level list of all packages […]

Can you elaborate on this? I don’t follow.

pxcvor 4 Jahren

The Nixpkgs packaging guide describes how to use the Nixpkgs packaging tools in the context of contributing to Nixpkgs, where to populate the package set with your custom packages, you add them on a top-level attribute in pkgs/top-level/default.nix.

But a lot of users want to leverage Nixpkgs while instead (or simply first) adding their custom packages to an overlay or a company repo of Nix packages.

shashurupvor 4 Jahren

From the [1]:

> 3. Add the package to the file pkgs/top-level/all-packages.nix. The Nix expression written in the first step is a function; it requires other packages in order to build it. In this step you put it all together, i.e., you call the function with the right arguments to build the actual package.

[1] https://nixos.org/manual/nix/stable/expressions/simple-expre...

crabbygrabbyvor 4 Jahren

I'm not so sure making nix comprehensible is realistic. When technologies fall under this level of complexity they require die hard fans to use it and keep it alive In all honesty, I would consider a complete rewrite at this point. I don't think there's a lot of hope for nix right now outside of hobbyists and researchers. Not saying this to be rude, but genuinely, take what you all have learned, take a sober look at the state of it, improve it. Almost no one will be willing to cope with it in it's current state docs or not. If it was the next docker/podman/etc you'd of known by now... It's not getting there, revert course

WickedSmokevor 4 Jahren

Recently someone has been trying to build a Nix package of an open source game I maintain, so I had a brief look at the project.

It does appear to be an experiment in package management that got out of hand. Basic usability is lacking, as I was unable to even get a listing of available packages on a NixOS virtual machine due to nix-env requiring well over 1GB of RAM. This flagrant waste of resources doesn't inspire confidence in the quality of the tools.

I'd rather see existing packaging data leveraged rather than adding a new language to the mix. Something like a translator or interpreter for RPM .spec files, for example.

jbboehrvor 4 Jahren

Listing all packages is a bit of a weak spot. The entire package set is described in a lazy, functional language and listing or searching it requires evaluating the entire thing. A normal build will only require evaluating a small subset of the entire package list. `nix search` should have caching of results but it still requires evaluation to be populated.

otabdeveloper4vor 4 Jahren

> outside of hobbyists and researchers

No data on this, but I'm pretty sure Nix is used mostly by for-profit companies trying to tame complexity on a limited budget.

It's not nice or pretty, but there's very little alternative unless you're ready to keep a whole team of developers just fixing Dockerfiles all day.

crabbygrabbyvor 4 Jahren

I've used it in industry and it was nothing short of a disaster. Only one person could fix builds, despite many genuinely brilliant engineers putting in hours to learn it. More so, the time sink... Literally, what takes 29 minutes for a fresher to do in docker takes them two weeks with nothing to show for it in nix. Absolutely do not recommend it for industry whatsoever. Most people, except the one advocate for it, see it as a failed experiment.

otabdeveloper4vor 4 Jahren

If you work in a place where you have teams (multiple) of developers doing Docker crap, then nix is probably not for you.

To each his own, it targets an entirely different niche.

ghuntleyvor 4 Jahren

See https://nix.dev

norman784vor 4 Jahren

I think nix could learn a few things from rust (maybe I’m out of date with nix, it was a while back when I tried it).

Rust has few things that makes it outstanding against other languages (in this context in particular):

- Tooling: I think nix needs an lsp (if isn’t any yet).

- Static typing: In this particular case might work better the typescript route, where you can gradually add typing (to not break old nix code).

- Docs.rs and documentation: In particular it is very powerful to have a centralized documentation that isn’t ambiguous and also this forces the packages to document themselves, so you could access that documentation directly from the source code and the tooling in your text editor.

This is just something that for me will make it easier to use, I think the days where you write text without help are over, it is much better IMHO work with rust because those points (there are few others, but besides those) for me is so important, and also ease the learning curve of nix language in general.

The other thing is the ecosystem, iirc you need some other packages or helpers to make nix work, there are also cases where you want to setup something and the official docs tell you that you need some package first, then you go a step back and try to figure out what’s that, and the same if you want to install another package, it requires you another package to setup first, I don’t remember exactly but in macos you needed some packager or darwin packager, I really don’t remember, and that was a huge set back, besides other issues with newer macos changes that did nix a PITA to install correctly.

notemakervor 4 Jahren

re LSP, there is https://github.com/nix-community/rnix-lsp

norman784vor 4 Jahren

Nice, thanks for sharing, it seems that they are working towards solving some of my pain points with nix.

babyvor 4 Jahren

I need something that works with vscode

capablewebvor 4 Jahren

"But it does work with vscode, check the README"

"I need something that does X"

"But it already does X"

"But what about Y?"

On a more serious note, the linked project really does work with Visual Studio Code, do check the README of the linked project and you'll see :)

babyvor 4 Jahren

I spent like a month trying to make it work with VS Code, it does not (at least on Mac), and that's what most programmers use nowadays.

capablewebvor 4 Jahren

Did you check existing issues / open a new one describing your issue?

lijogdfljkvor 4 Jahren

Agree completely. My primary OS is NixOS, and i use it on my Mac laptop as well. I love Nix, but i absolutely hate Nix Lang's "typeless" user experience. Ie i can't look at some config, in my editor, and have any clue what it does. It's seemingly purposefully obtuse.

I use it, but i hate it, but i'll never leave it. I really hope for advancements in either the LSP or a sane type system (Nickel? Perhaps) to replace NixLang.

I think modern Nix (Flakes, modern CLI, etc) would be nearly flawless if i could get good inline information on each field, types, etc. When Nix Lang has as many language features as Rust, ie docstrings, types, etc - i think i'll love it. Until then it's an abusive relationship for me hah.

smasher164vor 4 Jahren

Nowadays, every one of my projects starts with a flake.nix file. I even have templates (https://peppe.rs/posts/novice_nix:_flake_templates/) fo archetypes of projects, like a Go, Rust, GTK project, etc...

I love that flakes allow you to decouple a package from nixpkgs and place it in your repo, but in order to fully realize this dream, there needs to be a proper index of these packages, as well as non-experimental support for flakes.

Another (pipe) dream of mine is that Nix gets true cross-platform support, including non-WSL windows. This would allow it to fulfill a role that doesn't currently exist, which is a cross-platform package manager and build system that works across language ecosystems.

woilevor 4 Jahren

That sounds interesting. I have 0 experience with nix, could you explain what's the difference with doing `cargo new <name>`? What extra benefits do you get?

Thanks!

the_gipsyvor 4 Jahren

That gets you some Cargo.toml/lock files, from which you can build on another system. But there are some system dependencies, like dynamic libraries, which are not described nor pinned. Nix forces you to describe 100% of the build/run inputs.

rgoultervor 4 Jahren

Ah, in terms of just instantiating a template, the nix flake templates are going to be oriented around using nix.

A flake.nix file is roughly comparable to a Dockerfile. With a Dockerfile, you're describing how to build an image that can be deployed in Docker runtimes. With nix, you're describing how nix can build the package.

A neat side-effect of this is that if you've got a nix file which can build the package, nix will be able to provide the toolchain used to build the package. (Whereas rustup more specifically provides the rust toolchain).

dfeevor 4 Jahren

I’m not sure what it is, but you’ve got some odd font-size bugs going on.

smasher164vor 4 Jahren

Oh that page isn't mine, I just found it as a good explainer for nix templates.

maartenhvor 4 Jahren

I love Nix. I use it on my homesteaders, and private laptop.

I have seen how it works great in a team, if all are onboard.

No place I've worked at was interested in the advantages of Nix though.

I wonder if switching to something less recognizable to a functional programming language might help adoption.

The approach Starlark (the DSL for Bazel) has taken, is have mutable datastructures, but freeze them after they are evaluated. This seems a lot more explainable and debuggable for the average programmer.

Infinite recursion errors are not fun to debug in Nixlang.

At my current job, NX is used to be a kind of Nix, but it does not seem to try to solve the entire problem space. Just to solve the additional complexity that introducing a monorepo in regards to tracking build dependencies.

It seems to me that how awesome Nix is from a tech perspective, it doesn't make delivering actual software any easier without team buy-in and effort in integrating in their delivery pipelines.

noelwelshvor 4 Jahren

The problems with Nix have nothing to do with FP per se. The problems with Nix start long before you get to any programming. Take the home page. The first two sentences on the hero image:

1. "Reproducible builds and deployments." OK, this is exactly Docket etc. claim. No differentiation here.

2. "Nix is a tool that takes a unique approach to package management and system configuration." Uh, why should I care? What problem is this solving? What's the benefit to me?

And it continues throughout Nix. E.g. advice in this thread that the only sane way to use Nix is with Flakes, which are some semi-documented experimental feature of Nix. It's just terrible user experience right from the start. Just read the thread.

maartenhvor 4 Jahren

I agree that the FP part is not the only issue. It's that the community feels a bit more academic/I'll fix this for myself in the way that works best for me.

You can indeed achieve some reproducibility with Docker. It's tricky though, as you'd have to pin exact package versions of software. If you'd `FROM ubuntu:$VERION`, and would run an `apt-get update`, you're not guaranteed to get the same software.

Nix is like ZFS, as that it breaks the wall between two previously distinct area's. Those being building software, and installing/configuration software on your OS. It's quite different from the snapshot-everything methodology that Docker uses. Yeah, one can split in multi-stage images etc, but than you'll be keeping track of which dependencies need to be moved between the stages yourself, in a manner that cannot be abstracted away, so you're doomed to repeat the same patterns over and over again.

People also state that LVM + ext3 is more than sufficient compared to the complexity of ZFS. They miss out on the fact on how much more fine grained solutions are possible with ZFS.

I've used niv [0] before flakes arrived, and am actually still using that instead of flakes. The experimental nature of them has scared me away from them, as I'm not daily involved in this ecosystem at the moment.

[0] https://github.com/nmattia/niv#niv

toomanydoubtsvor 4 Jahren

Not to even mention that you just went into nixos.org and now it's talking about Nix(not NixOS).

ParetoOptimalvor 4 Jahren

> 1. Reproducible builds and deployments.

Docker doesn't give you reproducible, it gives you repeatable 90% of the time.

> 2. "Nix is a tool that takes a unique approach to package management and system configuration."

> Uh, why should I care? What problem is this solving? What's the benefit to me?

Because #2 enables Nix to actually deliver 100% reproducibile builds and deployments.

crabbygrabbyvor 4 Jahren

Complete opposite experience. Even with engineers who knew their stuff. It's been more of a burden in my opinion then a tool. Just use GitHub actions and docker.

ghuntleyvor 4 Jahren

See also "Tvix: We are rewriting Nix" - https://tvl.fyi/blog/rewriting-nix (discussion at https://news.ycombinator.com/item?id=29412971)

pxcvor 4 Jahren

I haven't seen any action related to Tvix in their repo in a long time, either in finished commits or merge requests. Is there any ongoing work on that rewrite at the moment?

I guess there are some related efforts that are looking to share components as well: https://riir-nix.github.io/

zimbatmvor 4 Jahren

There has been a bit of a pause while tazjin was exploring how to use Nix as a monorepo build system, together with Gerrit. Now that this is done, he recently started writing a new Nix interpreter that will better scale for big repos. I don't think that work is public yet.

There also has been some effort on https://github.com/nix-community/go-nix spearheaded by flokli to re-define some of the binary cache protocols. This will allow for more incremental transfers of build artefacts, reduced bandwidth and disk consumption.

pxcvor 4 Jahren

> new Nix interpreter that will better scale for big repos

That makes a lot of sense for TVL and could be a valuable characteristic to a of other organizations as well. :)

> I don't think that work is public yet.

I'd hoped there was something like that going on!

> There also has been some effort on https://github.com/nix-community/go-nix spearheaded by flokli to re-define some of the binary cache protocols. This will allow for more incremental transfers of build artefacts, reduced bandwidth and disk consumption.

Cool!

zimbatmvor 4 Jahren

Update: new code started landing in https://code.tvl.fyi

Cosi1125vor 4 Jahren

After yesterday's post about Nix I tried to install it on my Arch+fish machine. I couldn't.

Somewhat naively, I assumed that installing it from the official channel would do the job. It didnt't. I then searched the Internet and found a very "creative" workaround for Arch users that didn't work either.

My own problems aside, I would expect from a tool marketed as something that helps you keep your environment clean to not litter my disk with a ton of hidden files and directories spread all over the filesystem.

I hope it doesn't sound too harsh. I'm keeping my fingers crossed for the development of this amazing project!

rgoultervor 4 Jahren

> My own problems aside, I would expect from a tool marketed as something that helps you keep your environment clean to not litter my disk with a ton of hidden files and directories spread all over the filesystem.

On non-nixOS, I think the changes Nix makes are:

1. puts the majority of its stuff under /nix 2. in the user directory, puts some files under ~/.nix-channels, ~/.nix-defexpr and ~/.nix-profile 3. modifies the default shell script /etc/bashrc or so, with some stuff under /etc/nix. 4. adds multiple users nixbuild1 ... nixbuildN.

The uninstall command given in the manual is a one-line:

``` sudo rm -rf /etc/profile/nix.sh /etc/nix /nix ~root/.nix-profile ~root/.nix-defexpr ~root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels ```

SuperSandro2000vor 4 Jahren

The files in the user directory are mostly symlinks and there are only multiple because it was designed without xdg in mind years ago. There is an open RFC to change those directories to xdg compliant ones.

Thorentisvor 4 Jahren

Obscene.

slurpmakervor 4 Jahren

Don't worry after you install it it doesn't exactly get any better. Take this as a sign :P

forgotpwd16vor 4 Jahren

> I couldn't.

Did you tried the method shown in Nix's site?

https://nixos.org/download.html

>I assumed that installing it from the official channel would do the job. It didnt't.

"Official channel" you mean from repos? That's a problem of your distro's package then, isn't it?

Cosi1125vor 4 Jahren

As I said, I've tried installing it from the Arch repo and using Nix's installer (+ the adjustments described here [1]).

[1] https://github.com/NixOS/nix/issues/879#issuecomment-4109043...

SuperSandro2000vor 4 Jahren

The rsync requirement got recently removed.

pxcvor 4 Jahren

> naively, I assumed that installing it from the official channel would do the job.

Years ago, I submitted a PR that added this exact behavior. But it depended on interpolating a Nix-internal variable into a config file in the Fish package which at the time was undocumented (@NIX_STATE_DIR@) and upon the layout of profiles within it, so there was concern that it might break on future Nix releases. There were a few other issues, too, and at the time I don't think there was a great way to add tests, either. (That's important because if you screw this up, you can break login for Nix users— I know because I did it once, and that was awful.) Anyway, I felt discouraged and didn't know where to go with it next, so I got lazy and dropped the PR.

At any rate, I don't think your expectation there was unreasonable. I think that would be a good and sensible UX, and your intuition was reasonable.

> After yesterday's post about Nix I tried to install it on my Arch+fish machine. I couldn't.

So the problem is this: the official Nix installer leverages an initialization script, to be sourced at shell init time, which is written in POSIX shell. Fish can't just source it because it's not compatible with POSIX shell.

> I then searched the Internet and found a very "creative" workaround for Arch users that didn't work either.

Arch users shouldn't need anything special here.

You can use fenv¹ to source it, you can translate it to bash using babelfish, or you can exec into fish after running bash to log in (make sure to set $SHELL after if you choose that option). Just make sure you do it as early as possible if you want to use things installed by Nix in your other Fish config snippets.

I think Home Manager will also take care of this for you. (The other module systems, Nix-Darwin and NixOS, both do.)

If you wanna do it like NixOS does, you can install Fish via Nix and then create your own preinit environment script in /etc/fish. This has the advantage of setting up your Nix environment variables before any other config is sourced. You can take a look at it here, which also explains some of the Fish initialization process: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/p...

and here's where the hooks go in the Fish package, where the comments describe the Fish initialization process in detail as well as why things are done this way on NixOS: https://github.com/NixOS/nixpkgs/blob/master/pkgs/shells/fis...

—

1: https://github.com/oh-my-fish/plugin-foreign-env

2: https://github.com/bouk/babelfish

Cosi1125vor 4 Jahren

Thanks for the advice! I'll probably give Nix another try, rather sooner than later :-)

pxcvor 4 Jahren

> We also intend to regularly publish a shorter-term roadmap on discourse alongside our usual dev updates. So stay tuned!

Definitely check out the Tweag development update posts on discourse.nixos.org if you're wondering who they are! The update posts are great, and they give you a clear sense of what Tweag's commitments are, how much they regularly contribute to Nix and the wider ecosystem, and the way they conduct software development out in the open.

emersionvor 4 Jahren

"We want Nix to ne ubiquitus"

And there it is. Every time I hear about Nix, the goal is to convince me to adopt it, as if it was some kind of religion. Every time the Nix folks are being too pushy about it.

Just stop pestering random project maintainers with Nix. If a project maintainer likes Nix, they'll add the Nix files. If they don't have Nix files, that doesn't mean they want to sign up for Nix advertising.

blueflowvor 4 Jahren

I agree. At $dayjob, we are currently employing a nix consulting company comparable to the one in the link. They are very vocal about all the great things Nix is able to do, but when we actually attempt to do something in Nix, we run into countless bugs and problems, including inaccurate documentation.

Unfortunately many private FOSS enthusiasts do not know that reliability and documentation is a must for serious engineering environments.

Plus, a coworker pointed out that while its supposed to be pure and functional, many recipes still basically wrap a shell script. So its not that far from the build system we already have...

microtonalvor 4 Jahren

many recipes still basically wrap a shell script

Not defending Nix here, but using shell script as part of the build doesn't make Nix itself imperative/impure. As long as the evaluation of the Nix expression itself is pure [1], the build inputs are fully specified, and the build is performed in a sandbox, the shell script (and things like configure/cmake/make/ninja) should always be run in the same manner and result in the same output (modulo things like embedded time stamps in files).

[1] This hasn't traditionally been true, since you could e.g. read files or use environment variables from Nix, but this is something flakes address.

rgoultervor 4 Jahren

> So its not that far from the build system we already have.

Yup.

The key contribution Nix brings is arranging the outputs of builds into the nix store, in such a way that you can trust that building a package with the same set of inputs will result in a package that behaves the same way. (This then allows e.g. different versions of the same package).

There's some friction, since with some software, if you try and build things but disallow writing to arbitrary files (e.g. no writing to $HOME), the tools might have assumed they could, so things might not work. -- That's 'purity' in some sense, in that what inputs the package building uses is totally declared.

I liked the analogy introduced from a tweag post from a couple of weeks ago https://www.tweag.io/blog/2022-07-14-taming-unix-with-nix/ (big discussion yesterday https://news.ycombinator.com/item?id=32355327) that illustrated nix as 'pure', but interacting with the Unix system by way of files. -- You have a pure way of managing an impure system. (Which is loosely comparable to how Haskell can be pure, but still do impure stuff like Input/Output).

_hl_vor 4 Jahren

> But when we actually attempt to do something in Nix, we run into countless bugs and problems, including inaccurate documentation.

This is really unfortunate. Unless you invest time into learning which paths to walk, you're not going to have a great experience with Nix. That just doesn't scale to modern engineering teams, you can't expect everyone to know everything.

> Plus, a coworker pointed out that while its supposed to be pure and functional, many recipes still basically wrap a shell script. So its not that far from the build system we already have...

It's true that a lot of nix expressions boil down to a shell script. But by default they're running in a sandboxed environment that can be exactly replicated and guarantees (mostly) reproducible results. And that's great - after some ceremony to set everything up, Nix lets you work with the tools you already know, but puts them into a reproducible environment.

pxcvor 4 Jahren

> Plus, a coworker pointed out that while its supposed to be pure and functional, many recipes still basically wrap a shell script. So its not that far from the build system we already have...

The shell scripts' effects get frozen in the Nix store, though, and can't be touched after build time. So it's a bit like using a loop in the implementation of an interface that that returns an immutable value in a language like Scala or something.

crabbygrabbyvor 4 Jahren

Honestly switch out while you can. Sunk cost fallacy is a hazard for business. Could wrack up a million in consulting fees for a subpar tool here.

ameliusvor 4 Jahren

> as if it was some kind of religion

I don't know of a religion where you have to go through hell before reaching heaven.

classifiedvor 4 Jahren

git, for instance, became mainstream in spite of that.

LunaSeavor 4 Jahren

Christianity?

SkyMarshalvor 4 Jahren

I love NixOS, use it on all my workstations (and Nix on my Mac) and could never go back to a normal Linux.

But I agree, they need to drop or reword that as an official goal. I suspect what they're thinking is that they want it to become so good and more broadly accessible to a wider range of developers that it gets natural usage and traction and becomes ubiquitous organically.

But they make it sound more like they're missionaries proselytizing a religion, ostensibly for the audience's benefit, but really more for the religion's benefit. Or salesmen selling something by telling you how good it is for you, when it's really better for the salesman. People pick up on that and reject it.

Of course, they should keep trying to make Nix ubiquitous by making it really good and broadly accessible and by engaging with the dev community for ideas on how to do that, as they outline in this post. Just don't make ubiquity an official goal. Let that happen organically.

cmmvor 4 Jahren

who addressed to? who's pestering you?

wlpuvor 4 Jahren

Nix will never achieve any wide spread adoption, the issues are too glaring.

The Nix language itself is half of the problem, difficult to search for because "nix" doesn't refer to something specific as it's overloaded, what hubris there must of been to think that was a good idea, it's also terribly documented with too much of a focus being on "elegant code" instead of being clear. Java and Go obviously being too verbose, but you could use a subset of python like starlark to achieve clarity and readability.

This being one of many but I've lost the will to continue as Nix as a dev experience is abysmal and really something else needs to replace it because I don't the language being fixed anytime soon. Guix will not be the on either due to FSF dogma.

People can go on about having successfully adopted it at their org but simply put, if it was good enough as is the percentage of companies using it would be meaningful. FYI a little script that only a minority of the devs understand does not count.

ParetoOptimalvor 4 Jahren

> FYI a little script that only a minority of the devs understand does not count.

Most devs don't understand or care about the build process as long as it works and they know how to poke a stick at it in my experience

wlpuvor 4 Jahren

Someone has to be able to maintain it. What happens when the one person who understood it leaves and it needs to be modified or upgraded, if they're lucky they can hack it into working and even that's not a good thing as the build has now been modified in a way that is not understood and possibly broken. If they can't figure it out and the need is not that strong they'll leave it as is and just not change whatever they originally had wanted to, ultimately limiting themselves and their solution in someway. The last choice for them is do the only thing they should when there are components that are not understood or can't maintain and rip it out and in the process lose many days or weeks replacing it.

ameliusvor 4 Jahren

The problem with Nix is that it doesn't properly encapsulate imperative behavior (read: most existing software).

For example, running pip (python package manager) fails because it tries to change the current installation environment behind Nix's back. Instead, Nix should allow it to run, but monitor any changes it makes in a functional way. This may require running programs inside an emulation environment (catching all the kernel calls that make modifications), or perhaps use a specialized filesystem, which sucks, but you can't have a free lunch here.

henrydarkvor 4 Jahren

The problem of figuring out the dependencies of a package given that is was installed by an external package manager is unsolvable. For the pip example, you would have to wait for all runtime possibilities to occur to know what dynamic libraries can be loaded. It's not even clear how the main library should find these given that it probably has /usr/lib/libwhatever.so.13 hard coded inside - should nix try to intercept the syscall for this when it happens?

The way I go about this these days is to keep a nix derivation that builds a container image, and then I run pip installs freely in containers that I spin up. I use mach-nix.

If a package is deemed important, then I add it in the derivation. So there is a kind of flow from "runtime" to "build".

ameliusvor 4 Jahren

> should nix try to intercept the syscall for this when it happens

Yes, this is exactly how it should work, imho.

ajbvor 4 Jahren

I think the old vesta system used to do something like that: https://web.archive.org/web/20151018150600/http://www.vestas...

fowlievor 4 Jahren

I'm a senior software developer and experienced unix user, and I loved the nix principles. So I installed nixOS on my laptop and it worked like a charm! But after a few months those complex corner cases came up and I was forced to write my own packages. At that point there was no help reading the docs, so I gave up and deleted the whole OS. So I really hope things will improve, and if they do I'd give it another shot.

encryptluks2vor 4 Jahren

Playing devil's advocate, I think that these may be some great goals for a small team but I don't see how their roadmap really solves their intention. It is like Microsoft saying they want everyone to use Windows so they are going to add graphics support to the Windows Subsystem for Linux. People already using Linux aren't going to switch to Windows for that, unless they have to for a job.

People aren't going to switch to Nix unless they have a reason to, and frankly what they are proposing won't capture a lot of users. Maybe some Mac people who want more packages available, but that is about it. I like the idea of Nix, but it is going to take a lot more to get people to adopt it. Nix packages are ugly. Rather than build the API around package management like winget or to some extent pacman, they instead added a bunch of scripting so each package ends up looking like a clusterfuck. A lot of the features aren't even documented well.

otabdeveloper4vor 4 Jahren

Okay, but Nix isn't a package management tool.

yjftsjthsd-hvor 4 Jahren

What? Nix is exactly a package management tool (and build system, and language, and Linux distro). And when people refer to nix, they usually also mean nixpkgs at the same time.

babyvor 4 Jahren

Advice to nix devs: do a nixbyexample website. These are what attract the most people to a language. See

https://o1-labs.github.io/ocamlbyexample/

https://gobyexample.com/

https://doc.rust-lang.org/rust-by-example/

Also if it’s not on learnXinYminutes it should be

capablewebvor 4 Jahren

> These are what attract the most people to a language

You got any hard data to back this up? None of the things I've personally picked up, have been because of the existence of such websites, but instead because of practical reasons, being able to do new things other languages couldn't (as easily).

babyvor 4 Jahren

They're hugely popular and often cited as how people learned the languages

babyvor 4 Jahren

I guess I'll start: https://mimoo.github.io/nixbyexample/basics-setup.html

notfedvor 4 Jahren

> Also if it’s not on learnXinYminutes it should be

https://learnxinyminutes.com/docs/nix/

ruudavor 4 Jahren

One thorny issue that everyone using Nix on non-NixOS eventually runs into, is the need to set LOCALE_ARCHIVE. Getting rid of that impurity would be a big step towards "works reliably, everywhere", especially for development environments.

soraminazukivor 4 Jahren

LOCALE_ARCHIVE is required because you would want the locales to be looked up during runtime, not during package creation. So AFAICT removing it isn't an option. You can avoid problems by using the versioned variables though (e.g., LOCALE_ARCHIVE_2_27). If you use Home Manager, it would do the right thing for you.

darthrupertvor 4 Jahren

If somebody designs a system that does 90% of what Nix does and by stripping the hardest 10% manages to make it usable, that thing is going to be incredibly successful.

otabdeveloper4vor 4 Jahren

"90% of what Nix does" is a very different subset for different people.

indeyetsvor 4 Jahren

It's a bit sad, that "To get there, we plan to:" section consists only of marketing efforts, not R&D

soraminazukivor 4 Jahren

Nix is community project, and major changes have to be agreed upon before they get anywhere. Indicating a long term goal makes a lot of sense given their style of development.

pxcvor 4 Jahren

Also marketing here includes substantive work like coordinated documentation efforts. It doesn't just mean 'evangelism'!

rgoultervor 4 Jahren

I think these a a good set of medium/long term goals.

"flake.nix in every repo" seems a bit of a chicken & egg thing relating to "nix should work reliably everywhere". If nix runs into problems with common situations, it's less likely there'll be a flake.nix. But, if it's rare to spot a flake.nix file, it's harder to be interested in nix.

Rather, I think most of the comments I see from people who are less than happy about nix as-is relates to friction from trying to use nix, but things go wrong.

babyvor 4 Jahren

IMO nix should not get in places where package management is done correctly (rustup/cargo), but definitely should when it's not done correctly (OCaml, C, python, etc.)

mplanchardvor 4 Jahren

We use nix for our rust projects, for the following:

- consistent dev environment with all local dependencies (postgres, rustc, awk, etc.)

- building rust packages themselves

- building tiny docker images that run rust binaries

The official Rust tooling for nix still uses Cargo under the hood, and this makes it way easier to use and more consistently functional than the equivalent tooling for Python, JS, and similar, at least in my experience

deltasevenninevor 4 Jahren

The critical part of nix that the developer failed to notice is user friendliness.

Sure I love to program, but that doesn't mean I want to program every aspect of my OS just so I can start writing a web app.

If you increase user friendliness of the app with a GUI or easily understood commands not only will more regular people start using it, but MORE developers will start using it too. Many developers just want to work on their own project with an OS that just works.

Hence why more devs use mac and windows over linux.

Hnusvor 4 Jahren

If I were to speculate I think the absolute brutality of initial learning curve filtered anyone who perceives the usability as a problem so it never got improved.

di4navor 4 Jahren

No, the sad reality is that doing UX change are really hard and need deep change to the nix codebase. The nix codebase is more than unwieldy plus most people interested in UX... do not get paid to work on nix. Quite simply.

colinsanevor 4 Jahren

a good deal of users who overcome the usability issues of nix remain ill-equipped to contribute to nix beyond basic packaging. just because i can navigate Linux userland doesn’t mean i have a clue how to navigate the kernel, is i hope a not-terrible metaphor.

pxcvor 4 Jahren

Nixpkgs and NixOS also some prolific contributors who never had any prior functional programming experience, so clearly there's a way out. We just have to capture and reproduce that.

SuperSandro2000vor 4 Jahren

I still can't really program in a functional programming language.

pxcvor 4 Jahren

Exactly. You've learned what you needed to in order to be productive and achieve your goals with Nix, and that hasn't required you to stop everything to take a crash course in functional programming.

Individual functional programming techniques may become interesting to you as you become curious about some implementation details— you might read a bit about 'fixed point recursion' after you chat with some people about how overlays work. But you don't have to think of yourself as a functional programmer to make great use of Nix or even to help solve technical problems in Nixpkgs.

I think some folks hear all the FP talk surrounding Nix and don't realize that there's a lot they can achieve and contribute without being FP wizards. And that sucks. I hope that the ecosystem feels increasingly welcoming to people like that as the documentation and UX continue to improve.

pxcvor 4 Jahren

> If you increase user friendliness of the app with a GUI or easily understood commands not only will more regular people start using it, but MORE developers will start using it too.

Making a GUI for generating Nix config is a complex task (and maybe not a better approach than providing a highly discoverable, user-frindly IDE experience, like DrRacket or something), so nothing like that is complete yet. But there are efforts along those lines that are still actively being worked on.

NixOS has also recently seen the addition of a graphical installer, for reasons like you describe. It's nice to see that kind of progress.

IshKebabvor 4 Jahren

Doesn't the fact that it's pure and functional make it easier to do a GUI for? I dunno, I'm one of the people that loves the idea of Nice but was put off by the complexity and beginner unfriendliness.

IDE support (code intelligence etc.) is always good though.

oxffvor 4 Jahren

> we see that nearly all of our clients that are using Nix are, at best, annoyed by the accidental complexity that comes with it

That is because it is an exemplary in accidental complexity at least from a user point of view. I cannot in good faith introduce it to an organization as it is way too complex to use / configure, only thing with worse documentation I've seen was PHP (this in itself a big red flag).

bornfreddyvor 4 Jahren

What are you talking about? Of all the problems PHP has (and there are many), the documentation is not one of them. It (php.net) is complete, easy to browse or use as a reference, mostly accurate and even includes comments where community with good intentions (but admittedly, usually less good knowledge) helps clarify any corner cases. Which you can ignore of course.

The language itself sucks (and this bleeds into the code), but the idea of it and the documentation are great imho.

Kaze404vor 4 Jahren

Do you still need to scroll down to the comments to find whether the function you're looking at is safe to use or a ticking time bomb? Also, in my opinion, the fact that there's a comment section, and they're considered "helpful" to boot, is a sign that the documentation is lacking and people are trying to fill its holes.

pca006132vor 4 Jahren

Just know that nix shell is actually the biproduct, it is indispensable for me to separate development environments for different projects.

omegacharlievor 4 Jahren

Coming from Ansible in the past for configuration management and various deployments of Gentoo as a base-distribution, the fact Nix employees similar concepts but more declarative and less 'ad-hoc' has a certain intrigue to it.

Curious from others who used similar in the past and use Nix now think about it.

SuperSandro2000vor 4 Jahren

If you ever interrupted a not perfect Ansible playbook mid way through you will know the pain to repair that which NixOS will never have.

_y5hnvor 4 Jahren

Ansible is less declarative, but it's basically just ssh. No agents, no custom user installs no OS. So it depends what your aim is.

Ansible is great to deploy stuff with. Not perfect and not like terraform+k8s/docker.

Nix may be more for developers, as you'll end up developing for it.

rq1vor 4 Jahren

Peoples usual struggle is to make a difference between the language and nixpkgs’s or nixos frameworks (mkderivations, makescope, override, overrideAttr, overridePythonAttrs, flakes, overlays, modules, services… etc.)

The tutorials try to teach a soup of functional programming, build systems, configurations, nix language and nix framework, and that’s somewhat difficult to disentangle and tell what’s what.

Anyway I use it as my daily driver now, and overcame almost all the difficulties, but one still remains: the lack of static type checking. It’s hard to build and validate modules without having this built in the language itself.

I’m hoping that you’d accelerate the work on nickel and try to automatically translate the current nixpkgs/nixos.

markeibesvor 4 Jahren

I’d love for nix to actually ever work and not just everybody making annoying PRs to all repos all the time

ParetoOptimalvor 4 Jahren

Are you saying people are submitting broken nix PRs to repos all the time?

I find that hard to believe, do you have examples?

babyvor 4 Jahren

You guys are definitely doing hype right, this post, and the number of other posts I’m seeing on HN surely are going to contribute a lot to the growth of nix. When is the nix conference happening :)?

pxcvor 4 Jahren

The next one is this October!

https://2022.nixcon.org/

eointierneyvor 4 Jahren

Super.

I love the responsibility you're taking on, and why you're choosing the problems you do. You're doing dog's work.

Nix ix the best thing in applied computer science since I don't know what (suggestions in the comments below please)

It's evolving so quickly, and so easily, that it's hard to keep up. There's a general need for coherent porcelain, and the documentation to match. There's a lot of hope for what it can do, but it's still very hard to learn how to do a lot of it.

It's going to become ubiquitous

Thank you

leohvor 4 Jahren

It would be massively helpful if there were good IDE (tooling) for editing nix files.

e.g. on par with the quality of Java editing in IntelliJ

This would help massively for folks learning how to do things.

Besides that, I just keep wondering — and tweag said it too — if some tool that did nix-shell and nixpkgs would be enough.

At the end of the day, a lot of folks are really just using nix like Python‘s virtualenv — i.e. declare what you need, change $PATH to include it in a sandboxed environment.

didipvor 4 Jahren

I don’t like Nix.

It’s hard to use. It solves a problem that Docker already solved and Docker UX is much better.

And it has a DSL that I think is unnecessary for the problem at hand.

Smaug123vor 4 Jahren

Docker definitely solves the problem imperfectly, though. Docker images simply don't compose, and this has caused me great pain: if you want image A plus image B, you can't do much better than a multi-stage build where you just magically know what you need to pull out of the first image. Nix, by contrast, composes perfectly, because packaging something for Nix involves precisely specifying what you need (as opposed to packaging something for Docker, which involves… splurging your package arbitrarily across a clean container).

SuperSandro2000vor 4 Jahren

It really depends on your problem. If you need to patch docker images it often gets hairy.

alberthvor 4 Jahren

I wish FreeBSD had a way to do reproducible builds and deployment (like NixOS, not nix which FreeBSD already has)

But for that to happen I imagine PkgBase needs to be completed first.

https://wiki.freebsd.org/PkgBase

yjftsjthsd-hvor 4 Jahren

So you want NixOS/FreeBSD? I'm actually not sure that that does need PkgBase - in fact, it might be mostly irrelevant since you'd end up writing a nix package for the FreeBSD base system anyways.

jonstewartvor 4 Jahren

Nix just needs to have a feature freeze until the docs are up-to-date.

sullyj3vor 4 Jahren

It's true that the documentation situation is pretty rough. However, The problem with this is that common usage of Nix is in kind of a bizarre state at the moment. We're in this limbo between the old `nix-` style commands and the new `nix ` commands, and between Not-flakes and flakes. These new features are labelled "experimental", but in practice they are widely used, and regarded as the inevitable future of the use of Nix. You don't want to be putting enormous amounts of effort into documenting all of the confusing gotchas of a situation that will ideally go away soon.

It makes more sense to make a strong push for getting the new features fleshed out, made user friendly, and stabilized, and then slamming the brakes and doing a documentation push.

In practice, this will take a long time, and it probably makes sense to concurrently work on documenting things that won't change. But I don't think at this stage dropping all feature work for documentation makes sense.

The other thing is that I don't think Nix's learnability issues come entirely from lack of documentation. A big chunk of the problem is a lot of very strange UX decisions and confusing defaults. Rather than documenting these as is, it seems better to put effort into fixing them, making default behaviour more intuitive and less confusing.

SuperSandro2000vor 4 Jahren

No. There just needs to be an effort to improve the docs. Nix is not moving that fast that you can't catch up anymore.

forgotpwd16vor 4 Jahren

Where Nickel fits to this? Will it replace Nix or co-exist?

Smaug123vor 4 Jahren

"Nix" refers to many things. Nickel specifically replaces the language that is called Nix. It has nothing to do with the package manager called Nix, the build system called Nix, or the OS called NixOS.

notfedvor 4 Jahren

Nixlang is so alien that I've used NixOS for 2 years and still don't get it.

Nickel looks much prettier than Nixlang, but it carries over a feature that still confuses me about Nixlang: variable declaration. Perhaps you can shed light on why it is the way it is.

Why does it need to be of the format "let x = 5 in whatever(x);"? Why not, like Rust/F#/etc, allow the syntax "let x = 5; whatever(x)", i.e., allow the definition to stay in scope for the remainder of current block?

grumbelvor 4 Jahren

Everything in Nix evaluates to a single value, including 'let' and even complete files. There are no blocks like in an imperative language. The '{}' in Nix are not blocks, but sets, known in other languages as maps or dicts. Thus what comes after the 'in' in a 'let' is an expression itself and has to return a single value.

Meaning whenever you see a 'let', look at the surrounding, it will probably look like '{ a = (let x = 5; in x); }' which is just a convoluted way to write '{ a = 5; }'. Or often you'll see 'a = (foo: let ...)' in this case the ':' declares a function taking the argument 'foo'.

You can play around with the language in the 'nix repl'. The language itself is really simple, but not very obvious when just looking at a code snippet it.

And yes, there is an argument to be made that a purely expression based language is not very ergonomic, but it keeps the Nix language simple.

Smaug123vor 4 Jahren

I can only speculate, but I presume this is a historical detail from Standard ML.

forgotpwd16vor 4 Jahren

So, will it replace the language called Nix or co-exist with it? How will Nix package manager and build system handle this new language?

Smaug123vor 4 Jahren

Undecided; see https://github.com/nickel-lang/nickel-nix for a proof-of-concept of an integration that requires making no changes to either Nickel or Nix.

pxcvor 4 Jahren

Whether Nickel replaces the Nix language isn't up to Tweag, in the end. It's down to whether the Nix community finds it a compelling option. I, for one, am hopeful that it can preserve what is great about the Nix language and also offer a leap forward in debuggability and performance! But I don't think Nix's whole future hangs on it.

CMCDragonkaivor 4 Jahren

Can tweag help get Nix working natively for windows?

Smaug123vor 4 Jahren

Discussion ongoing at https://discourse.nixos.org/t/nix-on-windows/1113

philip142auvor 4 Jahren

It needs a more friendly app installer gui

ducktectivevor 4 Jahren

Latest stable version includes a GUI installer? Or perhaps you meant the app is not intuitive enough...

pxcvor 4 Jahren

Could they mean a GUI for package management rather than OS installation?

e3bc54b2vor 4 Jahren

GUI for package manager is fairly useless IMO. Homebrew is CLI only AFAIK and nobody cares, apt has aptitude but I've seen nobody use it in real life. Honestly only relevant package manager GUI I see are Play Store/App Store, and that's a different discussion.

pxcvor 4 Jahren

Linux has a few app store style GUIs, e.g., Discover and the GNOME Software Center.

Btw, aptitude includes an ncurses TUI, but it's also an improved CLI frontend for apt that includes a few really nice features. Check it out next time you're on a Debian-based distro and try some of the special search patterns at least.

[deleted]vor 4 Jahren

Linda703vor 4 Jahren