Typing Nix(tweag.io)
tweag.io
Typing Nix
http://www.tweag.io/posts/2017-05-23-typing-nix.html
126 comments
I love the Nix language and I generally distrust aesthetic feelings about programming languages.
Neither SML nor Haskell are optimized for expressing deeply nested records with many string literals, for example. The multiline interpolated strings in Nix are extremely much better than in SML or Haskell. The way records and arrays are written is great: SML and Haskell both suffer from the tedious problem of using separators between items instead of after each one; in Nix each item can always be moved without messing with separators.
I think Nix is an engineering marvel up to and including the language design. If you can make a better surface syntax and demonstrate it by translating some significant part of Nixpkgs, I'd be very interested, but I think in general the language is the way it is because that's what made most sense for the system's designers.
Neither SML nor Haskell are optimized for expressing deeply nested records with many string literals, for example. The multiline interpolated strings in Nix are extremely much better than in SML or Haskell. The way records and arrays are written is great: SML and Haskell both suffer from the tedious problem of using separators between items instead of after each one; in Nix each item can always be moved without messing with separators.
I think Nix is an engineering marvel up to and including the language design. If you can make a better surface syntax and demonstrate it by translating some significant part of Nixpkgs, I'd be very interested, but I think in general the language is the way it is because that's what made most sense for the system's designers.
> If you can make a better surface syntax
... http://gnu.org/s/guix
Lisp is pretty good at lists.
The only important language is the derivation language sent to the daemon. Nix spends so much time building up inputs... for shell scripts. I always felt like they would have been so much more successful if they chose some other, more popular language to generate derivation.
... http://gnu.org/s/guix
Lisp is pretty good at lists.
The only important language is the derivation language sent to the daemon. Nix spends so much time building up inputs... for shell scripts. I always felt like they would have been so much more successful if they chose some other, more popular language to generate derivation.
Is it possible to write a guix package that includes code which redefines the package object?
Yes!
http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/...
You'll see here that the package emacs-minimal actually inherits from the emacs package definition, and then all fields afterwards are essentially overwrites of whatever was in the emacs package.
Unfortunately, I just noticed that the manual alludes to this, but actually never explicitly documents it in the "Defining Packages" section :/
http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/...
You'll see here that the package emacs-minimal actually inherits from the emacs package definition, and then all fields afterwards are essentially overwrites of whatever was in the emacs package.
Unfortunately, I just noticed that the manual alludes to this, but actually never explicitly documents it in the "Defining Packages" section :/
> Neither SML nor Haskell are optimized for expressing deeply nested records with many string literals, for example.
Purescript's extensible row types handle this process elegantly and if Nix takes a cue from that, it will work great.
Purescript's extensible row types handle this process elegantly and if Nix takes a cue from that, it will work great.
So, JavaScript? The records literals are JSON, which is the most popular syntax for this at the moment anyways.
No, Nix syntax is much nicer than Javascript. Consider, for example, the similarity between a let expression and an attribute set (NOT JSON by the way):
let
foo = 42;
bar = "a string";
in {
baz = "deja";
quux = "vu";
}
const foo = 42;
const bar = "a string";
return {
baz: "similar",
quux: "but different"
};
Also, recursive definitions using "rec" which become nasty in JavaScript.
I tend to write JavaScript that looks a bit like Nix... but in general, the imperative constructs aren't suitable to the lazy evaluation strategy of Nix.
I may be in the minority, but I like the Nix language.
I don't have extensive ML or Haskell experience, coming from Ruby/Java background.
I think the Nix language works great as a configuration language, and I greatly prefer it to JSON, YAML, or Ruby, which are often used for similar purposes. The way arguments are passed in, merging of attribute sets, the library from Nixpkgs with various list and set operators and other utilities, and great multiline string support make it far superior to those other config languages in my opinion.
My biggest gripe is that the Emacs mode isn't very strong, so I'm often applying indentation manually.
I recently learned about Jsonnet[1] which has some similarity with Nix.
[1] http://jsonnet.org/
I don't have extensive ML or Haskell experience, coming from Ruby/Java background.
I think the Nix language works great as a configuration language, and I greatly prefer it to JSON, YAML, or Ruby, which are often used for similar purposes. The way arguments are passed in, merging of attribute sets, the library from Nixpkgs with various list and set operators and other utilities, and great multiline string support make it far superior to those other config languages in my opinion.
My biggest gripe is that the Emacs mode isn't very strong, so I'm often applying indentation manually.
I recently learned about Jsonnet[1] which has some similarity with Nix.
[1] http://jsonnet.org/
I could not agree with you more! I use nix daily, and every time I go to edit it a nix file a voice in the back of my head says, "you should really start that Haskell to Nix compiler you've been thinking about."
[edit] ha! just notice the next comment talking about hnix. Great minds... :)
You might be interested in dhall[0] and dhal-to-nix[1] which provides a really neat haskell-like language that compiles to nix.
[0]: https://github.com/Gabriel439/Haskell-Dhall-Library [1]: https://github.com/Gabriel439/Haskell-Dhall-Library
[0]: https://github.com/Gabriel439/Haskell-Dhall-Library [1]: https://github.com/Gabriel439/Haskell-Dhall-Library
If only Haskellers knew that "a blog post" and "type definitions in Hackage" are not documentation...
Also, the comma-first syntax like
{ foo = "bar"
, baz = "qux"
}
drives me nuts. In the name of all good things in the world, why?
Also, the comma-first syntax like
{ foo = "bar"
, baz = "qux"
}
drives me nuts. In the name of all good things in the world, why?
> comma-first syntax
oooh but this is one of those trivial things that I really liked in my brief days of using Haskell, and that has sometimes carried over to my C++, e.g. in initializer lists. I find it super useful because everything lines up nicely, and from the point of view of version control you don't have to edit the previous line to change '}' to ',' when adding a new line.
oooh but this is one of those trivial things that I really liked in my brief days of using Haskell, and that has sometimes carried over to my C++, e.g. in initializer lists. I find it super useful because everything lines up nicely, and from the point of view of version control you don't have to edit the previous line to change '}' to ',' when adding a new line.
Go solves this problem by permitting
foo := Foo{
x: 10,
y: 24,
}How about allowing for bloody danging commas?
Is there any real disadvantage to comma-first, other than aesthetics (which is entirely a personal preference)?
Yes, when the commas are at the beginning of the line they are really obvious. I have used commas-first style in JavaScript, and it definitely reduced the amount of times I forgot a comma.
I wonder if people's aversion to commas first has something to do with the fact that commas have a very familiar use in ordinary writing, and there they always follow a word. It is very common to see BNF grammars written with leading vertical bars, and no one seems to mind that -- in fact it would be unusual to use trailing vertical bars.
By the way, a bunch of languages permit a trailing comma after the last element of a list. This is also nice because it means that if you put each list item on a line, the list items all have the same format (thing plus comma). With the Haskell syntax, you lose this, but there is no reason a language couldn't support a leading comma before the first element of a list.
I wonder if people's aversion to commas first has something to do with the fact that commas have a very familiar use in ordinary writing, and there they always follow a word. It is very common to see BNF grammars written with leading vertical bars, and no one seems to mind that -- in fact it would be unusual to use trailing vertical bars.
By the way, a bunch of languages permit a trailing comma after the last element of a list. This is also nice because it means that if you put each list item on a line, the list items all have the same format (thing plus comma). With the Haskell syntax, you lose this, but there is no reason a language couldn't support a leading comma before the first element of a list.
My aversion has to do with the fact that the first element doesn't have one. The irregularity is annoying not just aesthetically, but also if you want to swap lines around, etc.
But there's always a line without a comma -- comma-last has it on the last line instead of the first. So you can have the same line-swapping problem in either arrangement. Comma first just makes it easier to spot.
But there's always a line without a comma -- comma-last has it on the last line instead of the first.
Not if the language allows trailing commas on the last line, like Python and Go do.
Not if the language allows trailing commas on the last line, like Python and Go do.
Not if you allow dangling commas (trailing commas as the sibling comment calls them, I think it's the more generic term)...
Then you have a comma at the end in ALL lines, including the last one, without an issue. That was what I proposed above (and lots of languages do it, I think JS is adding it too).
Then you have a comma at the end in ALL lines, including the last one, without an issue. That was what I proposed above (and lots of languages do it, I think JS is adding it too).
The wrath of God that it imposes on people who think it's acceptable, for one...
It's another thing to learn, and the total learning curve is the product of all of these idiosyncrasies (and features) (not the sum). And there's absolutely no advantage to comma-first.
It makes git diffs inaccurate compared to trailing commas.
[deleted]
How so?
Inserting a new element with comma-first changes 2 lines:
This doesn't break anything, but it makes looking at "git blame" so much easier.
- { old-one
+ { new-one
+ , old-one
This doesn't break anything, but it makes looking at "git blame" so much easier.
This is only true if you insert at the head. And it's not really leading comma placement but the choice to put the first element in the line with the brace that is the culprit, though both are typical of comma-first.
With separating comma, it either affects the first or the last position, depending on where you put the commas. But those are the most common places you modify lists.
Wow, I'm doing the opposite where I'm staring at all these other DSLs and configuration files and have to hold myself back from writing a bunch of nix to autogenerate them from.
Haha same here, I've been considering writing Nix expressions for window manager configurations for quite some time now, but some config files are so complex that it would take ages.
If I only look at the language properties then Nix makes sense to me. The fact that Nix is lazily evaluated allows for easy composability. This is used a lot in nixpkgs to allow overrides and build up packaging layers. And because the language is interpreted, only the files that are being used need to be evaluated. It wouldn't be very practical to hold all of nixpkgs in memory. It also allow to do things like dynamically import code from another git repo. And finally, the language only allows side-effects through derivations.
None of the above language have all these properties combined.
None of the above language have all these properties combined.
I am only talking about the syntax. Semantically, Nix absolutely rocks.
It's one of those languages that a developer can look at and think "Oh, so this is how non-developers see code".
It looks like someone threw a pinch of erlang in a Chef recipe. Like bash, but every English word operator has been replaced by space ships, angry squids, ternary operators, && all the other things that make code hard to read.
But... I still love Nix, and you get used to it.
But... I still love Nix, and you get used to it.
Bash is still worse. Probably. I know it is ubiquitous, but if I can avoid writing Bash scripts, I do.
I have no idea why you would think that. It's pretty standard usage of punctuation like (){}[] etc. If it's hard to read that's because the libraries and infrastructure for writing derivations is not simple or well-documented. But that's not a language issue.
Hi, author here. While I agree that the language could be improved, I wonder why you say that this makes it uglier, and how you'd imagine a better solution (this is still in development, and any idea is welcome).
Is there any option to break source-level compatibility? (Automatic translation to old syntax will still be possible and seamless, of course).
If yes, I can draft a proposal.
If yes, I can draft a proposal.
Unfortunately no, except maybe for some very minor tweaks that have a chance to be included in vanilla nix, but otherwise this would most probably either mean the death of this project or a split in the (already small) community, which I both don't want to see
Community could be larger if all these petty things were resolved :) But I understand, forking is definitely not an option.
So, if only gradual changes are allowed, I would have started with:
- making semicolons optional
- commas for implicit array literals (e.g. "a", "b", "c" in lieu of ["a" "b" "c"]
- something to do with 'with' / 'in' syntax.
In fact, I can live with anything, except non-optional semicolons. Is there a possibility to infer them?
So, if only gradual changes are allowed, I would have started with:
- making semicolons optional
- commas for implicit array literals (e.g. "a", "b", "c" in lieu of ["a" "b" "c"]
- something to do with 'with' / 'in' syntax.
In fact, I can live with anything, except non-optional semicolons. Is there a possibility to infer them?
The only place where I think we could make the semicolons optional is at the end of a let-binding or a record (like json does iirc), which would indeed be a small but blessed change.
For the array litterals, I don't really see which improvements your syntax brings (except suppressing the space-separated list, which conflicts with the syntax for function application). Is it used somewhere else ?
For the array litterals, I don't really see which improvements your syntax brings (except suppressing the space-separated list, which conflicts with the syntax for function application). Is it used somewhere else ?
(I can't reply to the previous message regarding no-semicolon syntax, so I have to do it here. I definitely do not want to propose whitespace-sensitive syntax — just newline-sensitive, like in Scala, Swift, modern ECMAscript, or just any modern programming language with C-like syntax. Semicolons are allowed, BTW — they are just optional and inferred from newlines).
Here is the example where I just removed every semicolon.
https://gist.github.com/atemerev/889806081ed8fcb77495666fac9...
For me, it looks infinitely better.
https://gist.github.com/atemerev/889806081ed8fcb77495666fac9...
For me, it looks infinitely better.
> In fact, I can live with anything, except non-optional semicolons. Is there a possibility to infer them?
I'm genuinely curious... what makes that a dealbreaker?
I'm genuinely curious... what makes that a dealbreaker?
I see. But that means using a whitespace-sensitive syntax, which some people (including me ;) ) don't really like
Very strongly agreed. Syntactically significant whitespace is the source of so many preventable errors that I believe it's simply not worth the aesthetic improvements.
There is no significant whitespace in what I am offering; just semicolon inference at newlines (like in modern Javascript).
Keep in mind that I am being simultaneously irreverent while also expressing how I feel about these two debates (semicolons and significant whitespace).
I'm someone who has worked in a large variety of languages, including Python (significant whitespace) and Javascript (no significant whitespace).
First: semicolon inference in Javascript is an abomination :). If the language says that whitespace is not significant, then using newlines to infer where semicolons maybe should be... is inconsistent with the statement "whitespace is insignificant".
Second: when I learned Python, I was coming from a primarily Perl and C background. The significant whitespace wasn't a big deal at all because that's how I indent my code anyway. And I would argue (and do in code reviews) that any code that is not indented properly is incorrect because it is extraordinarily misleading to a reader.
So, yeah, for any given programming language, I couldn't care less if whitespace is significant or not, or whether there's semicolons at the end of lines, but god damnit don't go half way on either of those. Indent your code as if whitespace is significant (because it IS to whoever's trying to decipher it later), and stick semicolons at the end of your lines whether or not some stupid inference rule will stick them there for you if you forget.
And as a final rant, "modern Javascript" didn't introduce semicolon inference; it's been there for a long time and was put there to try to lower the barrier for newbies. It wasn't put there because it's a great design decision.
I'm someone who has worked in a large variety of languages, including Python (significant whitespace) and Javascript (no significant whitespace).
First: semicolon inference in Javascript is an abomination :). If the language says that whitespace is not significant, then using newlines to infer where semicolons maybe should be... is inconsistent with the statement "whitespace is insignificant".
Second: when I learned Python, I was coming from a primarily Perl and C background. The significant whitespace wasn't a big deal at all because that's how I indent my code anyway. And I would argue (and do in code reviews) that any code that is not indented properly is incorrect because it is extraordinarily misleading to a reader.
So, yeah, for any given programming language, I couldn't care less if whitespace is significant or not, or whether there's semicolons at the end of lines, but god damnit don't go half way on either of those. Indent your code as if whitespace is significant (because it IS to whoever's trying to decipher it later), and stick semicolons at the end of your lines whether or not some stupid inference rule will stick them there for you if you forget.
And as a final rant, "modern Javascript" didn't introduce semicolon inference; it's been there for a long time and was put there to try to lower the barrier for newbies. It wasn't put there because it's a great design decision.
I come from Scala, and there semicolon inference works flawlessly. Why do you consider it to be an abomination in Javascript? (Except this famous corner case with continued line, almost never encountered in practice).
Well... This is a sort of whitespace sensitivity, and I don't really like it, because that means that some tokens may or may not be meaningful depending on the context, which increases the complexity of the syntax -- and syntax is probably the first thing that I think should be dumb simple and predictable.
Is that a definite no, and there's no chance of compiling whatever $newLanguage is written to the current nix code prior to execution?
If that was possible, and a few $000 was raised (looking at the comments here, I think people would chip in) then a new version of the Nixlang with the same semantics and full backwards compatibility but a cleaner syntax, would be a great project to fundraise for.
If that was possible, and a few $000 was raised (looking at the comments here, I think people would chip in) then a new version of the Nixlang with the same semantics and full backwards compatibility but a cleaner syntax, would be a great project to fundraise for.
Can't it just accept two different syntaxes (if really necessary for disambiguation, perhaps requiring people to put some magic string at the beginning of the file to request the new syntax)?
My tool could, but (for now, at least) it is a separate project from the main Nix interpreter, and this one is notably slow to evolve
Aside: I would love it if ML caught on. Such a good balance between expressiveness and usability. Imagine if Caml had caught on in the 90s instead of Java...
This is why I'm very much interested in Guix and GuixSD, which use Guile Scheme. Unfortunately I don't think they have an officially-supported story for running on the server, yet, or an equivalent to NixOps.
You tried guix ? I still have a fondness for sexps, but ML would be nice too.
I need CUDA drivers for what I am working with, and I have found no way to install it in Guix. Otherwise, it looks interesting enough.
(Sexps are usually not my first choice, I think it is a "lazy option" for people who do not want to write parsers / design their own syntax. Even then, they are better than current Nix expressions).
(Sexps are usually not my first choice, I think it is a "lazy option" for people who do not want to write parsers / design their own syntax. Even then, they are better than current Nix expressions).
> Sexps are usually not my first choice, I think it is a "lazy option" for people who do not want to write parsers / design their own syntax.
There are some real advantages to having one universal syntax: witness the explosion in XML and, later, JSON (both of which are generally inferior to S-expressions[1]). It'd be pretty great for one person to write one parser, and then everyone forever after to be able to use it. We'd be able to focus on semantics and not on syntax.
[1] Although it is nice that JSON supports first-class associative arrays.
There are some real advantages to having one universal syntax: witness the explosion in XML and, later, JSON (both of which are generally inferior to S-expressions[1]). It'd be pretty great for one person to write one parser, and then everyone forever after to be able to use it. We'd be able to focus on semantics and not on syntax.
[1] Although it is nice that JSON supports first-class associative arrays.
AFAICT, there are 2 major differences between Nix and Guix. Language choice is the obvious one. The other is that Guix takes a hard-line stance on the GNU philosophy and therefore refuses to distribute pre-compiled binaries. I don't think CUDA will ever be an option on Guix.
Pretty sure there is nothing language-wise stopping anyone from hosting a separate repository with pre-compiled binaries.
It's a volunteer-run service. It makes sense they don't want to host your proprietary binaries
It's a volunteer-run service. It makes sense they don't want to host your proprietary binaries
Does something actually stop you packaging proprietary binaries, perhaps privately, for Guix? It's clearly better to say No, but CUDA is a case where that's difficult.
(I don't see what's wrong with adopting sexps to avoid dealing with custom syntax -- quite the opposite,)
(I don't see what's wrong with adopting sexps to avoid dealing with custom syntax -- quite the opposite,)
The kernel it ships with won't load blobs.
OK, there's a distinction between the GuixSD free software distribution and Guix package management. I'm surprised you can't build and use a non-linux-libre kernel with GuixSD, though; that presumably excludes even free additions, like Lustre, unless someone has done the forward porting work. (One of the recent HPC-ish presentations on Guix mentioned nvidia stuff -- I don't remember which.)
Some academics say that sexp allow to focus on what's important: semantics. I kinda agree with that, so many time, effort is wasted on syntax that moves, and only make ideas further apart.
Anyway, thanks for the answer
Anyway, thanks for the answer
Well, right, but we are not talking about an academic paper here, we talk about practically convenient language. If syntax is unimportant, we could leave Nix files as is -- semantically, it is a good programming language. But judging from the comments, I am not the only one who thinks that syntax and overall esthetics are important.
Really? I write almost nix every day at work and I have no problem with the syntax. I think it's pretty elegant and well-suited to its use case. There are a few minor tweaks I would make but nothing big at the syntactic level.
EDIT: probably the only gripe I would make is to have comma-separated lists instead of having to use parentheses to separate non-atomic expressions.
EDIT: probably the only gripe I would make is to have comma-separated lists instead of having to use parentheses to separate non-atomic expressions.
> probably the only gripe I would make is to have comma-separated lists instead of having to use parentheses to separate non-atomic expressions.
Wait no separators are evil. This is the best thing about the syntax.
Wait no separators are evil. This is the best thing about the syntax.
> I cringe every time I have to edit a .nix file (and I have to do it a lot).
No need to cringe over syntax. Syntax is just syntax. If you are familiar with a better syntax to express Nix in then create a parser from that format to Nix' syntax. Then insert this parser as a pre-processor in the build process, and problem solved.
No need to cringe over syntax. Syntax is just syntax. If you are familiar with a better syntax to express Nix in then create a parser from that format to Nix' syntax. Then insert this parser as a pre-processor in the build process, and problem solved.
Honestly that's the only thing that keeps me from jumping to NixOS. I don't want to learn a weird language, I'll never use for anything else, just to maintain my OS.
The language is so simple you can learn all of it in a day. Read through chapter 15 in the nix[1] manual and that's it.
To me it doesn't feel weird at all: I can't think of any feature that was new or surprising when I started using it and I am not the biggest expert of functional programming. You have to learn more about the standard environment and tools if you want to start contributing to nixpkgs and write your own packages but if you only care about configuring your system nix is as difficult to learn as say, JSON.
[1]: https://nixos.org/nix/manual/#ch-expression-language
To me it doesn't feel weird at all: I can't think of any feature that was new or surprising when I started using it and I am not the biggest expert of functional programming. You have to learn more about the standard environment and tools if you want to start contributing to nixpkgs and write your own packages but if you only care about configuring your system nix is as difficult to learn as say, JSON.
[1]: https://nixos.org/nix/manual/#ch-expression-language
It is definitely not difficult to learn, just somewhat hard to read and visually unappealing :)
Personally, I'm not against the idea in principle. There are some real DSL-haters out there, and I'm not one of them. For a different problem domain, a different language can be more efficient.
It's true that some DSLs are not only different, they're also badly-designed languages or gratuitously different. I guess this is an occupational hazard of DSLs, but it's not unavoidable. It's possible to make the leap to a DSL only when actually beneficial and then to design a DSL that doesn't suck.
It's true that some DSLs are not only different, they're also badly-designed languages or gratuitously different. I guess this is an occupational hazard of DSLs, but it's not unavoidable. It's possible to make the leap to a DSL only when actually beneficial and then to design a DSL that doesn't suck.
Neat. Somewhat related is hnix [0], a community effort in re-implementing the Nix expression language in Haskell.
While so far the focus has mostly been on implementing the expression language as is, hnix being written in Haskell could be attractive for enriching the language with fancier features:
"Because now that it's in Haskell, now that a lot of other hackers could get involved, we could do things like add optional typing to the Nix language." — J. Wiegley
For more on hnix check out Haskell Cast's Episode 13 [1] with John Wiegley.
[0]: https://github.com/jwiegley/hnix
[1]: http://www.haskellcast.com/episode/013-john-wiegley-on-categ...
While so far the focus has mostly been on implementing the expression language as is, hnix being written in Haskell could be attractive for enriching the language with fancier features:
"Because now that it's in Haskell, now that a lot of other hackers could get involved, we could do things like add optional typing to the Nix language." — J. Wiegley
For more on hnix check out Haskell Cast's Episode 13 [1] with John Wiegley.
[0]: https://github.com/jwiegley/hnix
[1]: http://www.haskellcast.com/episode/013-john-wiegley-on-categ...
There's also Guix in case anyone is interested. It uses Guile/Scheme for everything instead of Haskell.
I think that all the efforts to fix or re-implement the Nix language have vindicated Ludovic, the Guix maintainer, who chose an existing general-purpose language instead.
Except GUIX is more of a political project and part of the GNU "stack". It's a noble goal and I think it's great they're doing it. (It's people doing work on their spare time, so more power to them!)
But I wonder if that will help or hinder it in the end. It's also unclear to me what Guile brings other than fracturing the Scheme/Lisp/Racket community further (there are other Schemes that are older, seemingly just as clean, and often more powerful). Seems like mostly NIMBYism, but I might be offbase
But I wonder if that will help or hinder it in the end. It's also unclear to me what Guile brings other than fracturing the Scheme/Lisp/Racket community further (there are other Schemes that are older, seemingly just as clean, and often more powerful). Seems like mostly NIMBYism, but I might be offbase
The only significant shortcoming of the Nix language is the lack of static types. Guix doesn't fix this. In fact there isn't an existing language in common use that offers statically-typed extensible records, which IMHO vindicates the bespoke approach.
But Scheme is untyped too AFAIK - isn't it? So it'd require the same degree of "fixing" as long as we're talking about the discussed article.
As I explained in another comment, Scheme is indeed dynamically-typed (like Nix), but it's also strongly typed: one can define new data types, they are disjoint, and there can be no casts and the like. In Guix "packages" and "operating systems" are different types, you get a type error if you use one instead of the other; in Nix both are "attribute sets" (key/value dictionaries.)
I think the one-type-fits-all ("attribute sets") vs. strong typing (disjoing record types) axis matters more than the static vs. dynamic axis here because most of the code (OS config files, package definitions, etc.) is loaded dynamically.
I think the one-type-fits-all ("attribute sets") vs. strong typing (disjoing record types) axis matters more than the static vs. dynamic axis here because most of the code (OS config files, package definitions, etc.) is loaded dynamically.
How does Nix compare with conda[0] ? I could not find any useful resource, in my team we were hesitating between the two, for packages management ; finally we went for conda because it (seems)is better documented and package description is easier to write for newbies. And also we are a Python shop, so other devs felt more comfortable with conda. Personally I prefer Nix, though, and I wanted to advocate for it but the only main difference I could find is that Nix sits on top of the kernel (including libc...) whereas conda uses the system libc.
[0]: https://conda.io
[0]: https://conda.io
I wrote this document: http://catern.com/posts/deps.html which briefly compares both Conda and Nix among other things.
(warning: it's very pro-Nix :) )
(warning: it's very pro-Nix :) )
Not to put down Nix, but some of the arguments about other systems have counterexamples. For instance, it's common in rpm or dpkg distributions to have multiple versions and implementations of things, and richer dependencies than simply on packages. You can also do unprivileged installation into a separate root with, say, fakechroot or PRoot, and you need something like that to install and run Nix unprivileged, don't you?
There are comparisons with alternatives in at least the propaganda for Spack and 0install, though the same criticism might apply to those -- I don't remember.
There are comparisons with alternatives in at least the propaganda for Spack and 0install, though the same criticism might apply to those -- I don't remember.
You can install Nix in a totally unprivileged fashion, without having to use hacks such as PRoot, you just need to configure it to use a directory you have write access to instead of /nix. But doing so breaks the ability to use the official binary caches (because relocating the store requires recompiling packages), and this isn't particular to nix (I remember that portage at least does this too, and probably some others).
The interesting part is that with an installed deamon, the admin can allow users to install packages in their own profile, while still benefiting from sharing of dependencies and other niceties.
About the multiple versions, the problem with "classical" package managers is that you have to do some manual renaming to ensure that both versions won't be installed in the same place in the file-system, which is tedious and doesn't scale well. Furthermore you may encounter some problems because both will be visible at the same time if the packager isn't careful enough. For example, if a software X has a dependency of libfoo2.1 and you happen to have libfoo3.1 also installed, the installation script for X may use libfoo3.1 instead of libfoo2.1, in which case you risk to encounter bugs because X hasn't been tested against libfoo3.1.
About the multiple versions, the problem with "classical" package managers is that you have to do some manual renaming to ensure that both versions won't be installed in the same place in the file-system, which is tedious and doesn't scale well. Furthermore you may encounter some problems because both will be visible at the same time if the packager isn't careful enough. For example, if a software X has a dependency of libfoo2.1 and you happen to have libfoo3.1 also installed, the installation script for X may use libfoo3.1 instead of libfoo2.1, in which case you risk to encounter bugs because X hasn't been tested against libfoo3.1.
rpm and dpkg don't support having multiple versions and implementations of things simultaneously installed. Not sure what you mean by richer dependencies.
>You can also do unprivileged installation into a separate root with, say, fakechroot or PRoot
Yes, but that's the only way to (safely) use such package managers without privileges; so that's not viable to do for production.
>and you need something like that to install and run Nix unprivileged, don't you?
Install, yes. Run, no. Setting up Nix does require root, but once it's installed on a system it can be safely used without privileges by all the users on the system. That's very valuable in itself, but it's especially interesting when you consider providing access to a single Nix store over a network filesystem, or providing it to multiple different virtual machines or containers.
>You can also do unprivileged installation into a separate root with, say, fakechroot or PRoot
Yes, but that's the only way to (safely) use such package managers without privileges; so that's not viable to do for production.
>and you need something like that to install and run Nix unprivileged, don't you?
Install, yes. Run, no. Setting up Nix does require root, but once it's installed on a system it can be safely used without privileges by all the users on the system. That's very valuable in itself, but it's especially interesting when you consider providing access to a single Nix store over a network filesystem, or providing it to multiple different virtual machines or containers.
On RHEL 6 I count eight different packages implementing the MPI standard -- multiple versions of three implementations of essentially the same API -- which can be installed together. The richness is that dependencies are not generally simply on packages; MPI packages provide things like "mpi", "openmpi(x86-64) = 1.10.3-3.el7", "libmpi.so.12", and config(openmpi) = 1.10.3-3.el7".
My Guix installation runs using a with privileged access to the store, and Nix is the same as far as I know. I don't know why Nix would be a particular advantage on our HPC networked filesystems, and it's not clear it's tenable for users to be able to DoS the store by installing an arbitrary amount of software in limited shared space.
It may well be that Nix makes the right set of trades-off against other possibilities for a given situation, but thta seems at least not completely clear cut.
My Guix installation runs using a with privileged access to the store, and Nix is the same as far as I know. I don't know why Nix would be a particular advantage on our HPC networked filesystems, and it's not clear it's tenable for users to be able to DoS the store by installing an arbitrary amount of software in limited shared space.
It may well be that Nix makes the right set of trades-off against other possibilities for a given situation, but thta seems at least not completely clear cut.
You claim that conda "isn't very well supported by Unix as a whole" and "doesn't really work." Can you provide some evidence or examples to back up your claim?
Sorry, I guess it basically requires you already know the ways in which Conda doesn't work. The basic problem is that Unix has a lot of hardcoded paths and no way to override them all at runtime. So you need to do lots of patching and hacks.
That is excellent, thanks. Double for mentioning Docker.
I use conda every day in my professional deep learning practice and it's a HUGE time saver. I can't press enough how I'm sure it's already saved me weeks if not months of package butfucking.
I'm pleasantly surprised to see so many Nix enthusiasts milling around in this thread and I'd like to encourage people to step up and do their part in helping improve and maintain nixpkgs. It's really not that hard, and it's the main area I would say Nix could do with some improvement. Adopt a package today!
I tried doing some non-trivial contributions, and unfortunately there's currently some degree of a problem where there are not enough people actively revieving contributions and having enough authority in the project to be able to merge with confidence. Is it maybe not enough sponsorship for the project, compared to its popularity? Or does it not have a person with enough organisational experience in non-profits?
The problem with Nix in terms of typing is that it relies a lot on key/value dictionaries ("attribute sets" in Nix parlance). These data structures are dynamic in nature. All a static type checker can say is "yeah this is an attribute set". In addition, Nix does not offer a way to define new disjoint data types.
Conversely in Guix most of the data structures are disjoint record types. Scheme (the implementation language of Guix) is dynamically-typed, but there are sanity checks we can do on records both at macro-expansion time and at run time, such as checking whether all the required fields are defined and no extra field is passed. Concretely, this means that users get clear syntax errors or run-time type errors.
Nix and Scheme are both dynamically typed, but they have a different typing story.
Disclaimer: Guix hacker here.
Conversely in Guix most of the data structures are disjoint record types. Scheme (the implementation language of Guix) is dynamically-typed, but there are sanity checks we can do on records both at macro-expansion time and at run time, such as checking whether all the required fields are defined and no extra field is passed. Concretely, this means that users get clear syntax errors or run-time type errors.
Nix and Scheme are both dynamically typed, but they have a different typing story.
Disclaimer: Guix hacker here.
Sure it would be nice to have types for refactoring, but a more useful set of features would be:
- baked in fetchurl, fetchgit for easier bootstrapping
- a functional dsl for rendering filesystem hierarchies - instead of the find/sed/awk galore. 50% of the time when my nix recipe breaks it's not a nix syntax issue but something with bash. (reading nix is an exercise in learning new unix features)
- drop the channel feature entirely and make releases immutable tar files with hashes. I use nix as a build system on macos and right now a nix-channel update is a sure way to break all my builds which is the opposite of nix's promise
EDIT: formatting
- baked in fetchurl, fetchgit for easier bootstrapping
- a functional dsl for rendering filesystem hierarchies - instead of the find/sed/awk galore. 50% of the time when my nix recipe breaks it's not a nix syntax issue but something with bash. (reading nix is an exercise in learning new unix features)
- drop the channel feature entirely and make releases immutable tar files with hashes. I use nix as a build system on macos and right now a nix-channel update is a sure way to break all my builds which is the opposite of nix's promise
EDIT: formatting
For your last point, you don't have to use the channels (which will anyway be deprecated at some point iirc), and you can pin the nixpkgs version in your nix files (like it is done in https://garbas.si/2015/reproducible-development-environments...)
for your first point., what about `builtins.fetchTarball` and `builtins.fetchurl` (`fetchgit` is in nix unstable afaik)
Funny: Nix is short (or colloquial) for "nichts" in German which further means "nothing" in English.
So the CTA "Get Nix" is kinda funny for German speaking people :)
PS: Greetings from Austria!
So the CTA "Get Nix" is kinda funny for German speaking people :)
PS: Greetings from Austria!
I'd like to bring up another sort-of alternative: http://habitat.sh
It's not exactly in the same space as Nix, but not very far from it, either. In Habitat, you use a language that felt to me quite similar to Arch Linux's PKGBUILDs, but extended with container-specific things like service ports etc. Habitat outputs containers that can be started in various orchestration technologies, like Kubernetes or Mesos/Marathon.
It's not exactly in the same space as Nix, but not very far from it, either. In Habitat, you use a language that felt to me quite similar to Arch Linux's PKGBUILDs, but extended with container-specific things like service ports etc. Habitat outputs containers that can be started in various orchestration technologies, like Kubernetes or Mesos/Marathon.
It's interesting and very similar to Nix, but I still didn't figure out purpose of it, especially why it heavily uses Docker. Many people use Docker for the same functionality that Nix provides: bundling application with its dependencies, not for isolating groups of processes from each other.
Docker images is one of the supported formats when exporting packages. Others at the moment are aci, mesos and tarball.
> Configurations in Nix are written in a full-fledged programming language, featuring all manner of primitive datatypes (numbers, strings, file paths, etc), anonymous records and first-class functions. This is a very big deal for writing configurations in the large. Functions allow code reuse and abstraction, two crucial ingredients without which writing out configurations by hand would become unwieldy.
Off topic but this brings up a good point I've been curious about for a long time. Does anyone know why VimL is such a terrible language?
It's fascinating to think what potential it could have with a modern language natively supported like this. The use of Python/Lua etc seem like complicated hacks on top of it, not to mention the API with tons of globals.
Off topic but this brings up a good point I've been curious about for a long time. Does anyone know why VimL is such a terrible language?
It's fascinating to think what potential it could have with a modern language natively supported like this. The use of Python/Lua etc seem like complicated hacks on top of it, not to mention the API with tons of globals.
Slightly related: Why does Github call Vimscript "VimL"?
It's not just Github, the Vim filetype is "viml". It's a standard name for the language. Most likely meaning Vim Language.
No, it's not. The filetype is "vim", and the filetype detection code explicitly calls it "Vim script": https://github.com/vim/vim/blob/4670490673ed98502a09b74fbabe...
You're right, apologies, I was thinking of the Syntastic/Neomake plugin which both use "VimL" as an identifer, which I had just set up this past weekend and I mistook that as the Vim filetype identifier.
Regardless Wikipedia says:
> Vim script (also called vimscript or VimL) is the scripting language built into Vim.
https://www.wikiwand.com/en/Vim_(text_editor)#/Vim_script
It's also colloquially referred to VimL across the web.
For those looking for an answer, the 3rd result from googling "VimL" is a question on StackOverflow "Why does VimL suck?" which does a good job of answering my question:
https://www.reddit.com/r/vim/comments/1bf672/why_does_viml_s...
Regardless Wikipedia says:
> Vim script (also called vimscript or VimL) is the scripting language built into Vim.
https://www.wikiwand.com/en/Vim_(text_editor)#/Vim_script
It's also colloquially referred to VimL across the web.
For those looking for an answer, the 3rd result from googling "VimL" is a question on StackOverflow "Why does VimL suck?" which does a good job of answering my question:
https://www.reddit.com/r/vim/comments/1bf672/why_does_viml_s...
Nice.
Half-OT: How is the new CLI coming along? AFAIK there was an effort to make Nix usage a bit more like known from other package managers.
Half-OT: How is the new CLI coming along? AFAIK there was an effort to make Nix usage a bit more like known from other package managers.
There's an effort to make Nix's CLI better, but not necessarily like other package managers.
The design is at https://gist.github.com/edolstra/efcadfd240c2d8906348
The issue is at https://github.com/NixOS/nix/issues/779
The design is at https://gist.github.com/edolstra/efcadfd240c2d8906348
The issue is at https://github.com/NixOS/nix/issues/779
One year, and still not closed...
Is this an ongoing process and most of it is already released or is this still in the making?
Is this an ongoing process and most of it is already released or is this still in the making?
Why they wouldn't take the most (syntactically) beautiful functional programming language out there — Standard ML? It would work perfectly for such a task. Or the second contender — Haskell. If C-like syntax is desired, the best contendant is probably Swift.
I cringe every time I have to edit a .nix file (and I have to do it a lot).