The Fundamental Law of Software Dependencies(matklad.github.io)
matklad.github.io
The Fundamental Law of Software Dependencies
https://matklad.github.io/2024/09/03/the-fundamental-law-of-dependencies.html
https://matklad.github.io/2024/09/03/the-fundamental-law-of-dependencies.html
In the root of each project repository, we have a shell.nix file that lists all dependencies needed to build and develop the project. Most of them are from the latest major nixpkgs tarball and we include its hash to account for the hashes of every package that it provides. Other packages are from e.g. PyPI, crates.io or internal repositories and each of those have their own hashes.
With the shell.nix file we can then get a working environment set up by running nix-shell or automatically when we enter the directory by using direnv. The first time it will fetch and build everything but it will be cached after that.
This has worked quite well for us so far, it makes it very convenient to maintain dependencies. We can try out new dependencies on a feature branch and easily run CI on it, and when we merge the branch to master the dependencies follow along automatically for both local and CI machines.
For older revisions or branches it is also very convenient to get a working environment as you just need to check it out and load its shell.nix file. No need to make sure your system has e.g. an older Python version available, it will be fetched automatically. In theory, older revisions should be usable indefinitively, as long as the sources we fetch the dependencies from are still available.
To try to ensure that all required dependencies are included in the shell.nix file our CI jobs run with nix-shell --pure to make sure it does not use any system programs. However, this does not prevent e.g. using files or binaries specified by an absolute path, if it is available both locally and from CI. Preventing unneeded dependencies from being listed is also difficult, as that requires removing a dependency and re-running everything to make sure nothing breaks without it.