A critique of package managers(gingerbill.org)
gingerbill.org
A critique of package managers
https://www.gingerbill.org/article/2025/09/08/package-managers-are-evil/
219 comments
The Author isn't arguing for not using third party dependencies.
He's arguing for developers to be more conscious of the dependencies they use, by manually vetting and handling them. That screams "I've been down the package manager route and paid the price". Not inexperience.
> He's arguing for developers to be more conscious of the dependencies they use
"be careful all the time" doesn't scale. Half of all developers have below-average diligence, and that's a low bar. No-one is always vigilant, don't think that you're immune to human error.
No, you need tooling, automation to assist. It needs to be supported at the package manager side. Managing a site where many files are uploaded, and then downloaded many times is not a trivial undertaking. It comes with oversight responsibilities. If it's video you have to check for CSAM. If it's executable code, then you have to check for malware.
Package managers are not evil, but they are a tempting target and need to be secured. This can't just be an individual consumer responsibility.
I can't speak for other ecosystems, but some NuGet measures are here:
https://devblogs.microsoft.com/dotnet/building-a-safer-futur...
https://learn.microsoft.com/en-us/nuget/concepts/security-be...
I believe that there have been (a few) successful compromises of packages in NuGet, and that these have been mitigated. I don't know how intense the arms race is now.
"be careful all the time" doesn't scale. Half of all developers have below-average diligence, and that's a low bar. No-one is always vigilant, don't think that you're immune to human error.
No, you need tooling, automation to assist. It needs to be supported at the package manager side. Managing a site where many files are uploaded, and then downloaded many times is not a trivial undertaking. It comes with oversight responsibilities. If it's video you have to check for CSAM. If it's executable code, then you have to check for malware.
Package managers are not evil, but they are a tempting target and need to be secured. This can't just be an individual consumer responsibility.
I can't speak for other ecosystems, but some NuGet measures are here:
https://devblogs.microsoft.com/dotnet/building-a-safer-futur...
https://learn.microsoft.com/en-us/nuget/concepts/security-be...
I believe that there have been (a few) successful compromises of packages in NuGet, and that these have been mitigated. I don't know how intense the arms race is now.
> "be careful all the time" doesn't scale
Yes, this is the C attitude, where you provide no safety rails or poka-yokes or, indeed, package managers, and therefore you get a lot of fragile reimplementations of package managers (autoconf, anyone?). But you get to keep the satisfaction of blaming the users.
nuget is pretty good. It helps that packages tend to be substantial things, not left-pad.
Yes, this is the C attitude, where you provide no safety rails or poka-yokes or, indeed, package managers, and therefore you get a lot of fragile reimplementations of package managers (autoconf, anyone?). But you get to keep the satisfaction of blaming the users.
nuget is pretty good. It helps that packages tend to be substantial things, not left-pad.
> autoconf, anyone?
GNU Autoconf isn't a package manager, it's more an analogue to a setup executable on MS Windows, to detect where the user wants stuff to be installed, where the user has stuff already installed and which features the user wants.
GNU Autoconf isn't a package manager, it's more an analogue to a setup executable on MS Windows, to detect where the user wants stuff to be installed, where the user has stuff already installed and which features the user wants.
> It helps that packages tend to be substantial things, not left-pad.
Agree, this is IMHO also a better pattern. 1-liners or even 20-liners are not worth the overhead of extracting a package. Or of depending on a package.
Agree, this is IMHO also a better pattern. 1-liners or even 20-liners are not worth the overhead of extracting a package. Or of depending on a package.
"Half of all developers have below-average diligence" - a lot of this is also not developer choice, but environmental. So much software is developed and maintained in very constrained economic environments, often by solo devs who also have other responsibilities. The choice here often is trading some "diligence" for "meeting business requirements in the time / budget constraints" imposed by your employer.
Absolutely true, but still indicates the need for tooling, for automation, and for oversight at the the package store.
"developers, be more conscious" isn't going to fix all the issues. In general, there are not individual effort fixes to systemic issues.
"developers, be more conscious" isn't going to fix all the issues. In general, there are not individual effort fixes to systemic issues.
That's not package manager problem that's registry problem. NuGet is not having as many problems as NPM but also NPM is much more popular.
NPM is also quite a wild west when it comes to publishing packages, any kid can make an account and publish 'left-pad' kind of crap.
We already have quite safe and working setup with APT and software repositories for Debian, Ubuntu etc. While it is not so easy to publish your software to Debian, you get dedicated maintainer and all kinds of requirements you have to fulfill.
But this way all the issues with trust are if not mitigated, they are minimized and for example XZ Utils hack didn't make it to production systems and it took 3 years to prepare and pull it off.
NPM is also quite a wild west when it comes to publishing packages, any kid can make an account and publish 'left-pad' kind of crap.
We already have quite safe and working setup with APT and software repositories for Debian, Ubuntu etc. While it is not so easy to publish your software to Debian, you get dedicated maintainer and all kinds of requirements you have to fulfill.
But this way all the issues with trust are if not mitigated, they are minimized and for example XZ Utils hack didn't make it to production systems and it took 3 years to prepare and pull it off.
> That's not package manager problem that's registry problem
I do not think that the two are cleanly separable. They are client and server ends of the same system.
And I think my point is that I view it as more of a server (registry) and governance problem than the OP author does.
Despite the fact that my employer also has an internal package feed, the security of nuget.org and the central public feed is intrinsic to the security of the whole system.
Nuget was closer to the NPM end of the spectrum, but has tightened up considerably over time. Particularly the "Package ID Prefix Reservations" feature tells me that package names that start with certain words are owned by the relevant entity, be it "System." or "Azure." from Microsoft, or "AWS" from Amazon.
This is important as it's used to distribute SDKs and optional but standard library components and updates.
There is certainly junk on there, but not much load-bearing junk.
I do not think that the two are cleanly separable. They are client and server ends of the same system.
And I think my point is that I view it as more of a server (registry) and governance problem than the OP author does.
Despite the fact that my employer also has an internal package feed, the security of nuget.org and the central public feed is intrinsic to the security of the whole system.
Nuget was closer to the NPM end of the spectrum, but has tightened up considerably over time. Particularly the "Package ID Prefix Reservations" feature tells me that package names that start with certain words are owned by the relevant entity, be it "System." or "Azure." from Microsoft, or "AWS" from Amazon.
This is important as it's used to distribute SDKs and optional but standard library components and updates.
There is certainly junk on there, but not much load-bearing junk.
Article is discussing „package manager” as generic concept.
My argument was that this concept is not the problem.
Problem is in governance of NPM while NuGet or Maven are stricter and therefore it is registry governance problem.
But on the other hand NPM is much more popular than any other registry.
My argument was that this concept is not the problem.
Problem is in governance of NPM while NuGet or Maven are stricter and therefore it is registry governance problem.
But on the other hand NPM is much more popular than any other registry.
> Article is discussing „package manager” as generic concept.
My argument was that this concept is not the problem.
Problem is in governance of NPM
Then we're in agreement that the article's author has the wrong end of the stick, by focusing on the client end of the file transfer connection.
Then we're in agreement that the article's author has the wrong end of the stick, by focusing on the client end of the file transfer connection.
Huh? I am not saying the repositories have (or should have) no responsibility, but you are also responsible for your own actions.
The popularity of such repositories and package managers are due to users of them.
And the concepts are trivially separable in my opinion. A package manager uses a repo of packages to download from. You don't need a package manager to use a repo. And a package manager could be just local to your machine and thus not need an external repo either. I know in practice the two are combined but that doesn't mean they are not distinct concepts.
The popularity of such repositories and package managers are due to users of them.
And the concepts are trivially separable in my opinion. A package manager uses a repo of packages to download from. You don't need a package manager to use a repo. And a package manager could be just local to your machine and thus not need an external repo either. I know in practice the two are combined but that doesn't mean they are not distinct concepts.
But titled the post "package managers are evil"
> The term “evil” is being used partially hyperbolic to make a point.
Kind of bonkers this even needs to be said, and even then it's missed/ignored.
Kind of bonkers this even needs to be said, and even then it's missed/ignored.
The title is provocative and attention grabbing. -- It's completely fair game to react to the provocation rather than the substance of the article itself. (Or, rather, it's silly to use attention grabbing rhetoric, then complain that people paid attention to the rhetoric).
I'd prefer instead a more balanced title like "Remember to Consider the Costs When Using Package Managers", or whatever.
I'd prefer instead a more balanced title like "Remember to Consider the Costs When Using Package Managers", or whatever.
> It's completely fair game to react to the provocation rather than the substance of the article itself.
Yeah, but its down right stupid to do so.
The title isn't even misleading or part of a Motte-and-bailey argument.
People just hear "Package Managers are Evil" and assume that the author means you shouldn't use third party dependencies. Which is NOT what's being argued.
But I guess you'd know that, if you read passed the title.
Yeah, but its down right stupid to do so.
The title isn't even misleading or part of a Motte-and-bailey argument.
People just hear "Package Managers are Evil" and assume that the author means you shouldn't use third party dependencies. Which is NOT what's being argued.
But I guess you'd know that, if you read passed the title.
In the article, the author does say "I am not advocating to write things from scratch", while also describing third party dependencies as liabilities (e.g. security vulnerabilities), that people are too trusting of third party dependencies, that people overestimate the quality of third party dependencies.
I think you're splitting hairs if you're saying that these points from the article argue against package managers but don't argue against using third party dependencies.
I similarly think you're splitting hairs if to consider "package managers are useful?" and "third party dependencies are useful?" as distinct points.
I think you're splitting hairs if you're saying that these points from the article argue against package managers but don't argue against using third party dependencies.
I similarly think you're splitting hairs if to consider "package managers are useful?" and "third party dependencies are useful?" as distinct points.
Liability: "Something for which one is liable; an obligation, responsibility, or debt."
Third party dependencies absolutely are liabilities. You are liable to vet them, inspect their licenses and keep them updated while ensuring that they continue working with your existing code.
This is not something package managers help you do. Package managers like NPM make it trivial to skip these steps entirely.
What is being argued for, is a more thoughtful approach to handling third party dependencies. Or at the very least, the need for people to realise that there are costs associated with bringing third party dependencies into your codebase.
Its not splitting hairs at all. Its more of an presumption on the part of a large number of readers, that the 2 points argued conflate to "Package manager suck, because third party dependencies suck and you should write everything from scratch instead".
Third party dependencies absolutely are liabilities. You are liable to vet them, inspect their licenses and keep them updated while ensuring that they continue working with your existing code.
This is not something package managers help you do. Package managers like NPM make it trivial to skip these steps entirely.
What is being argued for, is a more thoughtful approach to handling third party dependencies. Or at the very least, the need for people to realise that there are costs associated with bringing third party dependencies into your codebase.
Its not splitting hairs at all. Its more of an presumption on the part of a large number of readers, that the 2 points argued conflate to "Package manager suck, because third party dependencies suck and you should write everything from scratch instead".
[deleted]
Sorry but I lack any respect for authors that use clickbaits. Call them put and move on seem the best approach.
Its not clickbait though.
You should try reading the article before passing judgement.
Its not like the article is called "5 facts that will make you hate package managers. Number 5 will shock you"
You should try reading the article before passing judgement.
Its not like the article is called "5 facts that will make you hate package managers. Number 5 will shock you"
It was clickbait because the article, which I did read, did not support the contention that package managers are evil. Therefore "evil" seems to be used in a hyperbolic way to grab attention, which makes it clickbait, specifically ragebait.
I wouldn't class it as clickbait myself, but I will stand by the use of the word "evil". I am using evil in the very old fashioned sense: the privation of the good. Is the title provocative? Yes. But that's the point of the article in general. I am trying to argue that they are a net bad with virtually no good upsides to them for the programming world as a whole. They've automated something at scale which should not have been automated. And to be clear, there is no solution to the problems they are trying to solve, rather it's all about trade-offs.
I a little annoyed that HackerNews post renamed it to "A critique of package managers" because that implies very different connotations. I'd view an article written like that as if I have some criticisms that could be addressed, rather than the entire concept being bad from the start.
I a little annoyed that HackerNews post renamed it to "A critique of package managers" because that implies very different connotations. I'd view an article written like that as if I have some criticisms that could be addressed, rather than the entire concept being bad from the start.
> I am trying to argue that they are a net bad with virtually no good upsides to them for the programming world as a whole.
What I'm saying is that you have failed in this argument. You hardly even attempt to make it. Thus clickbait.
You said "this is why I am saying it is evil, as it will send you to hell quicker."
Okay, so then it's up to you to prove this hell actually exists. But you don't. You just assert its existence -- "Dependency hell is a real thing which anyone who has worked on a large project has experienced." By framing it this way, you can dismiss anyone who claims to not have experienced this as not having sufficient experience. But reading the comments here, a lot of people have experienced a sort of "dependency hell" (the kind that's talked about in the wiki you link to) that is solved by package managers.
So that's why it's classed as clickbait -- you (admittedly) wrote a provocative headline that you don't even remotely back up.
FYI for the future since you're lamenting in many comments that people are misinterpreting you, this is why. Given that you don't really make an attempt to prove this dependency hell and package managers are evil, and you don't acknowledge anything good about them, it's reasonable to assume your bias is just that dependencies are evil at their core. It's actually the most charitable reading because otherwise you seem confused.
What I'm saying is that you have failed in this argument. You hardly even attempt to make it. Thus clickbait.
You said "this is why I am saying it is evil, as it will send you to hell quicker."
Okay, so then it's up to you to prove this hell actually exists. But you don't. You just assert its existence -- "Dependency hell is a real thing which anyone who has worked on a large project has experienced." By framing it this way, you can dismiss anyone who claims to not have experienced this as not having sufficient experience. But reading the comments here, a lot of people have experienced a sort of "dependency hell" (the kind that's talked about in the wiki you link to) that is solved by package managers.
So that's why it's classed as clickbait -- you (admittedly) wrote a provocative headline that you don't even remotely back up.
FYI for the future since you're lamenting in many comments that people are misinterpreting you, this is why. Given that you don't really make an attempt to prove this dependency hell and package managers are evil, and you don't acknowledge anything good about them, it's reasonable to assume your bias is just that dependencies are evil at their core. It's actually the most charitable reading because otherwise you seem confused.
Then again, there is a trope going back to Knuth - "Premature optimization is the root of all evil" - which is an argument that it is not clickbait, but merely applying a pattern in discussions about computer programming.
Hyperbole is just a pretty common thing for humans to do
> The title is provocative and attention grabbing. -- It's completely fair game to react to the provocation rather than the substance
No it isn't.
No it isn't.
The title of the article comes from the direct words I said in the video, of which the article is effectively a polished transcription of.
Your "more balanced title" isn't even close to what I am saying. I am saying that Package Managers are just bad and should not be used. Not "remember to consider the costs". The net cost is bad for everyone, that's why I said "evil".
Your "more balanced title" isn't even close to what I am saying. I am saying that Package Managers are just bad and should not be used. Not "remember to consider the costs". The net cost is bad for everyone, that's why I said "evil".
I guess clickbait is evil
I disagree with this take. There should be just more governance on the registry side of things.
For NuGet or Maven I think dependency hell is not something you run into and I don’t have package manager manager for those languages.
There should be enough trust just like I can do sudo apt install.
His take screams „I want to push my niche approach and promote my language from my Ivory Tower of language creator”. He still might not have any relevant experience building businesses line software just like O don’t have experience with building compilers or languages.
For NuGet or Maven I think dependency hell is not something you run into and I don’t have package manager manager for those languages.
There should be enough trust just like I can do sudo apt install.
His take screams „I want to push my niche approach and promote my language from my Ivory Tower of language creator”. He still might not have any relevant experience building businesses line software just like O don’t have experience with building compilers or languages.
Inexperience of an author who develops quite successful programming language for like 10 years?
Quite a bold statement.
Actually his perspective is quite reasonable. Go is in the other part of the spectrum than languages encouraging "left-pad"-type of libraries, and this is a good thing.
Actually his perspective is quite reasonable. Go is in the other part of the spectrum than languages encouraging "left-pad"-type of libraries, and this is a good thing.
Not to mention we've have had decades of software development without automated package managers and people did just fine.
I've seen plenty of intelligent people acting pretty stupid.
As my psychology professor used to say. "Smart is how efficiently use your intelligence. Or don't."
So someone pretty low IQ can be smart - Forrest Gump. Or someone high IQ can be dumb occasionally - a professor so very attuned to his research topic at expense of everything else.
As my psychology professor used to say. "Smart is how efficiently use your intelligence. Or don't."
So someone pretty low IQ can be smart - Forrest Gump. Or someone high IQ can be dumb occasionally - a professor so very attuned to his research topic at expense of everything else.
How is this relating to the alleged inexperience of the original author? Not sure what do you mean.
The above comment is merely pointing out that a 10y+ experienced language designer can still have naive viewpoints on application development. Anyone who's built a non-trivial userspace application knows that realistically you'll have to reach outside a particular languages standard library in most cases to provide value without reinventing wheels.
In other words: when someone's knowledge is disproportionately localized/siloed to their prospective subfield or domain of expertise, it does not necessitate generalization to others.
I'm certainly not saying this is the case with this particular individual, as I'm personally not familiar with their background. I'm simply stating that it's a plausible explanation for when experts in one domain make naive assertions about another domain they might not have the same experience in.
In other words: when someone's knowledge is disproportionately localized/siloed to their prospective subfield or domain of expertise, it does not necessitate generalization to others.
I'm certainly not saying this is the case with this particular individual, as I'm personally not familiar with their background. I'm simply stating that it's a plausible explanation for when experts in one domain make naive assertions about another domain they might not have the same experience in.
I don't buy it.
A guy designing and then implementing a programming language has a much bigger chance to put a lot of rational thinking into the tooling like dependency manager, than a typical language consumer, who can and often is easily falling into the languages emo wars.
A guy designing and then implementing a programming language has a much bigger chance to put a lot of rational thinking into the tooling like dependency manager, than a typical language consumer, who can and often is easily falling into the languages emo wars.
As the original article points out, not all languages come out of the box with a sane/rationally designed dependency manager. I can think of only a handful in that category. The vast majority of languages fall short and rely on secondary community projects to prop up the dependency management for the language: maven, gradle, npm, pip/pypi, now uv, etc.
Language designers in general terms will fall into the "more knowledgeable than the average developer"category , but let's not pretend they're anything but mere mortals like the rest of us.
NGL Ive somehow lost the thread and can't tell if we're talking about language integrated dependency managers in the abstract (in the OP), or specifically speaking about golang, odin or something else. I don't know what the emo wars are specifically in reference to but I think we jumped the shark here.
Language designers in general terms will fall into the "more knowledgeable than the average developer"category , but let's not pretend they're anything but mere mortals like the rest of us.
NGL Ive somehow lost the thread and can't tell if we're talking about language integrated dependency managers in the abstract (in the OP), or specifically speaking about golang, odin or something else. I don't know what the emo wars are specifically in reference to but I think we jumped the shark here.
Put another way: what makes this time different? How does this designer's proclivity and push towards X learn from our collective past mistakes; what does it bring to the table?
Yes dependency hell is "bad", but we have several language and package management systems today from ninja to uv that make various, obvious trade offs. Optimizing developer time, ergonomics, reproducible builds, configuration complexity are just some of the axes these pre-existing systems focus on.
If you're extremely lucky you get to pick a system that aligns with your style of work and ideals for how software should be built. If you're not, and like the rest of us, you get stuck with everyone else's poor decisions and are forced to make do. All code is legacy code given the right time horizon, so think about software with all those manual dependencies included on disk and nowhere else. How do you safely apply those required security fixes, etc. Don't be user hostile, this will just lead to our past sins like the C of old.
From a purist perspective, you can forgo all other software that you have not written in-house / or does not come with the standard library. This is the monk approach, but outside a few niche work environments that's untenable.
Yes dependency hell is "bad", but we have several language and package management systems today from ninja to uv that make various, obvious trade offs. Optimizing developer time, ergonomics, reproducible builds, configuration complexity are just some of the axes these pre-existing systems focus on.
If you're extremely lucky you get to pick a system that aligns with your style of work and ideals for how software should be built. If you're not, and like the rest of us, you get stuck with everyone else's poor decisions and are forced to make do. All code is legacy code given the right time horizon, so think about software with all those manual dependencies included on disk and nowhere else. How do you safely apply those required security fixes, etc. Don't be user hostile, this will just lead to our past sins like the C of old.
From a purist perspective, you can forgo all other software that you have not written in-house / or does not come with the standard library. This is the monk approach, but outside a few niche work environments that's untenable.
> than a typical language consumer, who can and often is easily falling into the languages emo wars.
How is ginger bill excluded from this group? No one is more invested in a language than its creator(s).
Sure, he might have given it a lot of thought, but he came up with some completely bonkers conclusions. If you don't want dependencies, DON'T IMPORT DEPENDENCIES. Don't make your dependencies extremely hard to add.
How is ginger bill excluded from this group? No one is more invested in a language than its creator(s).
Sure, he might have given it a lot of thought, but he came up with some completely bonkers conclusions. If you don't want dependencies, DON'T IMPORT DEPENDENCIES. Don't make your dependencies extremely hard to add.
Yeah when speaking about emotions: the amount of emo reactions here, including shouting with all caps, lets me think we've fallen into the old story: the author kind-of praised Go, but it's unfashionable here; the contrary, the fad here is to hate Go, so the author needed to get his hate. As simple as that. The rest is just trying to hide the hate under seemingly rational arguments.
Yawn.. saw it before...next, please
Yawn.. saw it before...next, please
Yeah, god forbid you use bolding to emphasize your phrase on this site. It's considered emotinal response, but yours is purely logical?
I'm glad you saw through me like a Superman through a lead book. Which is to say, not at all. I wasn't even thinking of Go. Where did this come from? I never mentioned Go. I don't use it or know how it does its packaging.
Are you projecting your feelings onto me as a sort of substitute for the HN gestalt? The discussion was about package managers being evil.
Now please return to the topic at hand.
Let's say you have NPM package manager. What prevents you a rational individual from saying:
I'm glad you saw through me like a Superman through a lead book. Which is to say, not at all. I wasn't even thinking of Go. Where did this come from? I never mentioned Go. I don't use it or know how it does its packaging.
Are you projecting your feelings onto me as a sort of substitute for the HN gestalt? The discussion was about package managers being evil.
Now please return to the topic at hand.
Let's say you have NPM package manager. What prevents you a rational individual from saying:
{
"depedencies": {}
}You did not had Go in mind, but the [original commenter](https://news.ycombinator.com/item?id=45167394#45168550) apparently did as he has quoted exactly the line about Go. Then you (and me) commented under that comment.
So my snarky remark was about him, not about you. I think it's ok to rewind the tree up to see what is about whom. I can sincerely apologize that I have put replies to two distinct human beings, you and that other commenter, in one paragraph. Honestly, I can see that could let to confusion.
I think we can stop now..
So my snarky remark was about him, not about you. I think it's ok to rewind the tree up to see what is about whom. I can sincerely apologize that I have put replies to two distinct human beings, you and that other commenter, in one paragraph. Honestly, I can see that could let to confusion.
I think we can stop now..
I have? Pray tell.
Have what? Heavily invested in language you're building? I think that's a given.
Not clear-headed about this? https://old.reddit.com/r/programming/comments/1nbkwzt/packag...
Not clear-headed about this? https://old.reddit.com/r/programming/comments/1nbkwzt/packag...
> gingerbill[S] 1 point 2 hours ago
> So a tool that enables evil is not an evil tool?
You see you just completely missed my replies to that too.
> Let's put it this way, what does a package manager specifically (not the other distinctions I make in the article) do (other than enable bad laziness and lack of proper vetting) that is actually good?
https://old.reddit.com/r/programming/comments/1nbkwzt/packag...
> Let's put it this way, what does a package manager specifically (not the other distinctions I make in the article) do (other than enable bad laziness and lack of proper vetting) that is actually good?
https://old.reddit.com/r/programming/comments/1nbkwzt/packag...
And you missed the retort to that reply as well. It's a force multiplier and a time saver. Same as with any tool.
And to reply to your next post:
I'd rather use a hammer even if there is a higher chance to smack my fingers than to have to hit a nail repeatedely with my head.
And to reply to your next post:
> Getting to hell quicker is not a good thing. "Emerge on the other side quickly", the other side is still hell, you haven't emerged out of it.
I'd rather use a hammer even if there is a higher chance to smack my fingers than to have to hit a nail repeatedely with my head.
Thank you?
Is it "quite successful"? How would I distinguish such a "quite successful" language from say Hare or V or are these all "successful" in your mind?
I know very few people using Hare, especially since it only works on "FOSS platforms". And I will still maintain that V is vapourware. They still have the same false claims on the website that they've had from the beginning for ~6 years.
Odin is "successful enough" so far. Also, you know about it, so that says something.
Odin is "successful enough" so far. Also, you know about it, so that says something.
I know about Hare and V too, so, then what exactly does it say for me to know about a programming language? Not much.
I have technically written more Odin than Hare (one Godbolt example, arguably two if you count my explaining how to modify the example to illustrate another problem) but that just means I have more justification to say I don't like it.
I've written a lot more Scheme and I had so thoroughly forgotten writing Scheme that I had to go read the source for myself when I got email about it decades later to be sure it wasn't just a coincidence of author names.
I'm not convinced there is space for any of the "C successor" languages in the twenty-first century and in the event space is made or given for one I doubt there'll somehow be room for more. So with today's field I would bet on Zig.
I have technically written more Odin than Hare (one Godbolt example, arguably two if you count my explaining how to modify the example to illustrate another problem) but that just means I have more justification to say I don't like it.
I've written a lot more Scheme and I had so thoroughly forgotten writing Scheme that I had to go read the source for myself when I got email about it decades later to be sure it wasn't just a coincidence of author names.
I'm not convinced there is space for any of the "C successor" languages in the twenty-first century and in the event space is made or given for one I doubt there'll somehow be room for more. So with today's field I would bet on Zig.
Odin is not trying to be a "C successor" rather as the website states: "Odin is the C alternative for the Joy of Programming".
And there doesn't have to be "one winner". This isn't Highlander. It is just wonderful that there is now choice in this domain beyond just the old and obvious.
And there doesn't have to be "one winner". This isn't Highlander. It is just wonderful that there is now choice in this domain beyond just the old and obvious.
There's commercial software produced in Odin that has made money. Not sure the same can be said of Hare or V.
Why the need for distinguishing and an urge for comparison? We're talking about Odin, that's it. As a project that (as I understand) didn't have any big corp investment, it's impressive.
The claim was that we should assume Odin's author is experienced because he wrote a successful language. If we've decided it doesn't matter whether it's successful then the claim was entirely circular. Yes, the creator of Odin is indeed its creator. Nobody was disputing that.
Even if he was unsuccesful but tried long enough, that would still make him experienced.
I don't see how a detailed comparing of language successfullness would bring anything valuable to the point being made about the author being experienced. That seemed just a noise.
I don't see how a detailed comparing of language successfullness would bring anything valuable to the point being made about the author being experienced. That seemed just a noise.
To me, this whole comment just screams inability to steelman.
Sure... and, to prove your point, Go has a package manager too (although it's a relatively new addition). But Go still follows a "batteries included" approach, where "standard" stuff (yes, even database handling) is handled by the standard library. Which still leaves lots of other things for which you need third party packages, but those will be typically far fewer than in other languages.
I think the argument presented, is that whatever a Go package does, it does well.
Btw the Js ecosystem also has quite a few good packages (and a ton of terrible ones, including some which everyone seems to consider as the gold standard).
Btw the Js ecosystem also has quite a few good packages (and a ton of terrible ones, including some which everyone seems to consider as the gold standard).
I don't see the value in making it even harder to build software. I want to make things. Downloading a dependency manually and then cursing at the compiler because "it's right there! why won't it load!!" is just gonna make me want to build software less.
Anyone I want to work with on a project is going to have to have the same frustration and want to work on the project less. Only even more because you see they downloaded version 2.7.3-2 but the version I use is 2.7.3-1.
Anyone I want to work with on a project is going to have to have the same frustration and want to work on the project less. Only even more because you see they downloaded version 2.7.3-2 but the version I use is 2.7.3-1.
> Downloading a dependency manually and then cursing at the compiler because "it's right there! why won't it load!!"
Odin's compiler knows what a package is and will compile it into your program automatically.
Odin's compiler knows what a package is and will compile it into your program automatically.
Isn't that a (built-in) package manager if it works for general packages? Or does it work only for selected dependencies?
From what I've seen. The Odin has three package collections: `base`, `core` and `vendor`.
`base` is intrinsically necessary to port Odin. `core` seems to be its standard library, your `libc`, `xml`, etc.
And `vendor` is everything else. So you basically get the Python's '`core` is where packages go to die' approach iff they take backwards compatibility seriously. Otherwise, they have breaking changes mid-language version change.
EDIT: Package collections not packages per gingerBill.
`base` is intrinsically necessary to port Odin. `core` seems to be its standard library, your `libc`, `xml`, etc.
And `vendor` is everything else. So you basically get the Python's '`core` is where packages go to die' approach iff they take backwards compatibility seriously. Otherwise, they have breaking changes mid-language version change.
EDIT: Package collections not packages per gingerBill.
Those are library/package collections which contain multiple different packages, not the packages themselves.
And we will take backwards compatibility seriously when we hit 1.0, and only "break" on major versions.
And we will take backwards compatibility seriously when we hit 1.0, and only "break" on major versions.
> And we will take backwards compatibility seriously when we hit 1.0, and only "break" on major versions.
I'm talking about post 1.0 language choices:
- Choose backwards compatibility. Packages frozen in time, you get "Packages go to std to die." - Choose to break backwards compatibility. The ecosystem is split, some choose to go Odin 2 some are Odin 3.
I'm talking about post 1.0 language choices:
- Choose backwards compatibility. Packages frozen in time, you get "Packages go to std to die." - Choose to break backwards compatibility. The ecosystem is split, some choose to go Odin 2 some are Odin 3.
I already know all that, that's why we are being very conservative and slow when it comes to figuring out what is meant to be in 1.0.
I don't think it will matter. There will be things people want that Odin won't have in the vendor package list.
Then those people will have to manage dependencies, which is a hell on its own. Which will cause problems. Because people are super lazy so they will automate it. In the end only thing no package manager gets you is multiple package managers to juggle.
Many languages started without package managers and eventually got them - Java, JavaScript, Python, C, C++
Then those people will have to manage dependencies, which is a hell on its own. Which will cause problems. Because people are super lazy so they will automate it. In the end only thing no package manager gets you is multiple package managers to juggle.
Many languages started without package managers and eventually got them - Java, JavaScript, Python, C, C++
Again, I am not against third party packages, and manual management of dependencies just slows down your progression to hell. There is no "solution" to this problem, only trade-offs.
I know people are lazy and will automate hell. That's the entire point of the article: not everything that can be automated ought to be automated.
And the argument about multiple package managers to juggle is only the case IFF there are multiple competing ones, which with Odin, I honestly doubt it would happen if we enforced what a package is in the language. I just don't want to officially endorse one ever because I do view them to be evil.
And I don't care many languages started without them, I am not going to give in.
I know people are lazy and will automate hell. That's the entire point of the article: not everything that can be automated ought to be automated.
And the argument about multiple package managers to juggle is only the case IFF there are multiple competing ones, which with Odin, I honestly doubt it would happen if we enforced what a package is in the language. I just don't want to officially endorse one ever because I do view them to be evil.
And I don't care many languages started without them, I am not going to give in.
> I honestly doubt it would happen if we enforced what a package is in the language. I just don't want to officially endorse one ever because I do view them to be evil.
I don't see how changing package definition is going to help. JS had no concept of package and it was bolted on with NPM. If Odin becomes big enough, the community will override the will of the author.
Plus I don't see huge benefits to not having a package manager other than saving disk space.
Security isn't that much meaningfully better than NPM.
Trust problem exists regardless of package manager.
And people aren't far to trusting, but far too lazy. And importing packages gets job done quickly.
I don't see how changing package definition is going to help. JS had no concept of package and it was bolted on with NPM. If Odin becomes big enough, the community will override the will of the author.
Plus I don't see huge benefits to not having a package manager other than saving disk space.
Security isn't that much meaningfully better than NPM.
Trust problem exists regardless of package manager.
And people aren't far to trusting, but far too lazy. And importing packages gets job done quickly.
> If Odin becomes big enough, the community will override the will of the author.
Dunno, hanging out in the Odin discord, it definitely attracts a crowd that thinks similarly to Bill. All the "automate everything" crowd have definitely gone to Zig, where you can create automated monstrosities with the comptime stuff and build.zig files. And the crowd that likes NPM gravitates to Rust. So Odin is just fine IMO. People on the discord share libraries that actually do things, versus an entire dependency to write a few basic procedures.
And speaking of JavaScript, nowadays ES6 does have an idea of what packages/modules/libraries are and it's so much better. All my JS dependencies for my Rails projects are just .esm.js files. I choose modules carefully, don't pull in obfuscated files, read the source, so I have 2 JS dependencies in one project and a single one in another, I write the rest myself in vanilla JS and life is great.
Dunno, hanging out in the Odin discord, it definitely attracts a crowd that thinks similarly to Bill. All the "automate everything" crowd have definitely gone to Zig, where you can create automated monstrosities with the comptime stuff and build.zig files. And the crowd that likes NPM gravitates to Rust. So Odin is just fine IMO. People on the discord share libraries that actually do things, versus an entire dependency to write a few basic procedures.
And speaking of JavaScript, nowadays ES6 does have an idea of what packages/modules/libraries are and it's so much better. All my JS dependencies for my Rails projects are just .esm.js files. I choose modules carefully, don't pull in obfuscated files, read the source, so I have 2 JS dependencies in one project and a single one in another, I write the rest myself in vanilla JS and life is great.
> Dunno, hanging out in the Odin discord, it definitely attracts a crowd that thinks similarly to Bill.
Sure, hence the big enough part. If you get big enough, you'll get people who are using it as a day job language, not their special darling. Having used JS and Java without package managers in a professional setting, they sucked to use.
You import a package, run the main program, see compiler/browser errors then search local repo or th Net for the missing library. Essentially you're the package manager. Which does little for bloat. You can still have a folder and import stuff en masse.
Sure, hence the big enough part. If you get big enough, you'll get people who are using it as a day job language, not their special darling. Having used JS and Java without package managers in a professional setting, they sucked to use.
You import a package, run the main program, see compiler/browser errors then search local repo or th Net for the missing library. Essentially you're the package manager. Which does little for bloat. You can still have a folder and import stuff en masse.
So they are trying the Linux distribution model of packages, right? (Compare `vendor` with, say, Ubuntu's `universe`.)
It doesn't necessarily "manage" the packages. It just sees them in your project and compiles them. You manage them yourself.
If that's what happens, I think in the following claim:
> Odin's compiler knows what a package is and will compile it into your program automatically.
...the word "automatically" should be dropped. Of course compilers compile any supplied dependency "automatically", but it is so obvious that we don't often use the adverb just for that.
> Odin's compiler knows what a package is and will compile it into your program automatically.
...the word "automatically" should be dropped. Of course compilers compile any supplied dependency "automatically", but it is so obvious that we don't often use the adverb just for that.
> Of course compilers compile any supplied dependency "automatically", but it is so obvious that we don't often use the adverb just for that.
They often don't though. Rust, C, C++ need either long command line invocations or a build system for anything beyond hello world. Zig needs a build file for anything beyond hello world.
With Odin, you just invoke "odin build ." and all your dependencies are taken in without needing a build system, build file, make file, etc...
They often don't though. Rust, C, C++ need either long command line invocations or a build system for anything beyond hello world. Zig needs a build file for anything beyond hello world.
With Odin, you just invoke "odin build ." and all your dependencies are taken in without needing a build system, build file, make file, etc...
We call that a build system. It is not like that there is no build system; you have an integrated build system that is optimized for typical situations. Which is great by its own, but other languages and toolings would have optimized other metrics (for example, you can't ignore Cargo when talking about Rust's build system) so it is not a fair comparison.
It is a build system in the technical sense but it's hard to explain to people because they expect it to be separate from the language entirely. If I said Odin had a build system, they'd be expecting an external build script. And when you say you don't need that, they usually get really confused.
So how do you explain such a system to someone? This is a genuine question I am not sure how to answer.
So how do you explain such a system to someone? This is a genuine question I am not sure how to answer.
I mean, is a "build system" even real? Or does it just exist because of a shortcoming with a compiler? All "compilers" have multiple steps usually invoking other programs at some point: parsing, actual code transformation, linking, etc..., some also find packages, some rely on an external tool or long command line invocations.
But yes, Odin builds it into their compiler. Rust doesn't but does have Cargo. Both are easy, as far as typical usage goes. Rust automates dependency management, Odin doesn't automate it per se but does make it easy. Which is what the whole discussion is about. A bunch of HNers whining that Odin makes it too hard, even though everyone sane uses Git anyway, and you can add dependencies using Git, and Odin will compile them without a build tool.
So for a Rust project you use Cargo + Git, for an Odin project you use Odin + Git, for a C/C++ project you use Meson (or something else if you hate life) + Git. In the end it's mostly the same, Bill just doesn't seem to want to deal with an NPM or Crates.io situation (and fair enough!).
But yes, Odin builds it into their compiler. Rust doesn't but does have Cargo. Both are easy, as far as typical usage goes. Rust automates dependency management, Odin doesn't automate it per se but does make it easy. Which is what the whole discussion is about. A bunch of HNers whining that Odin makes it too hard, even though everyone sane uses Git anyway, and you can add dependencies using Git, and Odin will compile them without a build tool.
So for a Rust project you use Cargo + Git, for an Odin project you use Odin + Git, for a C/C++ project you use Meson (or something else if you hate life) + Git. In the end it's mostly the same, Bill just doesn't seem to want to deal with an NPM or Crates.io situation (and fair enough!).
> All "compilers" have multiple steps usually invoking other programs at some point: parsing, actual code transformation, linking, etc..., some also find packages, some rely on an external tool or long command line invocations.
There is no technical reason the compiler can do the job of build system, but they are typically separated because of separation of concerns. Rustc needs not be tightly coupled with Cargo---it just has to understand enough of package concepts (`--extern`) for the actual compilation, so they can be independently managed and evolve. Odin's would be the polar oppsite, while Zig's approach is somewhere in the middle (compiler-as-a-library).
There is no technical reason the compiler can do the job of build system, but they are typically separated because of separation of concerns. Rustc needs not be tightly coupled with Cargo---it just has to understand enough of package concepts (`--extern`) for the actual compilation, so they can be independently managed and evolve. Odin's would be the polar oppsite, while Zig's approach is somewhere in the middle (compiler-as-a-library).
This seems like a loss of separation of concerns. As a result Odin's compiler is full of completely unrelated stuff because it takes on this far larger role supervising everything about a project not just being a compiler.
Both Zig and Rust supply all of what you needed in the box, so all that Odin is doing here is commingling these features inside a single executable - there's no end user benefit that I can see.
Both Zig and Rust supply all of what you needed in the box, so all that Odin is doing here is commingling these features inside a single executable - there's no end user benefit that I can see.
?? Zig's compiler does way more, from automatically generating bindings to C files to including the entire C toolchain in its executable to running Zig build files (which allow you to do stuff like fetch files from the Internet) and being a compiler.
Odin's compiler does more than rustc or clang, about the same as javac and less than Zig or Go's executables.
Odin's compiler does more than rustc or clang, about the same as javac and less than Zig or Go's executables.
This is an argument for a good build system, not a package manager.
These aren’t always separate.
Some distos might try to support multiple versions of a library. That could require installing it to different prefixes instead of the default. Thus, the build system will have to comprehend that.
Some distos might try to support multiple versions of a library. That could require installing it to different prefixes instead of the default. Thus, the build system will have to comprehend that.
Not everything in the world revolves around Linux. Distros terrible choices around shared library architecture has nothing to do with build systems for most languages.
True. The most popular Unix is after all macOS, followed shortly by WSL2
There are way more servers running linux than there are macos installs. And wsl2 is linux.
Number of servers isn’t the right metric imho.
Literally all code I write runs on Windows, macOS, Android, and Linux. In roughly that order of priority. No I do not and will not use WSL2, it’s an abomination.
Literally all code I write runs on Windows, macOS, Android, and Linux. In roughly that order of priority. No I do not and will not use WSL2, it’s an abomination.
Build systems are yet another special circle of hell.
This article, although is trying to provide some arguments as for why package managers are "evil", I found the argumentation pretty weak/non-descriptive. It's good if you have the experiences that confirm a specific point of view, but I think these experiences need to be explained in some more detail, because people reading your article may have similar experiences and therefore would find it hard to agree with your points - just like me.
To give a concrete example, you said that javascript does not have a definition of a "package" in its langauge. But what does that really mean, and why should it lead to package manager managers? Because for me, a person who has worked with javascript just a little bit, I know package.json exists and most of the package managers I've worked with agree on what the contents of this file mean. If we limit our understanding to just npm, yarn and probably bun, we don't see how that causes or contributes to the dependency hell problem (sure it exists, but how?).
You said that Go mitigates the issue of dependency hell to some degree, but this is an interesting thought, give it more exploration! Why should something like Go not have this problem not be not as severe as in Javascript?
I may not remember the details of what you said in the article and I would like to check, but currently I can't access the site because it times-out for me.
To give a concrete example, you said that javascript does not have a definition of a "package" in its langauge. But what does that really mean, and why should it lead to package manager managers? Because for me, a person who has worked with javascript just a little bit, I know package.json exists and most of the package managers I've worked with agree on what the contents of this file mean. If we limit our understanding to just npm, yarn and probably bun, we don't see how that causes or contributes to the dependency hell problem (sure it exists, but how?).
You said that Go mitigates the issue of dependency hell to some degree, but this is an interesting thought, give it more exploration! Why should something like Go not have this problem not be not as severe as in Javascript?
I may not remember the details of what you said in the article and I would like to check, but currently I can't access the site because it times-out for me.
package.json is a convention, not a language definition, hence package managers may implement "package" management differently; in reality conventions are followed, until they aren't, and that's where hell begins (if something can be abused - it will be); go and odin define package in the language itself as a folder containing source files, and they mitigate many management issues by just having a good standard library, so you wouldn't need as many packages to begin with;
In general, I think the dependency hate is overblown. People hear about problems with dependencies because dependencies are usually open source code used by a lot of people so it is public and relevant. You don't hear as much about problems in the random code of one particular company unless it ends up in a high profile leak. For example, something like the heartbleed bug was a huge deal and got a lot of press, but imagine how many issues we would be in if everyone was implementing their own SSL. Programmers often don't follow best practices when they do things on their own. That is how you end up with things like SQL injection attacks in 2025.
Dependencies do suck but it is because managing a lot of complicated code sucks. You need some way to find issues over time and keep things up to date. Dependencies and package managers at least offer us a path to deal with problems. If you are managing your own dependencies, which I imagine would mean vendoring, then you aren't going to keep these dependencies up to date. You aren't going to find out about exploits in the dependencies and apply them.
Dependencies do suck but it is because managing a lot of complicated code sucks. You need some way to find issues over time and keep things up to date. Dependencies and package managers at least offer us a path to deal with problems. If you are managing your own dependencies, which I imagine would mean vendoring, then you aren't going to keep these dependencies up to date. You aren't going to find out about exploits in the dependencies and apply them.
> imagine how many issues we would be in if everyone was implementing their own SSL.
No, the alternative is to imagine how many issues we would be in if every project pulled in 5 different SSL libraries. Having one that everybody uses and that is already installed on everyone's system is avoiding dependency hell. Even better if it's in stdlib.
No, the alternative is to imagine how many issues we would be in if every project pulled in 5 different SSL libraries. Having one that everybody uses and that is already installed on everyone's system is avoiding dependency hell. Even better if it's in stdlib.
I see this a lot with Rust where I will depend on one or two external crates for a simple application and then I am shocked to see dozens of dependencies being pulled in when I go to build. I actually think Cargo's support for feature gates and conditional compilation could in theory be a strong mitigation against this as crates can avoid pulling in dependencies unless you actually need a feature that relies on them, but in practice it doesn't seem to work that way as I often see these complaints about Rust.
I sympathise with the arguments but IMO laziness will always win out. If Rust didn't have Cargo to automate dependency hell, someone would create a third party script to fill the gap.
I sympathise with the arguments but IMO laziness will always win out. If Rust didn't have Cargo to automate dependency hell, someone would create a third party script to fill the gap.
It is an organizational not a technical problem.
When I worked at Google every single dependency was strictly vendored (and not in the mostly useless way that Cargo vendors things). There was generally only one version of a dep in the mono repo, and if you wanted something.. you generally got to own maintaining it, and you had to make sure it worked for every "customer" -- the giant CI system made sure that you knew if an upgrade would break things. And you reached out to stakeholders to manage the process. Giant trains of dependencies were not a thing. You can do that when you have seemingly infinite budget.
But technology can indeed make it worse. I love Rust, but I'm not a fan of the loose approach in Cargo and esp Crates.io, which seems to have pulled inspiration from NPM -- which I think is more of a negative than positive example. It's way too easy to make a mess. Crates.io is largely unmoderated, and its namespace is full of abandoned or lightly maintained projects.
It's quite easy to get away with a maze of giant transitive deps w/ Cargo because Rust by default links statically, so you don't usually end up in DLL hell. But just doing cargo tree on the average large Rust project is a little depressing -- to see how many separate versions of random number generators, SHA256, MD5, etc libs you end up with in a single linkage. It may not be the case that every single one is contributing to your binary size... but it's also kind of hard to know.
Understanding the blast radius of potential issues that come from unmoderated 3rd-party deps is I think something that many engineers have to learn the hard way. When they deal with a security vulnerability, or a fundamental incompatibility issue, or have to deal with build time and binary size explosions.
I wish there was a far more mature approach to this in our industry. The trend seems to be going in the opposite direction.
When I worked at Google every single dependency was strictly vendored (and not in the mostly useless way that Cargo vendors things). There was generally only one version of a dep in the mono repo, and if you wanted something.. you generally got to own maintaining it, and you had to make sure it worked for every "customer" -- the giant CI system made sure that you knew if an upgrade would break things. And you reached out to stakeholders to manage the process. Giant trains of dependencies were not a thing. You can do that when you have seemingly infinite budget.
But technology can indeed make it worse. I love Rust, but I'm not a fan of the loose approach in Cargo and esp Crates.io, which seems to have pulled inspiration from NPM -- which I think is more of a negative than positive example. It's way too easy to make a mess. Crates.io is largely unmoderated, and its namespace is full of abandoned or lightly maintained projects.
It's quite easy to get away with a maze of giant transitive deps w/ Cargo because Rust by default links statically, so you don't usually end up in DLL hell. But just doing cargo tree on the average large Rust project is a little depressing -- to see how many separate versions of random number generators, SHA256, MD5, etc libs you end up with in a single linkage. It may not be the case that every single one is contributing to your binary size... but it's also kind of hard to know.
Understanding the blast radius of potential issues that come from unmoderated 3rd-party deps is I think something that many engineers have to learn the hard way. When they deal with a security vulnerability, or a fundamental incompatibility issue, or have to deal with build time and binary size explosions.
I wish there was a far more mature approach to this in our industry. The trend seems to be going in the opposite direction.
In many ways traditional Linux distros operate on similar model as I imagine googles monorepo. Both aim to this "globally consistent" dependency situation where you have one version of each library and you patch up things from upstream when they don't fit.
I feel we need more of these kinds of distros so you don't need to manage dependencies directly from upstream and deal with the integration effort yourself. What if we had a Rust disto following this same model, where there is only one version of each dep, some reasonable curation, and also you had nice clear release cycles? I feel that could real boon for the industry.
I feel we need more of these kinds of distros so you don't need to manage dependencies directly from upstream and deal with the integration effort yourself. What if we had a Rust disto following this same model, where there is only one version of each dep, some reasonable curation, and also you had nice clear release cycles? I feel that could real boon for the industry.
Unfortunately I think it'd be too much putting the toothpaste back in the tube at this point. The way people are used to working in Rust is like filling their shopping cart with crate treats and then gluing them together.
I dunno maybe what is needed is a crates.io alternative that is highly highly moderated and highly highly selective. A subscription service with a paid staff that manages the packages and makes sure their deps are minimal, consistent with each other, secure, etc.
I can see that being a service that some corporations might pay for. I just came off a gig at a medical devices company that was using Rust and the software BoM side of things kept me up at night. The list of dependencies in the root workspace was long, and in my imagination, full of terrors.
Maybe they'll be fine, but it's not a practice I would recommend if I were starting such a project from scratch.
I dunno maybe what is needed is a crates.io alternative that is highly highly moderated and highly highly selective. A subscription service with a paid staff that manages the packages and makes sure their deps are minimal, consistent with each other, secure, etc.
I can see that being a service that some corporations might pay for. I just came off a gig at a medical devices company that was using Rust and the software BoM side of things kept me up at night. The list of dependencies in the root workspace was long, and in my imagination, full of terrors.
Maybe they'll be fine, but it's not a practice I would recommend if I were starting such a project from scratch.
> If Rust didn't have Cargo to automate dependency hell, someone would create a third party script to fill the gap.
Possibly but not guaranteed. Some other languages without a built in package manager haven't had an external one manage to take over the ecosystem, most (in)famously C and C++, while others have.
Possibly but not guaranteed. Some other languages without a built in package manager haven't had an external one manage to take over the ecosystem, most (in)famously C and C++, while others have.
Most language users will follow the "spirit" of the language - e.g. Bill is against package managers, people who use his language mostly agree with his ideas, and there's not a huge standard Odin package manager.
I rather appreciate that C and C++ don't have a default package manager that took over - yes, integrating libraries is a bit more difficult, but we also have a lot of small, self-contained libraries that just "do the thing" without pulling in a library that does colored text for logging, which pulls in tokio, which pulls in mio, which pulls in wasi, which pulls in serde, which is insane.
I rather appreciate that C and C++ don't have a default package manager that took over - yes, integrating libraries is a bit more difficult, but we also have a lot of small, self-contained libraries that just "do the thing" without pulling in a library that does colored text for logging, which pulls in tokio, which pulls in mio, which pulls in wasi, which pulls in serde, which is insane.
C and C++ do have package managers. It's just that these languages evolved for OS implementation and also that these package managers are old and stable so they have support for a lot of languages, so that you probably know them as OS package managers.
Which is the correct thing to do as things written in language X can depend on things written in language Y.
The package manager for C/C++ is apt, or rpm, or whatever package manager your system uses. These package managers were designed for the world of C/C++ software so it's less surprising that these languages haven't found as much of a push towards language package managers.
Yes, but that's quite different from how language package managers are used. A typical C/C++ project doesn't tell you to build it via apt/rpm instead you are expected to provide the dependencies using whatever means you choose or they are included in the repo. This means you don't need to fight a package manager to get fully offline or reproducible builds for example.
RPM and APT packages are also usually not maintained by the upstream developer but by distro developers who care about making different packages work together so you don't get the dependency hell problem as a user.
RPM and APT packages are also usually not maintained by the upstream developer but by distro developers who care about making different packages work together so you don't get the dependency hell problem as a user.
Yes, C programs go to lengths to give the user choice. But it is recommended to use source packages for e.g. dpkg. And it's not too rare to have the .debian directory in the original VCS.
It is not. Most distro have their own build scripts that target the specific library name that is needed. Dependencies is mostly done through a mix of convention, helper programs/scripts and compiler args.
Most distros have their own package manager including source packages and an automated way to go from there to binaries. So the build scripts you mention are part of the package manager.
[deleted]
Rust’s big issue here is the anemic standard library. I think overall the strategy makes some amount of sense; since there’s so much crazy alchemy like depending on nightly, no_std, etc in Rust, including stuff in std has more downside in Rust than in a language that’s more stable like Go.
But it’s annoying to have to deal with 3 different time libraries and 3 different error creation libraries and 2 regex libraries somehow in my dependency tree. Plus many packages named stuff like “anyhow” or “nom” or other nonsense words where you need to google for a while to figure out what a package is supposed to do. Makes auditing more difficult than if your library is named structured-errors or parser-combinator.
I don’t like go programming language but I do like go tooling & go ecosystem. I wish there was a Rust with Go Principles. Swift is kinda in the right ballpark, packages are typically named stuff that makes sense and Swift is closer to Rust perf and Rust safety than Go perf and Go safety. But Swift is a tiny ecosystem outside of stuff that depends on the Apple proprietary universe, and the actual APIs in packages can be very magical/clever. ¯\_(ツ)_/¯
But it’s annoying to have to deal with 3 different time libraries and 3 different error creation libraries and 2 regex libraries somehow in my dependency tree. Plus many packages named stuff like “anyhow” or “nom” or other nonsense words where you need to google for a while to figure out what a package is supposed to do. Makes auditing more difficult than if your library is named structured-errors or parser-combinator.
I don’t like go programming language but I do like go tooling & go ecosystem. I wish there was a Rust with Go Principles. Swift is kinda in the right ballpark, packages are typically named stuff that makes sense and Swift is closer to Rust perf and Rust safety than Go perf and Go safety. But Swift is a tiny ecosystem outside of stuff that depends on the Apple proprietary universe, and the actual APIs in packages can be very magical/clever. ¯\_(ツ)_/¯
The very sparse std is one of the few genuine mistakes I think Rust has made. I know the arguments for it, but I don't find them persuasive. A batteries included standard library, in my view, is just plain better and every modern language should have one.
Oh man I feel the opposite. Rust is an example where a sparse stdlib is clearly superior and better and more
successful.
I mostly write C++ whose committee is incompetent and sniffs glue. And I deal a lot with Khronos committed who design pure garbage. “Design by Committee” is a pejorative for a reason.
I mostly write C++ whose committee is incompetent and sniffs glue. And I deal a lot with Khronos committed who design pure garbage. “Design by Committee” is a pejorative for a reason.
But what batteries to include? I'm sure your list of batteries is probably very different from mine, reflecting the difference in the work we both do. A lot of times when I hear a language has "batteries included", it's a bunch of features for web devs and nothing I would consider a "battery" for my projects.
What do you consider your set of "Batteries Include"?
Basically the stuff in here: https://www.ros.org
I've been looking through the repository list and that is not "batteries included", that's "everything, the kitchen sink, and the city".
Unless I am reading this (https://index.ros.org/?search_repos=true) wrong.
Unless I am reading this (https://index.ros.org/?search_repos=true) wrong.
That's pretty much my point -- my batteries aren't your batteries. If you're building houses, kitchen sinks are your parts. I'm high up on the abstraction stack so I need SLAM, and Kalman filters, advanced logging, mesh pub/sub networking in my stdlib. Here's another example of one of my "batteries" that is perhaps a little more concise: https://www.mathworks.com/products/image-processing.html
Something like OpenCV in the language stdlib is my idea of "batteries included". Like I said: what you consider batteries are not what people in my line of work consider batteries; Odin advertises "batteries included" but looking through the list I wouldn't use any of that in my day to day.
That's why Rust has a small stdlib of just the essentials, because doing so keeps things general, and everyone gets to choose their own idea of batteries.
Something like OpenCV in the language stdlib is my idea of "batteries included". Like I said: what you consider batteries are not what people in my line of work consider batteries; Odin advertises "batteries included" but looking through the list I wouldn't use any of that in my day to day.
That's why Rust has a small stdlib of just the essentials, because doing so keeps things general, and everyone gets to choose their own idea of batteries.
I agree, though also I note Python has an extensive standard library and isn't much better in terms of package sprawl.
Yeah, Python is a cautionary tale here, and I think one that informed the Rust stance.
Python is much older than Go, and has had more packages move from 3rd party into the stdlib to become a "battery", and then atrophy over the years while people move back to 3rd party alternatives with more features that are actually receiving maintenance. Eventually some of those modules were removed from core.
Perhaps the Go model only works when you have a very dedicated core group (for Go, mostly Google employees) around to continuously build and maintain the Cathedral of the standard library + toolchain together. Golang feels very much like UNIX (eg FreeBSD) for this reason, and Rust/Python more like Linux.
Python is much older than Go, and has had more packages move from 3rd party into the stdlib to become a "battery", and then atrophy over the years while people move back to 3rd party alternatives with more features that are actually receiving maintenance. Eventually some of those modules were removed from core.
Perhaps the Go model only works when you have a very dedicated core group (for Go, mostly Google employees) around to continuously build and maintain the Cathedral of the standard library + toolchain together. Golang feels very much like UNIX (eg FreeBSD) for this reason, and Rust/Python more like Linux.
Just as the Rust community has largely converged on tokio as the standard async runtime, is there any reason why there couldn't exist a community-developed "batteries-included" standard library other than writing a standard library being a tedious and thankless task?
> How do I manage my code without a “package manager”? [...] Through manual dependency management.
Slackware Linux does precisely that.
I'm a Slackware user. Slackware does have a package manager that can install or remove packages, and even a frontend that can use repositories (slackpkg), but it does have manual dependency resolution. Sure, there are 3rd-party managers that can add dependency resolution, but they do not come with the distro as default.
This is a very personal opinion, but manual dependency management is a feature. Back in the day, I remember installing Mandrake Linux 9.2 and activating the (then new-ish) framebuffer console. The distro folks had no better idea than to force a background "9.2" image on framebuffer consoles, which I hated. I finally found the package responsible for that. Removing it with urpmi, however, meant removing all the graphical desktop components (including X11) because that stupid package was listed as a dependency of everything graphical.
That prompted me to seek alternatives to Mandrake and ended up using Slackware. Its simplicity had the added bonus of offering manual dependency resolution.
Slackware Linux does precisely that.
I'm a Slackware user. Slackware does have a package manager that can install or remove packages, and even a frontend that can use repositories (slackpkg), but it does have manual dependency resolution. Sure, there are 3rd-party managers that can add dependency resolution, but they do not come with the distro as default.
This is a very personal opinion, but manual dependency management is a feature. Back in the day, I remember installing Mandrake Linux 9.2 and activating the (then new-ish) framebuffer console. The distro folks had no better idea than to force a background "9.2" image on framebuffer consoles, which I hated. I finally found the package responsible for that. Removing it with urpmi, however, meant removing all the graphical desktop components (including X11) because that stupid package was listed as a dependency of everything graphical.
That prompted me to seek alternatives to Mandrake and ended up using Slackware. Its simplicity had the added bonus of offering manual dependency resolution.
Sounds like "alias dpkg=dpkg --force-depends"?
Perhaps; I'm not really knowledgeable on the ways of Debian.
> Dependency hell [0] is a real thing which anyone who has worked on a large project has experienced. Projects having thousands, if not tens of thousands, of dependencies where you don’t know if they work properly, where are the bugs, you don’t how anything is being handled—it’s awful.
[0] https://en.wikipedia.org/wiki/Dependency_hell
I find it strange that they use a term with a common meaning, link to that meaning, and then talk about something else?
[0] https://en.wikipedia.org/wiki/Dependency_hell
I find it strange that they use a term with a common meaning, link to that meaning, and then talk about something else?
Same. The first thing I thought was "wait a second, that isn't dependency hell".
The second thing is that their version of dependency hell - having lots of dependencies introducing lots of bugs that you would not have written - is not my experience. 99% of the time, my bugs are in my own code, lol. Maybe once you become a much better programmer than me, you stop writing bugs in your own code and instead start having to deal with bugs in the PNG parsing library you depend on or something, and at that point writing your own PNG parsing library becomes a good use of your time. But I'm certainly not at that point.
I've had to fix bugs in dependencies of course. Here is one I fixed yesterday [0]. But it's much closer to the exception than the rule.
[0]: https://github.com/sanity/pav.rs/pull/4
The second thing is that their version of dependency hell - having lots of dependencies introducing lots of bugs that you would not have written - is not my experience. 99% of the time, my bugs are in my own code, lol. Maybe once you become a much better programmer than me, you stop writing bugs in your own code and instead start having to deal with bugs in the PNG parsing library you depend on or something, and at that point writing your own PNG parsing library becomes a good use of your time. But I'm certainly not at that point.
I've had to fix bugs in dependencies of course. Here is one I fixed yesterday [0]. But it's much closer to the exception than the rule.
[0]: https://github.com/sanity/pav.rs/pull/4
Where?
I don't know what the solution to this problem is, but I do remember a time (around 20 years ago) when this wasn't a big problem. Was working on a fairly large (each module between 50k - 100k LOC) C++ system. The process for using libraries:
1) Have problem that feels too complicated to hand-code.
2) Go on Internet/forums, find a library. The library is usually a small, flat collection of atomic functions.
3) A senior engineer vets the library and approves it for use.
4) Download the stable version: header file, and the lib file for our platform (on rare occasions, build it from source)
5) Place the .h file in the header path, and the lib file in the lib path; update the Makefile.
6) #include the header and call functions.
7) Update deployment scripts (bash script) to scp the lib file to target environment, or in some cases, use static linking.
8) Subscribe to a mailing list and very occasionally receive news of a breaking change that requires a rebuild.
This may sound like a lot of work, but somehow, it was a lot less stressful than dealing with NPM and node_modules today.
1) Have problem that feels too complicated to hand-code.
2) Go on Internet/forums, find a library. The library is usually a small, flat collection of atomic functions.
3) A senior engineer vets the library and approves it for use.
4) Download the stable version: header file, and the lib file for our platform (on rare occasions, build it from source)
5) Place the .h file in the header path, and the lib file in the lib path; update the Makefile.
6) #include the header and call functions.
7) Update deployment scripts (bash script) to scp the lib file to target environment, or in some cases, use static linking.
8) Subscribe to a mailing list and very occasionally receive news of a breaking change that requires a rebuild.
This may sound like a lot of work, but somehow, it was a lot less stressful than dealing with NPM and node_modules today.
I think the main thing that makes this workable is "The library is usually a small, flat collection of atomic functions."
I find that it's the hell of transitive dependencies--you as a developer can reasonably vet a single layer of 10-30 standalone libraries. But if those libraries depend on other libraries, etc, then it balloons into hundreds or thousands of dependencies, and then you're sunk.
For what it's worth, I don't think much of this is essential complexity. Often a library is complicated because it supports 10 different ways of using it, but when you use the library, you're only using 1 of those ways. If everyone is only using 10% of thousands of transitive dependencies, the overall effect is incredibly complicated, but could have been achieved with 10-100% more short-term effort. Sure, "it took twice as long to develop but at least we don't have 10x the dependencies" is a hard sell to management (and often to ourselves), but that's because we usually choose to ignore the costs of depending on software we don't understand and don't control. We think that we're cleverly avoiding having to maintain and secure those libraries we outsourced, but most open-source developers aren't doing a great job of that anyway.
Often it really is easier to develop something from scratch, rather than learn and integrate a library. Not always though, of course.
I find that it's the hell of transitive dependencies--you as a developer can reasonably vet a single layer of 10-30 standalone libraries. But if those libraries depend on other libraries, etc, then it balloons into hundreds or thousands of dependencies, and then you're sunk.
For what it's worth, I don't think much of this is essential complexity. Often a library is complicated because it supports 10 different ways of using it, but when you use the library, you're only using 1 of those ways. If everyone is only using 10% of thousands of transitive dependencies, the overall effect is incredibly complicated, but could have been achieved with 10-100% more short-term effort. Sure, "it took twice as long to develop but at least we don't have 10x the dependencies" is a hard sell to management (and often to ourselves), but that's because we usually choose to ignore the costs of depending on software we don't understand and don't control. We think that we're cleverly avoiding having to maintain and secure those libraries we outsourced, but most open-source developers aren't doing a great job of that anyway.
Often it really is easier to develop something from scratch, rather than learn and integrate a library. Not always though, of course.
In C and C++ you don't need the transitive dependencies for compilation, you only need the header of the direct dependencies. As for linking they are only needed when linking dynamically, which was much less prevalent 20 years ago.
It's not about compilation, it's about interactions, and leaky abstractions.
This then means that the problem is more the quality of the library itself and not the package manager/dependency resolver/build system. You can have leaky abstractions just fine when all you do is add a single binary static library with nothing else going on.
This reads much more like a critique of traditional open-source development than package managers themselves.
The author asserts that most open-source projects don't hit the quality standards so that their libraries can be just included, and they'll do what they say.
I assert that this is because there's no serious product effort behind most libraries (as in no dedicated QA/test/release cycle), no large commercial products use it (or if they do, either they do it in a very limited fashion, or just fork it).
Hobbyists do QA as long as it interests them/fits their usecase, but only the big vendors do bulletproof releases (which in the desktop realm seems to be only MS/Apple)
This might have to do with the domain the author chose - desktop development has unfortunately had the life sucked out of it with every dev either being a fullstack/cloud/ML/mobile dev, its mindshare and the resources going toward it have plummeted.
(I also have a sneaking suspicion the author might've encountered those bugs on desktop Linux, which, despite all the cheerleading (and policing negative opinions), is as much as a buggy mess as ever.
In my experience, it's quite likely to run into a bug that nobody has written about on the internet ever.
The author asserts that most open-source projects don't hit the quality standards so that their libraries can be just included, and they'll do what they say.
I assert that this is because there's no serious product effort behind most libraries (as in no dedicated QA/test/release cycle), no large commercial products use it (or if they do, either they do it in a very limited fashion, or just fork it).
Hobbyists do QA as long as it interests them/fits their usecase, but only the big vendors do bulletproof releases (which in the desktop realm seems to be only MS/Apple)
This might have to do with the domain the author chose - desktop development has unfortunately had the life sucked out of it with every dev either being a fullstack/cloud/ML/mobile dev, its mindshare and the resources going toward it have plummeted.
(I also have a sneaking suspicion the author might've encountered those bugs on desktop Linux, which, despite all the cheerleading (and policing negative opinions), is as much as a buggy mess as ever.
In my experience, it's quite likely to run into a bug that nobody has written about on the internet ever.
This critique applies to even closed-source development that uses open-source code bases.
I have an article on my unstructured thoughts on the problems of OSS/FOSS which goes into more depth about this: https://www.gingerbill.org/article/2025/04/22/unstructured-t...
I have an article on my unstructured thoughts on the problems of OSS/FOSS which goes into more depth about this: https://www.gingerbill.org/article/2025/04/22/unstructured-t...
This is why I'm so glad that I work in a closed monorepo now. There is no package management, only build tooling.
I find myself nodding along to many of the technical and organizational arguments. But I get lost in the licensing discussion.
If it is a cultural problem that people insist on giving things away for free (and receiving them for free), then viral licenses can be very helpful, not fundamentally pernicious.
Outside of the megaprojects, my mental model for GPL is similar to proprietary enterprise software with free individual licenses. The developer gets the benefits of open projects: eyeballs, contributors, adoption, reputational/professional benefits, doing a good deed (if that motivates them) while avoiding permissively giving everything away. The idea that it's problematic that you can't build a business model on their software is akin to the "forced charity" mindset—"why did you make something that I can't use for free?"
If you see a GPL'd bit of code that you really want to use in your business, email the developers with an offer of $X,000 for a perpetual commercial license and a $Y,000/yr support contract. Most are not so ideologically pure to refuse. It's a win-win-win: your business gets the software, the developers don't feel exploited, noncommercial downstream users can enjoy the fruits of open software, and everybody's contributed to a healthier attitude on open source.
I find myself nodding along to many of the technical and organizational arguments. But I get lost in the licensing discussion.
If it is a cultural problem that people insist on giving things away for free (and receiving them for free), then viral licenses can be very helpful, not fundamentally pernicious.
Outside of the megaprojects, my mental model for GPL is similar to proprietary enterprise software with free individual licenses. The developer gets the benefits of open projects: eyeballs, contributors, adoption, reputational/professional benefits, doing a good deed (if that motivates them) while avoiding permissively giving everything away. The idea that it's problematic that you can't build a business model on their software is akin to the "forced charity" mindset—"why did you make something that I can't use for free?"
If you see a GPL'd bit of code that you really want to use in your business, email the developers with an offer of $X,000 for a perpetual commercial license and a $Y,000/yr support contract. Most are not so ideologically pure to refuse. It's a win-win-win: your business gets the software, the developers don't feel exploited, noncommercial downstream users can enjoy the fruits of open software, and everybody's contributed to a healthier attitude on open source.
> "This is the automation of dependency hell. The problem is that not everything needs to be automated, especially hell. Dependency hell is a real thing which anyone who has worked on a large project has experienced. Projects having thousands, if not tens of thousands, of dependencies where you don’t know if they work properly, where are the bugs, you don’t how anything is being handled—it’s awful.
This the wrong thing to automate. You can do this manually, however it doesn’t stop you getting into hell, rather just slow you down, as you can put yourself into hell (in fact everyone puts themselves into hell voluntarily). The point is it makes you think how you get there, so if you have to download manually, you will start thinking “maybe I don’t want this” or “maybe I can do this instead”. And when you need to update packages, being manual forces you to be very careful."
I sympathise with this, but I have to respond that we have to live within existing ecosystems. Getting rid of npm and doing things manually won't make building SPAs have fewer dependencies, build would be incredibly slow and painful.
This the wrong thing to automate. You can do this manually, however it doesn’t stop you getting into hell, rather just slow you down, as you can put yourself into hell (in fact everyone puts themselves into hell voluntarily). The point is it makes you think how you get there, so if you have to download manually, you will start thinking “maybe I don’t want this” or “maybe I can do this instead”. And when you need to update packages, being manual forces you to be very careful."
I sympathise with this, but I have to respond that we have to live within existing ecosystems. Getting rid of npm and doing things manually won't make building SPAs have fewer dependencies, build would be incredibly slow and painful.
Packages themselves are not bad. NPM is just fine - so long as you don't let it do dependency resolution and lock the version of every package. Note that this means you have to get notified when each package is updated (how!) and make a decision on how to update it (or if you decide not to update make a decision to maintain it).
The other thing is your package manager cannot go out to the internet randomly. You need it to download from a place you are comfortable with (which might or might not be the default) existing as long as you need packages, and that will keep the versions of packages you want around. If you are a company project that means an internal server/mirror because otherwise something you depend on will disappear in the future. (most of they decide nobody is using it, delete it, but sometimes it is discovered the thing is an illegal copyright violation - but you have ask your lawyers what this means for you - perhaps a license is easy to get)
The other thing is your package manager cannot go out to the internet randomly. You need it to download from a place you are comfortable with (which might or might not be the default) existing as long as you need packages, and that will keep the versions of packages you want around. If you are a company project that means an internal server/mirror because otherwise something you depend on will disappear in the future. (most of they decide nobody is using it, delete it, but sometimes it is discovered the thing is an illegal copyright violation - but you have ask your lawyers what this means for you - perhaps a license is easy to get)
> Getting rid of npm and doing things manually won't make building SPAs have fewer dependencies, build would be incredibly slow and painful.
Honestly, I don't think this is true in the slightest. Rather, I hypothesize that people want to use such tooling and think the alternatives are slower, which I don't think is true.
If people actually did use fewer dependencies, people would have actually have websites that didn't take ages to load and were responsive.
So the existing ecosystems are just bad.
Honestly, I don't think this is true in the slightest. Rather, I hypothesize that people want to use such tooling and think the alternatives are slower, which I don't think is true.
If people actually did use fewer dependencies, people would have actually have websites that didn't take ages to load and were responsive.
So the existing ecosystems are just bad.
Some years ago, I had to reproduce a neural model build that had only been done previously by a single colleague on her laptop, not using a package manager.
Part of my reproducing the build was to conduct all the library downloading in a miniconda environment, so at the end I had a reproducible recipe.
Is the original author seriously claiming that anybody was better off with the original, "pure" ad-hoc approach?
Part of my reproducing the build was to conduct all the library downloading in a miniconda environment, so at the end I had a reproducible recipe.
Is the original author seriously claiming that anybody was better off with the original, "pure" ad-hoc approach?
> Getting rid of npm and doing things manually won't make building SPAs have fewer dependencies, build would be incredibly slow and painful.
You don't think making adding dependencies incredibly slow and painful would make people have fewer of them?
You don't think making adding dependencies incredibly slow and painful would make people have fewer of them?
In the context of my team, us getting rid of npm wouldn't change the whole SPA ecosystem. Or the various requirements we have that effectively mandate SPA like applications.
But in the context of newer ecosystems or ones that are more tightly controlled things might be different. For example if apple massively expanded the swift standard library and made dependency management painful, iOS apps might end up having fewer dependencies.
But in the context of newer ecosystems or ones that are more tightly controlled things might be different. For example if apple massively expanded the swift standard library and made dependency management painful, iOS apps might end up having fewer dependencies.
You would decrease number of dependencies yes. However your dependencies or your code would then become huge.
Same number of lines but in fewer dependencies.
Same number of lines but in fewer dependencies.
Yes, because of human limits of time and of skills.
I remember installing software in the early 90s: download the source code, read the README, find and download the dependencies, read their READMEs, repeat a few times. Sometimes one dependency could not compile because of any incompatibility or bug. Some could be fixed, some couldn't. Often everything ended up with a successful compilation and install and in one day of work I could have what I'm getting in a few minutes now.
Actually those were small programs by today standards. My take is that we would achieve less if we have to use less dependencies.
By the way, the last time I compiled something from source was yesterday. It was openvpn3 on Debian 13, which is still unsupported. TLDR, it works but the apt-get are a little different from the ones in BUILD.md
I remember installing software in the early 90s: download the source code, read the README, find and download the dependencies, read their READMEs, repeat a few times. Sometimes one dependency could not compile because of any incompatibility or bug. Some could be fixed, some couldn't. Often everything ended up with a successful compilation and install and in one day of work I could have what I'm getting in a few minutes now.
Actually those were small programs by today standards. My take is that we would achieve less if we have to use less dependencies.
By the way, the last time I compiled something from source was yesterday. It was openvpn3 on Debian 13, which is still unsupported. TLDR, it works but the apt-get are a little different from the ones in BUILD.md
There already is a (partial) solution to dependency hell: Nix.
It will at least massively help prevent things from breaking unexpectedly.
It won't prevent you from having to cascade a necessary upgrade (such as a security fix) across the entire project until resolution/new equilibrium is achieved.
My solution to the latter is simply to try to depend on as few things as possible. But eventually, the cancer will overtake the project if it keeps growing.
Source: Have worked on a million-LOC Ruby app
It will at least massively help prevent things from breaking unexpectedly.
It won't prevent you from having to cascade a necessary upgrade (such as a security fix) across the entire project until resolution/new equilibrium is achieved.
My solution to the latter is simply to try to depend on as few things as possible. But eventually, the cancer will overtake the project if it keeps growing.
Source: Have worked on a million-LOC Ruby app
Nix isn't a solution to the problem of package managers. It just a better way to package management system, which thus makes it easier to go to dependency hell. So I'd argue it puts fuel on the flames.
The solution is just to depend on less and manage them manually.
The solution is just to depend on less and manage them manually.
If we depend on less then we are capping what we can build to only what we can write and manually integrate. This is dis-empowering.
I'd argue quite the opposite. You can build a lot more than you think, you just need to be encouraged.
if you depend on more, you are capping what you can build to only what you can manage to get working and integrate, plus you also lose control over the code; for me personally, it's a lot easier to write a complicated algorithm, than it is to figure out a complex build system for a random library in the wild (with few exceptions like miniaudio); and it's especially worth it, because when i understand the problem, i can usually define a way better api for my usecase than any library writer can
I don't think that is a solution. In fact I don't think there are any "solutions". I do think there are tradeoffs though.
There's a fair bit of semantic quibbling here.
Regardless of how they define these terms, producing a list of hashes which function as a commitment to specific versions of dependencies is a technique essential to modern software development. Whatever the tools are called, and whatever they do, they need to spit out a list of hashes that can be checked into version control.
You could just use git submodules, but in practice there are better user experiences provided by language package managers (`go mod` works great).
A good amount of this ranting can probably be attributed to projects and communities that aren't even playing the list of hashes game. They are resolving or upgrading dependencies in CI or at runtime or something crazy like that.
Regardless of how they define these terms, producing a list of hashes which function as a commitment to specific versions of dependencies is a technique essential to modern software development. Whatever the tools are called, and whatever they do, they need to spit out a list of hashes that can be checked into version control.
You could just use git submodules, but in practice there are better user experiences provided by language package managers (`go mod` works great).
A good amount of this ranting can probably be attributed to projects and communities that aren't even playing the list of hashes game. They are resolving or upgrading dependencies in CI or at runtime or something crazy like that.
The semantic quibbling is just to explain what a package manager isn't.
Also, use git subtrees, not git submodules. What people think submodules are, are actually subtrees and most people don't know about them.
As for "good" package managers, they are still bad because of what I said in the article.
Also, use git subtrees, not git submodules. What people think submodules are, are actually subtrees and most people don't know about them.
As for "good" package managers, they are still bad because of what I said in the article.
The argument here is (in brief) "Package management is hell, package managers are evil. So let's handle the hell manually to feel the pain better".
And honestly speaking: It is plain stupid.
We can all agree that abusing package management with ~10000 of micro packages everywhere like npm/python/ruby does is completely unproductive and brings its own considerable maintenance burden and complexity.
But ignoring the dependency resolution problem entirely by saying "You do not need dependencies" is even dumber.
Not every person is working in an environment where shipping a giant blob executable built out of vendored static dependencies is even possible. This is a privilege of the Gamedev industry has and the author forgets a bit too easily it is domain specific.
Some of us works in environment where the final product is an agglomerate of >100 of components developed by >20 teams around the world. Versioned over ~50 git repositories. Often mixed with some proprietary libraries provided by third-party providers. Gluing, assembling and testing all of that is far beyond the "LOL, just stick to the SDL" mindset proposed here.
Some of us are developing libraries/frameworks that are used embedded in >50 products with other libraries with a hell of multiples combinations of compilers / ABI / platforms. This is not something you want to test nor support without automation.
Some of us have to maintain cathedrals that are constructed over decades of domain specific knowhow (Scientific simulators, solvers, Petrol prospection tools, financial frameworks, ... ) in multiple languages (Fortran, C, C++, Python, Lua, ...) that can not just be re-written in few weeks because "I tell you: dependencies sucks, Bro"
Managing all of that manually is just insane. And generally finishes with an home-made half-baked bunch of scripts that try to badly mimic the behavior of a proper package manager.
So no, there is no replacement for a proper package manager: Instead of hating the tool, just learn to use it.
Package manager are tools, and like every tool, they should be used Wisely and not as a Maslow's Hammer.
And honestly speaking: It is plain stupid.
We can all agree that abusing package management with ~10000 of micro packages everywhere like npm/python/ruby does is completely unproductive and brings its own considerable maintenance burden and complexity.
But ignoring the dependency resolution problem entirely by saying "You do not need dependencies" is even dumber.
Not every person is working in an environment where shipping a giant blob executable built out of vendored static dependencies is even possible. This is a privilege of the Gamedev industry has and the author forgets a bit too easily it is domain specific.
Some of us works in environment where the final product is an agglomerate of >100 of components developed by >20 teams around the world. Versioned over ~50 git repositories. Often mixed with some proprietary libraries provided by third-party providers. Gluing, assembling and testing all of that is far beyond the "LOL, just stick to the SDL" mindset proposed here.
Some of us are developing libraries/frameworks that are used embedded in >50 products with other libraries with a hell of multiples combinations of compilers / ABI / platforms. This is not something you want to test nor support without automation.
Some of us have to maintain cathedrals that are constructed over decades of domain specific knowhow (Scientific simulators, solvers, Petrol prospection tools, financial frameworks, ... ) in multiple languages (Fortran, C, C++, Python, Lua, ...) that can not just be re-written in few weeks because "I tell you: dependencies sucks, Bro"
Managing all of that manually is just insane. And generally finishes with an home-made half-baked bunch of scripts that try to badly mimic the behavior of a proper package manager.
So no, there is no replacement for a proper package manager: Instead of hating the tool, just learn to use it.
Package manager are tools, and like every tool, they should be used Wisely and not as a Maslow's Hammer.
I am not sure how you got this conclusion from the article.
> So let's handle the hell manually to feel the pain better
This is far from my position. Literally the entire point is to make it clearer you are heading to dependency hell, rather than feel the pain better whilst you are there.
I am not against dependencies but you should know the costs of them and the alternatives. Package managers hide the complexity, costs, trade-offs, and alternative approaches, thus making it easier to slip into dependency hell.
> So let's handle the hell manually to feel the pain better
This is far from my position. Literally the entire point is to make it clearer you are heading to dependency hell, rather than feel the pain better whilst you are there.
I am not against dependencies but you should know the costs of them and the alternatives. Package managers hide the complexity, costs, trade-offs, and alternative approaches, thus making it easier to slip into dependency hell.
> I am not against dependencies but you should know the costs of them and the alternatives.
You are against the usage of a tool and you propose no alternative.
Handling the dependency by vendoring them manually, like you propose in your blog, is not an alternative.
This is an over simplification of the problem (and the problem is complex) that can be applied only to your specific usage and domain.
You are against the usage of a tool and you propose no alternative.
Handling the dependency by vendoring them manually, like you propose in your blog, is not an alternative.
This is an over simplification of the problem (and the problem is complex) that can be applied only to your specific usage and domain.
It is an alternative, just clearly not one you like. And it's not an oversimplification of the problem.
Again, what is wrong with saying you should know the costs of the dependencies you include AND the alternative approaches of not using the dependencies?—e.g. using the standard library, writing it yourself, using another dependency already that might fit, etc.
Again, what is wrong with saying you should know the costs of the dependencies you include AND the alternative approaches of not using the dependencies?—e.g. using the standard library, writing it yourself, using another dependency already that might fit, etc.
I mostly agree, but
> Some of us works in environment where the final product is an agglomerate of >100 of components developed by >20 teams around the world. Versioned over ~50 git repositories. Often mixed with some proprietary libraries provided by third-party providers. Gluing, assembling and testing all of that is far beyond the "LOL, just stick to the SDL" mindset proposed here.
Does this somehow prevent you from vendoring everything?
> Some of us works in environment where the final product is an agglomerate of >100 of components developed by >20 teams around the world. Versioned over ~50 git repositories. Often mixed with some proprietary libraries provided by third-party providers. Gluing, assembling and testing all of that is far beyond the "LOL, just stick to the SDL" mindset proposed here.
Does this somehow prevent you from vendoring everything?
> Does this somehow prevent you from vendoring everything?
Yes. Because in these environment soon or later you will be shipping libraries and not executable.
Shipping libraries means that your software will need to be integrated in other stacks where you do not control the full dependency tree nor the versions there.
Vendoring dependencies in this situation is the guarantee that you will make the life of your customer miserable by throwing the diamond dependency problem right in their face.
Yes. Because in these environment soon or later you will be shipping libraries and not executable.
Shipping libraries means that your software will need to be integrated in other stacks where you do not control the full dependency tree nor the versions there.
Vendoring dependencies in this situation is the guarantee that you will make the life of your customer miserable by throwing the diamond dependency problem right in their face.
You're making your customer's life miserable by having dependencies. You're a library, your customer is using you to solve a specific problem. Write the code to solve that and be done with it.
In the game development sphere, there's plenty of giant middleware packages for audio playback, physics engines, renderers, and other problems that are 1000x more complex and more useful than any given npm package, and yet I somehow don't have to "manage a dependency tree" and "resolve peer dependency conflicts" when using them.
In the game development sphere, there's plenty of giant middleware packages for audio playback, physics engines, renderers, and other problems that are 1000x more complex and more useful than any given npm package, and yet I somehow don't have to "manage a dependency tree" and "resolve peer dependency conflicts" when using them.
When you're a library, your customer is another developer. By vendoring needlessly, you potentially cause unavoidable bloat in someone else's product. If you interoperate with standard interfaces, your downstream should be able to choose what's on the other end of that interface.
> You're making your customer's life miserable by having dependencies. You're a library, your customer is using you to solve a specific problem. Write the code to solve that and be done with it.
And you just don't know what you are talking about.
If I am providing (lets say) a library that provides some high level features for a car ADAS system on top of a CAN network with a proprietary library as driver and interface.
This is not up to me to fix or choose the library and the driver version that the customer will use. He will choose the certified version he will ship, he will test my software on it and integrate it.
Vendoring dependency for anything which is not a final product (product as executable) is plain stupid.
It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.
If you want to vendor, do vendor, but stick to executables with well-defined IPC systems.
And you just don't know what you are talking about.
If I am providing (lets say) a library that provides some high level features for a car ADAS system on top of a CAN network with a proprietary library as driver and interface.
This is not up to me to fix or choose the library and the driver version that the customer will use. He will choose the certified version he will ship, he will test my software on it and integrate it.
Vendoring dependency for anything which is not a final product (product as executable) is plain stupid.
It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.
If you want to vendor, do vendor, but stick to executables with well-defined IPC systems.
> If I am providing (lets say) a library that provides some high level features for a car ADAS system on top of a CAN network with a proprietary library as driver and interface.
If you're writing an ADAS system, and you have a "dependency tree" that needs to be "resolved" by a package manager, you should be fired immediately.
Any software that has lives riding on it, if it has dependencies, must be certified against a specific version of them, that should 100% of the time, without exceptions, must be vendored with the software.
> It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.
The exact opposite. Vendoring is the ONLY way to prevent the ABI madness of "v1.3.1 of libfoo exports libfoo_a but not libfoo_b, and v1.3.2 exports libfoo_b but not libfoo_c, and in 1.3.2 libfoo_b takes in a pointer to a struct that has a different layout."
If you MUST have libfoo (which you don't), you link your version of libfoo into your blob and you never expose any libfoo symbols in your library's blob.
If you're writing an ADAS system, and you have a "dependency tree" that needs to be "resolved" by a package manager, you should be fired immediately.
Any software that has lives riding on it, if it has dependencies, must be certified against a specific version of them, that should 100% of the time, without exceptions, must be vendored with the software.
> It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.
The exact opposite. Vendoring is the ONLY way to prevent the ABI madness of "v1.3.1 of libfoo exports libfoo_a but not libfoo_b, and v1.3.2 exports libfoo_b but not libfoo_c, and in 1.3.2 libfoo_b takes in a pointer to a struct that has a different layout."
If you MUST have libfoo (which you don't), you link your version of libfoo into your blob and you never expose any libfoo symbols in your library's blob.
You keep confirming that you don't know what you are talking about.
The vendoring step happens at something like Yocto or equivalent and that's what ends up being certified, not random library repos.
The vendoring step happens at something like Yocto or equivalent and that's what ends up being certified, not random library repos.
Yes exactly.
And in addition: Yocto (or equivalent) will also be the one providing you the traceability required to guarantee that what you ship is currently what you certified and not some random garbage compiled in a laptop user directory.
And in addition: Yocto (or equivalent) will also be the one providing you the traceability required to guarantee that what you ship is currently what you certified and not some random garbage compiled in a laptop user directory.
Did Yocto ever clean up how they manage the sysroot?
It used to have a really bad design flaw. Example: - building package X explicitly depends on A to be in the sysroot - building package Y explicitly depends on B in the sysroot, but implicitly will use A if present (thanks autoconf!)
In such a situation, building X before Y will result in Y effectively using A&B — perhaps enabling unintended features. Building Y then X would produce a different Y.
Coupled with the parallel build environment, it’s a recipe for highly non deterministic binaries — without even considering reproducibility.
It used to have a really bad design flaw. Example: - building package X explicitly depends on A to be in the sysroot - building package Y explicitly depends on B in the sysroot, but implicitly will use A if present (thanks autoconf!)
In such a situation, building X before Y will result in Y effectively using A&B — perhaps enabling unintended features. Building Y then X would produce a different Y.
Coupled with the parallel build environment, it’s a recipe for highly non deterministic binaries — without even considering reproducibility.
> Did Yocto ever clean up how they manage the sysroot?
It's better than before but you still need to sandbox manually if you want good reproducibility.
Honestly, for reproducibility alone. There is better than Yocto nowadays. It is hard to beat Nix at this game. Even Bazel based build flows are somewhat better.
But in the embedded world, Yocto is pretty widespread and almost the de-facto norm for Linux embedded.
It's better than before but you still need to sandbox manually if you want good reproducibility.
Honestly, for reproducibility alone. There is better than Yocto nowadays. It is hard to beat Nix at this game. Even Bazel based build flows are somewhat better.
But in the embedded world, Yocto is pretty widespread and almost the de-facto norm for Linux embedded.
> but implicitly will use A if present (thanks autoconf!)
When you want reproducibility, you need to specify what you want, not let the computer guess. Why can't you use Y/configure --without-A ? In the extreme case you can also version config.status.
When you want reproducibility, you need to specify what you want, not let the computer guess. Why can't you use Y/configure --without-A ? In the extreme case you can also version config.status.
One certainly can, but such is not default.
Things using autotools evolved to be “manual user friendly” in the sense that application features are automatically enabled based on auto detected libraries.
But for automated builds, all those smarts get in the way when the build environment is subject to variation.
In theory, the Yocto recipe will fully specify the application configuration regardless of how the environment varies…
Of course, in theory the most Byzantine build process will always function correctly too!
Things using autotools evolved to be “manual user friendly” in the sense that application features are automatically enabled based on auto detected libraries.
But for automated builds, all those smarts get in the way when the build environment is subject to variation.
In theory, the Yocto recipe will fully specify the application configuration regardless of how the environment varies…
Of course, in theory the most Byzantine build process will always function correctly too!
"Vendoring step" You cannot make this shit up.
You're providing a library. That library has dependencies (although it shouldn't). You've written that library to work against a specific version of those dependencies. Vendoring these dependencies means shipping them with your library, and not relying on your user, or even worse, their package manager to provide said dependencies.
I don't know what industry you work in, who the regulatory body that certifies your code is, or what their procedures are, but if they're not certifying the "random library repos" that are part of your code, I pray I never have to interact with your code.
You're providing a library. That library has dependencies (although it shouldn't). You've written that library to work against a specific version of those dependencies. Vendoring these dependencies means shipping them with your library, and not relying on your user, or even worse, their package manager to provide said dependencies.
I don't know what industry you work in, who the regulatory body that certifies your code is, or what their procedures are, but if they're not certifying the "random library repos" that are part of your code, I pray I never have to interact with your code.
> I don't know what industry you work in
I dabbled my fingers in enough of them to tame my hubris a bit and learn that various fields have specific needs that end up represented in their processes (and this includes gamedev as well). Highly recommended before commenting any further.
I dabbled my fingers in enough of them to tame my hubris a bit and learn that various fields have specific needs that end up represented in their processes (and this includes gamedev as well). Highly recommended before commenting any further.
> I don't know what industry you work in, who the regulatory body that certifies your code is, or what their procedures are, [..], I pray I never have to interact with your code.
You illustrate perfectly the attitude problem of the average "gamedev" here.
You do not know shit about the realities and the development practice of an entire domain (here the safety critical domain).
But still you brag confidently about how 'My dev practices are better' and affirm without any shame that everybody else in this field that disagree is an idiot.
Just to let you know: In the safety critical field, the responsibility of the final certification is on the integrator. That is why we do not want intermediate dependency to randomly vendor and bundle crap we do not have control of.
Additionally, it is often that the entire dependency tree (including proprietary third party components like AUTOSAR) are shipped as source available and compiled / assemblied from sources during the integration.
Thats why the usage of package manager like Yocto (or equivalent) is widespread in the domain: It allows to precisely track and version what is used an how for analysis and traceability back to the requirements.
Additionally again, when the usage of binary dependencies is the only solution available (like for Neutrino QNX and its associated compilers). Any serious certification organism (like the TUV) will mandate to have the exact checksum of each certified binary that you use in your application and a process to track them back to the certification document.
This is not something you do by dumping random fu**ng blob in a git repository like you are proposing. You generally do that, again, by using a proper set of processes and generally a package manager like Yocto or similar.
Finally, your comment on "v1.3.1 of libfoo" is completely moronic. You seem to have no idea of the consequence of duplicated symbols in multiples static libraries with vendored dependencies you do not control nor the consequences it can have on functional safety.
You illustrate perfectly the attitude problem of the average "gamedev" here.
You do not know shit about the realities and the development practice of an entire domain (here the safety critical domain).
But still you brag confidently about how 'My dev practices are better' and affirm without any shame that everybody else in this field that disagree is an idiot.
Just to let you know: In the safety critical field, the responsibility of the final certification is on the integrator. That is why we do not want intermediate dependency to randomly vendor and bundle crap we do not have control of.
Additionally, it is often that the entire dependency tree (including proprietary third party components like AUTOSAR) are shipped as source available and compiled / assemblied from sources during the integration.
Thats why the usage of package manager like Yocto (or equivalent) is widespread in the domain: It allows to precisely track and version what is used an how for analysis and traceability back to the requirements.
Additionally again, when the usage of binary dependencies is the only solution available (like for Neutrino QNX and its associated compilers). Any serious certification organism (like the TUV) will mandate to have the exact checksum of each certified binary that you use in your application and a process to track them back to the certification document.
This is not something you do by dumping random fu**ng blob in a git repository like you are proposing. You generally do that, again, by using a proper set of processes and generally a package manager like Yocto or similar.
Finally, your comment on "v1.3.1 of libfoo" is completely moronic. You seem to have no idea of the consequence of duplicated symbols in multiples static libraries with vendored dependencies you do not control nor the consequences it can have on functional safety.
It certainly gets in the way. The more dependencies, the more work it is to update them, especially when for some reason you're choosing _not_ to automate that process. And the larger the dependencies, the larger the repo.
Would you also try to build all of them on every CI run?
What about the non-source dependencies, check the binaries into git?
Would you also try to build all of them on every CI run?
What about the non-source dependencies, check the binaries into git?
This article is a bit all over the map with discussions of high trust societies and their relation to language design and software dependency management.
That said I think the final takeaway is that systems that allow you to pin versions, vendor all those dependencies and resolve/reproduce the same file tree regardless of who's machine it's on (let's assume matching architectures for simplicity here) is the goal.
Note that removing 'manually' here, this still works:
> Copying and vendoring each package {manually}, and fixing the specific versions down is the most practical approach to keeping a code-base stable, reliable, and maintainable.
The article's emphasis on the manual aspect of management of dependencies is a bit of loss, as I don't particularly believe it _has to be manual_ in the sense of manually copying files from their origin into your file tree; that certainly is a real world option, but few (myself included) would take that monk-like path again. I left this exact situation in C land and would not consider going back unless adopting something like ninja.
What the OP is actually describing is a "good" package manager feature set and many (sadly not most/all) do support this exact feature set today
PS I did chuckle when they defined evil in terms of something that gets you to dependency hell faster. However, we shouldn't be advocating for committing the same sins of our fathers.
That said I think the final takeaway is that systems that allow you to pin versions, vendor all those dependencies and resolve/reproduce the same file tree regardless of who's machine it's on (let's assume matching architectures for simplicity here) is the goal.
Note that removing 'manually' here, this still works:
> Copying and vendoring each package {manually}, and fixing the specific versions down is the most practical approach to keeping a code-base stable, reliable, and maintainable.
The article's emphasis on the manual aspect of management of dependencies is a bit of loss, as I don't particularly believe it _has to be manual_ in the sense of manually copying files from their origin into your file tree; that certainly is a real world option, but few (myself included) would take that monk-like path again. I left this exact situation in C land and would not consider going back unless adopting something like ninja.
What the OP is actually describing is a "good" package manager feature set and many (sadly not most/all) do support this exact feature set today
PS I did chuckle when they defined evil in terms of something that gets you to dependency hell faster. However, we shouldn't be advocating for committing the same sins of our fathers.
As I age, I do everything I can to avoid "one more dependency". There's a perverse nerd bragging effect that takes place where people equate their value as a programmer to how many dependencies they can name drop and mash up into their solution. It makes sense, since we evaluate each other, and the more dependencies you can reference in your workpast, the more stepping stones you've stepped on and therefor, been longer on the path.
Anymore, as I evaluate fellow programmers, I'm looking for whether they've discovered "one more dependency" is like signing up for "one more subscription you have to remember to pay for" and what they do to try and mitigate it.
Anymore, as I evaluate fellow programmers, I'm looking for whether they've discovered "one more dependency" is like signing up for "one more subscription you have to remember to pay for" and what they do to try and mitigate it.
I definitely used to look for reasons to include cool new dependencies that I found, just to try and do something with cool libraries.
But as I got bit by the various issues with dependencies multiple times over the years, I have ended up preferring as few as possible and ideally zero beyond the standard library for hobby projects if I can get away with it.
But as I got bit by the various issues with dependencies multiple times over the years, I have ended up preferring as few as possible and ideally zero beyond the standard library for hobby projects if I can get away with it.
Missed one of the biggest problems: Most assume the world is simpler than it really is. Not all the world is Rust, Python, Go, C++, or whatever language you advocate. I have millions of lines of C++, I'm interested in other languages, but they needs to interoperate with our C++. When we have a C++ library that does what you want I don't want you adding a new package to do the same thing as now we need to support both - sometimes will accept that using the package is the right answer, but often we need bug compatibility.
Not sure why this argument doesn't also apply to operating systems. Maybe everyone should be writing all their programs to run on a custom micro-kernel. Surely we can't trust other programmers to write something as complicated as an operating system.
There is the question. See "Reflections on Trusting trust" (a classic paper). However in the end you cannot do everything you might want to and so you must trust someone else. Operating systems are common, audited by many, and used by enough that you can have high trust they work in general (but there are some not worthy of trust). Package managers tend to contain many packages that are not in common use and if the only one who audit them might be you so you better do it yourself for each release.
If you only use a package manager for libraries that you have high trust in then you don't need to worry - but there are so few projects you can have high trust in that manual management isn't a big deal. Meanwhile there are many many potentially useful packages that can save you a lot of effort if you use them - but you need to manually audit each because if you don't nobody will and that will bite you.
If you only use a package manager for libraries that you have high trust in then you don't need to worry - but there are so few projects you can have high trust in that manual management isn't a big deal. Meanwhile there are many many potentially useful packages that can save you a lot of effort if you use them - but you need to manually audit each because if you don't nobody will and that will bite you.
What about the CPU microcode? Can’t trust that either. :)
I'd rather be able to update my dependencies automatically with a few commands instead of manually vendor all my dependencies, keeping up to date is really important for security. I get that game developers who only ever work on building single player games might have different opinions on "package managers", but they are in a very small niche.
One of the worst things working at companies shipping C++ was the myriad of meta-build systems that all tries to do dependency management as a part of the build system without having a separate concept of what a "package manager" is, this is truly the worst of both worlds, where people are happy to add dependencies, never update them, and never share code between projects and departments. I do not wish that way of working on my worst enemies.
Whatever problems package management brings is such a better problem to have than not having a package manager. That said I think everyone can get better at being more discriminatory of what they add to their project.
One of the worst things working at companies shipping C++ was the myriad of meta-build systems that all tries to do dependency management as a part of the build system without having a separate concept of what a "package manager" is, this is truly the worst of both worlds, where people are happy to add dependencies, never update them, and never share code between projects and departments. I do not wish that way of working on my worst enemies.
Whatever problems package management brings is such a better problem to have than not having a package manager. That said I think everyone can get better at being more discriminatory of what they add to their project.
I wouldn't say I'm a dependency maximalist but it not far off.
Yes, shared code has costs
- more general than you likely need, affecting complexity, compile times, etc
- comes with risks for today (code) and the future (governace)
But the benefits are big. My theory for one of the causes for Rust having so many good cli's is Cargo because it keeps the friction low for pulling in high quality building blocks so you can better focus on your actual problem.
Instead of resisting dependencies, I think it would be better to spend time finding ways to mitigate the costs, e.g.
- I'd love for crates.io to integrate diff.rs, provenance reporting (https://lawngno.me/blog/2024/06/10/divine-provenance.html), etc
- More direct support for security checking in cargo
- Integrating cargo-vet and/or cargo-crev into cargo
Yes, shared code has costs
- more general than you likely need, affecting complexity, compile times, etc
- comes with risks for today (code) and the future (governace)
But the benefits are big. My theory for one of the causes for Rust having so many good cli's is Cargo because it keeps the friction low for pulling in high quality building blocks so you can better focus on your actual problem.
Instead of resisting dependencies, I think it would be better to spend time finding ways to mitigate the costs, e.g.
- I'd love for crates.io to integrate diff.rs, provenance reporting (https://lawngno.me/blog/2024/06/10/divine-provenance.html), etc
- More direct support for security checking in cargo
- Integrating cargo-vet and/or cargo-crev into cargo
> The problem is that not everything needs to be automated, especially hell.
What a great quote.
What a great quote.
> People abuse cars and ram into crowds, so let's not have cars in our city!
> People abuse knives and stab people, so let's not have knives in our kitchen!
> People abuse package managers and create dependency hells, so let's not have package managers in our programming language!
No matter how you see it, this fits the definition of dumbing down;
Is this what you really want?
If that is the case, then we can shake hands and I will use a different programming language.
> People abuse knives and stab people, so let's not have knives in our kitchen!
> People abuse package managers and create dependency hells, so let's not have package managers in our programming language!
No matter how you see it, this fits the definition of dumbing down;
Is this what you really want?
If that is the case, then we can shake hands and I will use a different programming language.
> My general view is that package managers (and not the things I made distinctions about) are probably in general a net-negative for the entire programming landscape, and should be avoided if possible.
I am arguing that their "benefits" are only very short-term, if there is actually benefit for them in the first place. The strawman that you present has been repeated already and is not considering that all of those other things are actually useful and good alternatives.
I am arguing that their "benefits" are only very short-term, if there is actually benefit for them in the first place. The strawman that you present has been repeated already and is not considering that all of those other things are actually useful and good alternatives.
The obvious benefit to me are the automations, I have directly experienced these time benefits
I read you think these automations lead to more harm than good so are a net negative, and I understand that point of view
However, I think this is not a tautology
I think dependency hells and bad dependencies absolutely happen indeed, but are conditioned to badly managed programming projects
And I do not want to suffer from the dumbing down of stripping out package managers
I read you think these automations lead to more harm than good so are a net negative, and I understand that point of view
However, I think this is not a tautology
I think dependency hells and bad dependencies absolutely happen indeed, but are conditioned to badly managed programming projects
And I do not want to suffer from the dumbing down of stripping out package managers
Choose your own hell then. I'm not going to stop you.
And no, it's not a tautology, it's an empirical observation.
And as I said, not everything ought to be automated, especially hell.
And no, it's not a tautology, it's an empirical observation.
And as I said, not everything ought to be automated, especially hell.
If you've ever lived in a place with a localized car ban, it's awesome.
I'm very thankful for the Debian team's efforts to include most of my most commonly software packages in their repo. Out of all the differences between me and my colleagues workflows on MacOS and windows, this is the most impactful one. I don't remember the last time I had any kinds of dependency issues. I keep updating my packages when I log on and there are no version and/or dependency issues whatsoever.
> In real life, when you have a dependency, you are responsible for it. If the thing that is dependent on you does something wrong, like a child or business, you might end up in jail, as you are responsible for that.
Isn't this backwards? In real life, if you have a dependent, you are responsible for it. On the other hand, if you have a dependency on something, you rely on that thing, in other words it should be responsible for you. A package that is widely used in security-critical applications ought to be able to be held accountable if its failure causes harm due to downstream applications. But because that is in general impossible and most library authors would never take on the risk of making such guarantees, the risk of each dependency is taken on by the person who decides it is safe to use it, and I agree package managers sometimes make that too easy.
Isn't this backwards? In real life, if you have a dependent, you are responsible for it. On the other hand, if you have a dependency on something, you rely on that thing, in other words it should be responsible for you. A package that is widely used in security-critical applications ought to be able to be held accountable if its failure causes harm due to downstream applications. But because that is in general impossible and most library authors would never take on the risk of making such guarantees, the risk of each dependency is taken on by the person who decides it is safe to use it, and I agree package managers sometimes make that too easy.
> Each dependency is a potential liability.
I mean, sure. So what does the solution look like? From my perspective it looks like a tool that is able to update your dependencies so that you can easily pick up bug fixes in your dependencies, which sounds an awful lot like a package manager.
> JavaScript is great example of this as there are multiple different package managers for the language (npm being one of the most popular), but because each package manager defines the concept of a package differently, it results in the need for a package manager manager.
This doesn't seem like a strong point to me. Yes, there are things like yarn, pnpm, etc. But IIUC practically all npm alternatives still define packages in the same way (a package.json at the root hosted by npmjs (or your private repo)), and the differences are ergonomic/performance related.
> [that each package manager defines the concept of a package differently] is why I am saying it is evil, as it will send you to hell quicker.
Then I think it's more of a language problem, not a problem with the concept of a package manager.
I mean, sure. So what does the solution look like? From my perspective it looks like a tool that is able to update your dependencies so that you can easily pick up bug fixes in your dependencies, which sounds an awful lot like a package manager.
> JavaScript is great example of this as there are multiple different package managers for the language (npm being one of the most popular), but because each package manager defines the concept of a package differently, it results in the need for a package manager manager.
This doesn't seem like a strong point to me. Yes, there are things like yarn, pnpm, etc. But IIUC practically all npm alternatives still define packages in the same way (a package.json at the root hosted by npmjs (or your private repo)), and the differences are ergonomic/performance related.
> [that each package manager defines the concept of a package differently] is why I am saying it is evil, as it will send you to hell quicker.
Then I think it's more of a language problem, not a problem with the concept of a package manager.
> t looks like a tool that is able to update your dependencies so that you can easily pick up bug fixes in your dependencies, which sounds an awful lot like a package manager.
If only it where that easy.
Often the update isn't source compatible with the package that uses it so you can't update. There are some projects I use that I can't update because I use 6 different plugins, and each updates to the main project on a different schedule on their own terms - meaning the only version I can use is 10 years out of date and there appears no chance they will all update. (if this was critical I'd update it myself, but there are always more important things to work on so I never will in practice)
Sometimes a package will change license and you need to check the legalese before you update.
Sometimes a package is hijacked (see xv) and so you really should be doing an audit of every update you apply.
If only it where that easy.
Often the update isn't source compatible with the package that uses it so you can't update. There are some projects I use that I can't update because I use 6 different plugins, and each updates to the main project on a different schedule on their own terms - meaning the only version I can use is 10 years out of date and there appears no chance they will all update. (if this was critical I'd update it myself, but there are always more important things to work on so I never will in practice)
Sometimes a package will change license and you need to check the legalese before you update.
Sometimes a package is hijacked (see xv) and so you really should be doing an audit of every update you apply.
I agree with all of the problems that you're highlighting, but would say that all of those problems exist whether or you're doing manual dependency management or using a package manager.
The solution IMO (which is non-existent afaik) would be to integrate some kind of third party auditing service into package managers. For example, for your npm project you could add something like this to your package.json:
` "requireAuditors": [ { "name": "microsoft-scanning-service", "url": "https://npmscanner.microsoft.com/scanner/", "api_key": "yourkeyhere, default to getting it from .env" } ] `
And when you npm install, the version / hash is posted to all of your required auditor's urls. npm should refuse to install any version that hasn't been audited. You can have multiple auditing services defined, maybe some of them paid/able to scan your own internal packages, etc.
I've thought about building a PoC of this myself a couple of times because it's very much on my mind, but haven't spent any time on it and am not really positioned to advocate for such a service.
The solution IMO (which is non-existent afaik) would be to integrate some kind of third party auditing service into package managers. For example, for your npm project you could add something like this to your package.json:
` "requireAuditors": [ { "name": "microsoft-scanning-service", "url": "https://npmscanner.microsoft.com/scanner/", "api_key": "yourkeyhere, default to getting it from .env" } ] `
And when you npm install, the version / hash is posted to all of your required auditor's urls. npm should refuse to install any version that hasn't been audited. You can have multiple auditing services defined, maybe some of them paid/able to scan your own internal packages, etc.
I've thought about building a PoC of this myself a couple of times because it's very much on my mind, but haven't spent any time on it and am not really positioned to advocate for such a service.
The Boost library went the audit route, but AFAIK, few other repositories (or libraries) have done that. I believe it's a cost and lack of manpower that prevents that.
You may not have the time to audit dozens/hundreds of dependencies pulled into your projects, but there's still something you can do. For Rust/Cargo, you can run tools that check every dependency against a vulnerability list. As you have source of dependencies, you can also run static code analyzers/auditors to scan for code smells, lack of unit tests, etc.
For Java, I use the OWASP plug-in of Maven to check dependencies for security vulnerabilities. I bet other languages' package managers/build tools have similar plug-ins.
Some auditing is better than none at all. You shouldn't do no checking just because you can't full auditing!
You may not have the time to audit dozens/hundreds of dependencies pulled into your projects, but there's still something you can do. For Rust/Cargo, you can run tools that check every dependency against a vulnerability list. As you have source of dependencies, you can also run static code analyzers/auditors to scan for code smells, lack of unit tests, etc.
For Java, I use the OWASP plug-in of Maven to check dependencies for security vulnerabilities. I bet other languages' package managers/build tools have similar plug-ins.
Some auditing is better than none at all. You shouldn't do no checking just because you can't full auditing!
Yeah, yarn and co came about because npm was slow, buggy and didn't honor its own lockfile.
Nowadays it is mostly improved, and the others differentiate by enchantments to workspaces (better monorepo support) or more aggressive caching by playing games with where the installed packages physically exist on the system.
The core functionality- what a package is- has always been the same across the package managers though, because the runtime behavior is defined by node, not the package manager.
Nowadays it is mostly improved, and the others differentiate by enchantments to workspaces (better monorepo support) or more aggressive caching by playing games with where the installed packages physically exist on the system.
The core functionality- what a package is- has always been the same across the package managers though, because the runtime behavior is defined by node, not the package manager.
> So what does the solution look like?
There are no solutions, only trade-offs. And the point is that not everything needs to be, nor ought to be, automated. And package managers are a good point of this.
And yes, a language with an ill-defined concept of a package in the language itself is a problem of the language, but the package managers are not making it any better.
There are no solutions, only trade-offs. And the point is that not everything needs to be, nor ought to be, automated. And package managers are a good point of this.
And yes, a language with an ill-defined concept of a package in the language itself is a problem of the language, but the package managers are not making it any better.
> And yes, a language with an ill-defined concept of a package in the language itself is a problem of the language, but the package managers are not making it any better.
If a language does not provide a definition of a package but a package manager _does_, then I would say that that package manager did make that aspect of the problem better.
If a language does not provide a definition of a package but a package manager _does_, then I would say that that package manager did make that aspect of the problem better.
It makes it worse in my opinion, not better. Because it leads to the problem that other package managers don't agree on what a package is, and it might even lead to the need for an external build system to co-ordinate all of it too. It's a never-ending problem, and is now even worse.
And if the language does provide the definition?
"I mean, sure. So what does the solution look like? From my perspective it looks like a tool that is able to update your dependencies so that you can easily pick up bug fixes in your dependencies, which sounds an awful lot like a package manager."
Exactly! Who has the time or the discipline to do that manually?
Exactly! Who has the time or the discipline to do that manually?
> I mean, sure. So what does the solution look like?
Obviously taking on fewer such liabilities?
Obviously taking on fewer such liabilities?
The article specifically excludes this as _the_ solution to the problem:
> I am not advocating to write things from scratch.
and is clear in its target:
> That’s my general criticism: the unnecessary automation.
Yes, fewer dependencies is a solution, but it does not seem to be the author's position.
> I am not advocating to write things from scratch.
and is clear in its target:
> That’s my general criticism: the unnecessary automation.
Yes, fewer dependencies is a solution, but it does not seem to be the author's position.
That is my position... again, I am not sure how you got this conclusion from the article.
The "I am not advocating to write things from scratch" is more of a caveat to the people I know will comment NIH nonsense rather than anything productive.
But yes, my position is minimize dependencies and slow and carefully vet them too, and do not automate this process.
The "I am not advocating to write things from scratch" is more of a caveat to the people I know will comment NIH nonsense rather than anything productive.
But yes, my position is minimize dependencies and slow and carefully vet them too, and do not automate this process.
The post goes on to say that random packages are not necessarily better than what members of your team could make. At the end it gets to:
> Through manual dependency management. Regardless of the language, it is a very good idea that you know what you are depending on in your project. Copying and vendoring each package manually, and fixing the specific versions down is the most practical approach to keeping a code-base stable, reliable, and maintainable. Automated systems such as generic package managers hide the complexity and complications in a project which are much better not hidden away.
So that makes all of us human package managers. It's also true that you can get a package manager from internet folk that works better than the processes and utilities your team cobbles together to ease the burden.
> Through manual dependency management. Regardless of the language, it is a very good idea that you know what you are depending on in your project. Copying and vendoring each package manually, and fixing the specific versions down is the most practical approach to keeping a code-base stable, reliable, and maintainable. Automated systems such as generic package managers hide the complexity and complications in a project which are much better not hidden away.
So that makes all of us human package managers. It's also true that you can get a package manager from internet folk that works better than the processes and utilities your team cobbles together to ease the burden.
Yes. That's the entire point, but it should not be automated which is my point.
> SDL3 might fix it all but the time to integrate SDL3 would be the same time I could write it from scratch.
The Programmers’ Credo: we do these things not because they are easy, but because we thought they were going to be easy
The Programmers’ Credo: we do these things not because they are easy, but because we thought they were going to be easy
We've effectively written all of this already because of the amount of fixes we've had to do to the SDL2 code. So yes, we know what we are doing.
I’ve had major Nissan Altima effect with this lately. A few weeks ago I set out to make a simple C and C++ package manager that just ignores dependency hell in favor of you explicitly specifying packages. And no binaries, just build from source and use Git as a backend for managing what source maps to what builds.
Plus Lua for package recipes. It’s going really well!
Plus Lua for package recipes. It’s going really well!
>just ignores dependency hell in favor of you explicitly specifying packages
Isn't that basically manual dependency hell?
Isn't that basically manual dependency hell?
I don't find the argument very convincing. If anything you should have more time to carefully vet your dependencies when you don't need to spend time manually doing the tedious bit. Making things more difficult than they need to be just to introduce friction is a ridiculous proposition.
I don't know if this is just coincidental but this was submitted at the same time as a large NPM malware sitution was exposed. https://news.ycombinator.com/item?id=45169657
Completely coincidental. I wrote the article before the situation, AND the article is a transcription of a video recorded in July.
Package managers are constraint solvers. You could manually figure out if XYZ shared library works with somebody else's code, or, you could expect that they would label the range of shared library versions their code needs.
Note to self... don't use Odin.
Note to self... don't use Odin.
I remember when a project failed on Kickstarter due to golang dependency hell in particular.
https://news.ycombinator.com/item?id=5796597
https://news.ycombinator.com/item?id=5796597
Where's the discussion about package sets? Always comparing yourself to Javascript, Java, and friends is obviously easy
There are three points of prioritization here: you can use other peoples' code, manually vet all the code you're running, or accept that you need to trust a social network to vet stuff for you. Pick two. This is not a solvable problem.
EDIT: I've been rate limited, so the point is: unless you're Terry Davis, you're not going to be able to write software of any real complexity. Few people are going to even bother to vet the standard library, let alone the compiler, the runtime, etc etc.
EDIT: I've been rate limited, so the point is: unless you're Terry Davis, you're not going to be able to write software of any real complexity. Few people are going to even bother to vet the standard library, let alone the compiler, the runtime, etc etc.
if only people knew how easy it is to implement the standard library and make it way simpler than what is usually provided, everyone would be writing their own standard libraries; you can implement one with string manipulation, files, memory management, threading, and basic timing, in less than 1000loc of c code, as i have done before, and the biggest parts by far were console printing and filesystem stuff, and it's mostly because of windows utf-16 conversion nonsense
Use the first two, and not rely on the third at all. That's what the article is saying.
i had this idea for vendor based “package manager”
what if packages were meant to be read, and config was set inside the file directly
what if we transitioned to think of packages as templates, rather than generic black boxes
i think it would drastically reduce dependencies, package complexity, and improve understanding
what if packages were meant to be read, and config was set inside the file directly
what if we transitioned to think of packages as templates, rather than generic black boxes
i think it would drastically reduce dependencies, package complexity, and improve understanding
Honestly, he's not wrong. I use Ruby and 99% of the gems on rubygems.org are absolute trash. I use Rails and stuff like Nokogiri or Faraday, also RubyLLM, but little else because of reasons.
NPM is even worse, you import one thing and get 1000s of trash libraries so nowadays the only JS I write is vanilla and I import ES Modules manually.
Also, Odin doesn't make adding dependencies that difficult, you can literally just throw an Odin library into your project as a folder and it's available. The Odin compiler does everything else for you.
NPM is even worse, you import one thing and get 1000s of trash libraries so nowadays the only JS I write is vanilla and I import ES Modules manually.
Also, Odin doesn't make adding dependencies that difficult, you can literally just throw an Odin library into your project as a folder and it's available. The Odin compiler does everything else for you.
is sdl2 really that bad?
It's impossible to know what issues they have, since they don't specify.
But no, for the vast majority of people, SDL2 is perfectly fine, although SDL3 is a vast improvement. It's as stable and battle-tested as a cross platform multimedia library is bound to get. Opening a window and polling input is trivial.
Then again I've never even heard of the language they're using (Odin) so maybe that doesn't play well with a C library.
But no, for the vast majority of people, SDL2 is perfectly fine, although SDL3 is a vast improvement. It's as stable and battle-tested as a cross platform multimedia library is bound to get. Opening a window and polling input is trivial.
Then again I've never even heard of the language they're using (Odin) so maybe that doesn't play well with a C library.
The bugs have nothing to do with the language and exist in C too.
I'll just link to some of the bugs directly that posted as issues to SDL:
https://github.com/libsdl-org/SDL/issues/4789 (not fixed) https://github.com/libsdl-org/SDL/issues/4816 (closed) https://github.com/libsdl-org/SDL/issues/4790 (closed)
And these being the bugs we found ourselves, not other bugs that have already been found, and many marked "as not planned" since SDL2 is now finished.
I'll just link to some of the bugs directly that posted as issues to SDL:
https://github.com/libsdl-org/SDL/issues/4789 (not fixed) https://github.com/libsdl-org/SDL/issues/4816 (closed) https://github.com/libsdl-org/SDL/issues/4790 (closed)
And these being the bugs we found ourselves, not other bugs that have already been found, and many marked "as not planned" since SDL2 is now finished.
Fine, now what if you need to connect to a database, or parse a PDF, or talk to a grpc backend. What a hilariously short-sighted example.
To me, this whole article just screams inexperience.